Researchers have proposed a new method to evaluate the accountability of AI evaluators, which are systems that make judgments or decisions. The current approach only checks if the final label is correct, but doesn't consider whether the judgment was based on valid evidence, consistent rules, or proper rule application. The new method, called counterfactual receipts, provides a way to explain how an evaluator's judgment changes over time by identifying the minimal set of source replacements that reproduce revised verdicts. This approach has been tested on a benchmark with 19,520 cases and 7,200 controls, showing that it can improve the robustness of AI evaluators. --- Why it matters: This research matters to engineers because it highlights the importance of evaluating the reasoning behind an AI evaluator's judgments, rather than just its accuracy. By doing so, they can identify potential flaws in the system and develop more trustworthy AI evaluators. Source: https://arxiv.org/abs/2608.20938