Researchers have developed an automatic metric evaluator called EvalCEGAR that can write its own evaluation operators. This system uses a pool of small Python operators to flag potential defects in model answers and then searches for collisions between identical scores from correct and incorrect answers. The loop refines the operators by widening their reading scope rather than resampling, allowing it to learn effective evaluators with minimal human intervention. EvalCEGAR outperformed hand-written operators on several benchmarks, including a 15-line operator that closed 15.4% of the gap between no filtering and perfect filtering on unseen tasks. --- Why it matters: This work matters because it addresses the long-standing issue of evaluating model performance in applications like report generation, where human evaluators are scarce or unreliable. The ability to automatically write effective evaluation operators could significantly reduce the burden on researchers and developers. Source: https://arxiv.org/abs/2608.18744