Researchers have proposed a new framework for using large language models (LLMs) as judges in evaluating model outputs. The framework addresses the challenge of objective tasks by controlling the risk of accepted verdicts and guaranteeing their reliability at a specified level. It uses uncertainty thresholds to route instances to either a parametric mode or a retrieval-augmented mode, depending on the confidence of the LLM. This approach maintains a target error rate while achieving higher coverage than single-mode baselines. --- Why it matters: This matters because it provides a way to control the risk and reliability of LLM judging in objective tasks, which is essential for applications where accuracy and trustworthiness are critical, such as in high-stakes decision-making or safety-critical systems. Source: https://arxiv.org/abs/2608.17994