AI

Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning

Researchers have developed a new approach to training AI models for long-context reasoning tasks. They found that traditional on-policy distillation methods can favor locally plausible responses over globally correct ones. To address this issue, they introduced Group-Calibrated On-Policy Distillation (GC-OPD), which normalizes verifier rewards and trajectory-level OPD scores within each rollout group. This approach improves performance on five long-context benchmarks, achievi
Researchers have developed a new approach to training AI models for long-context reasoning tasks. They found that traditional on-policy distillation methods can favor locally plausible responses over globally correct ones. To address this issue, they introduced Group-Calibrated On-Policy Distillation (GC-OPD), which normalizes verifier rewards and trajectory-level OPD scores within each rollout group. This approach improves performance on five long-context benchmarks, achieving higher accuracy than vanilla OPD. The code for GC-OPD is available online. --- Why it matters: This matters to AI researchers because it provides a new method for improving the performance of long-context reasoning tasks, which are critical in many applications such as question answering and text summarization. Source: https://arxiv.org/abs/2608.19181

This article was originally published at: https://arxiv.org/abs/2608.19181