Researchers have proposed a new method for improving on-policy distillation in language models. The current approach to on-policy distillation assumes that teacher-derived rewards are a good proxy for reasoning progress, but this assumption doesn't always hold. A team of researchers has developed Reasoning-Progress-Aware Reward Filtering for On-Policy Distillation (R2-OPD), which addresses this issue by selectively suppressing supervision that conflicts with genuine reasoning progress. This approach shows consistent improvement over standard on-policy distillation, especially in terms of reasoning performance. --- Why it matters: This matters to researchers and engineers working on language models because it provides a way to improve the effectiveness of on-policy distillation, a widely used framework for fine-tuning language models. By addressing the issue of conflicting rewards, R2-OPD can lead to better reasoning performances in language models. Source: https://arxiv.org/abs/2608.19408