Researchers have proposed a new method called SOD (Step-wise On-policy Distillation) to improve the performance of small language models in tasks that require tool-integrated reasoning. The authors found that traditional on-policy distillation methods can lead to a problem where incorrect tool calls are amplified, making it difficult for the model to learn from its teacher. To address this issue, SOD adaptively adjusts the level of guidance from the teacher based on how well the student's actions align with the teacher's. Experiments show that SOD outperforms other methods by up to 20.86% and can effectively transfer agentic reasoning to lightweight models. --- Why it matters: This matters because it provides a solution for small language models to scale tool-integrated reasoning, which is essential for tasks like math, science, and code understanding. The proposed method can improve the performance of these models by up to 20.86%. Source: https://arxiv.org/abs/2605.07725