Researchers have found that self-distillation can sometimes degrade the performance of large language models (LLMs). In mathematical reasoning tasks, self-distillation reduces response length but also harms performance. The study attributes this degradation to the suppression of epistemic verbalization - the model's expression of uncertainty during reasoning. Exposing appropriate levels of uncertainty is crucial for robust reasoning, and optimizing reasoning behavior beyond mere correct answer traces is essential. --- Why it matters: This matters because it highlights a potential pitfall in using self-distillation as a post-training paradigm for LLMs, which are increasingly used in real-world applications. The findings suggest that models may not always benefit from self-distillation, especially when dealing with unseen problems. Source: https://arxiv.org/abs/2603.24472