Researchers have identified a problem with large language models (LLMs) called factual access failure. This occurs when LLMs can recognize the correct answer to a question but fail to produce it in a closed-book generation task. The authors propose a solution called Recall-Anchored Distillation (RAD), which preserves out-of-distribution generation behavior by aligning the adapted model with its original base model's soft continuation distribution on unlabeled text. RAD requires no additional labeled data or external judges and is shown to recover lost recall in three different backbones fine-tuned on MedMCQA. --- Why it matters: This matters because it provides a solution to a specific problem that can occur when adapting LLMs to new domains, which is crucial for their deployment in real-world applications. By preserving out-of-distribution generation behavior, RAD helps ensure that LLMs maintain their ability to recall facts even after being fine-tuned on a target domain. Source: https://arxiv.org/abs/2608.20794