Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalization
Researchers have developed a new post-training framework called IAR (Inject, Align, and Recover) to improve large language models' ability to answer questions about a bounded document collection without retrieving the source documents. The framework consists of three stages: injecting structured document knowledge into the model, aligning the injected model with answer-only QA supervision, and recovering general capabilities by merging it with the base instruction model. Expe
Researchers have developed a new post-training framework called IAR (Inject, Align, and Recover) to improve large language models' ability to answer questions about a bounded document collection without retrieving the source documents. The framework consists of three stages: injecting structured document knowledge into the model, aligning the injected model with answer-only QA supervision, and recovering general capabilities by merging it with the base instruction model. Experiments show that IAR outperforms other methods in 7 out of 8 dataset-model settings, with average gains of 3.6 percentage points in domain QA accuracy and 12.1 percentage points in mean general performance.
---
Why it matters: This matters to researchers in AI because it addresses a key challenge in large language models: how to internalize document knowledge without relying on retrieval at inference time. IAR's success has implications for applications where access to the source documents is limited or not feasible, such as in low-resource settings or when dealing with sensitive information.
Source: https://arxiv.org/abs/2608.20281
This article was originally published at: https://arxiv.org/abs/2608.20281