Researchers propose a new approach to improve the alignment of large language models (LLMs) with their intended policies. They introduce Selective Importance Sampling (SIS), which 'transfers' off-policy tokens into on-policy tokens, reducing the need for importance scores and variance explosion. The method is theoretically proven to reduce the gap between token-level and sequence-level off-policy gradient estimators and can be combined with various reinforcement learning post-training algorithms. Experiments show that SIS consistently improves LLM performance across different benchmarks. --- Why it matters: This matters because it addresses a key challenge in training large language models: aligning their behavior with the intended policy. The proposed method has the potential to improve the robustness and reliability of these models, which are increasingly used in applications such as natural language processing and decision-making systems. Source: https://arxiv.org/abs/2607.04728