Researchers have proposed a new approach to improve the performance of large language models (LLMs) in dense retrieval tasks. They draw inspiration from classical word-based language modeling and introduce an auxiliary task to maximize query likelihood. The model, called LLM-QL, incorporates two key components: attention blocks that prevent predictive tokens from attending to document tokens after a certain point, and document corruption, where a portion of the document's tokens are masked during prediction. This approach is shown to improve performance on in-domain and out-of-domain datasets compared to other LLM-based retrievers. --- Why it matters: This work matters because it aims to bridge the gap between language generation capabilities of LLMs and their ability to model global information, which is crucial for dense retrieval tasks. Source: https://arxiv.org/abs/2504.05216