Researchers have developed a new system called Nexus that improves the performance of large language models (LLMs). The system reduces the time it takes for LLMs to retrieve and use tools by decoupling routing from schema-prefill costs. This is achieved through an INT8 semantic lookaside buffer and a cross-encoder margin gate, which selects tools by retrieval rather than re-prefilling key/value cache. The system also includes a depth-adaptive suffix redecode that repairs attention when rotary position embedding phase drift occurs. --- Why it matters: This matters to researchers in AI because it can significantly speed up the performance of large language models, making them more efficient and effective for tasks such as text generation and question-answering. By reducing the time it takes for LLMs to retrieve tools, Nexus can improve the overall efficiency of these models. Source: https://arxiv.org/abs/2608.20397