Researchers have proposed a new attention mechanism called STS that efficiently reduces the computational cost of processing large language models. The method, which doesn't require retraining the model, works by identifying important tokens in a smaller draft model and using those to dynamically construct a sparsity mask for the target model. This allows for significant speedups - up to 2.67x - while maintaining accuracy. STS outperforms previous techniques by enabling higher levels of sparsity for a given accuracy budget. --- Why it matters: This matters because large language models are increasingly being used in applications that require processing multi-million token sequences, such as agentic tasks. Efficient attention mechanisms like STS can help mitigate the memory and computational bottlenecks associated with these applications. Source: https://arxiv.org/abs/2605.15508