Researchers have developed a new framework called ESTP (Entropy-and-Semantic Token Pooling) to improve the accuracy of predicting output lengths in large language models. Existing methods rely on token-wise entropy, but this can lead to important tokens being underweighted and less informative tokens receiving too much emphasis. ESTP addresses this issue by combining entropy with attention-based importance scores derived from self-attention weights computed during the prefill phase. This allows ESTP to capture both uncertainty and semantic importance with minimal additional computation. The framework outperforms baseline methods on the ForeLen benchmark, achieving better prediction accuracy and lower error rates in most scenarios. --- Why it matters: This matters because predicting output lengths accurately is crucial for efficient large language model serving, which can save compute resources and improve throughput. By improving length prediction, ESTP can help reduce padding overhead and enhance overall performance in applications like long-context reasoning and reinforcement learning. Source: https://arxiv.org/abs/2608.15592