Researchers have introduced the LZ penalty, a new method for reducing repetition in autoregressive language models without sacrificing performance. The LZ penalty is based on the codelengths used in the LZ77 compression algorithm and is designed to remove highly compressible information from the model's predictions. This approach has been shown to enable state-of-the-art open-source reasoning models to operate with greedy decoding without degenerate repetition, which can occur when a model repeats itself excessively. --- Why it matters: This matters because excessive repetition in language models can lead to inefficient use of computational resources and decreased performance. The LZ penalty provides a new tool for researchers to tackle this issue and improve the efficiency of their models. Source: https://arxiv.org/abs/2504.20131