AI

XTC: Head-Aware Sampling by Excluding Top Choices

Researchers have developed a new decoding operator called XTC (Exclude Top Choices) for autoregressive language models. This operator targets a common regime of open-ended generation where several continuations are plausible but too much probability mass remains concentrated on the most generic choice. XTC identifies tokens whose probabilities exceed an absolute plausibility threshold and removes the dominant eligible choices with a certain probability, retaining only the wea
Researchers have developed a new decoding operator called XTC (Exclude Top Choices) for autoregressive language models. This operator targets a common regime of open-ended generation where several continuations are plausible but too much probability mass remains concentrated on the most generic choice. XTC identifies tokens whose probabilities exceed an absolute plausibility threshold and removes the dominant eligible choices with a certain probability, retaining only the weakest plausible alternative before renormalization. The authors conducted 60 experiments across various models and found that XTC improves diversity-repetition Pareto frontier, increases creative generation metrics, and preserves prompt-level strict accuracy while recovering most of the diversity gain. --- Why it matters: This matters to researchers in AI because it provides a new approach to decoding language models, which can improve the quality and diversity of generated text. The results demonstrate that XTC can be used to enhance creativity and reduce repetition in generated text. Source: https://arxiv.org/abs/2608.22758

This article was originally published at: https://arxiv.org/abs/2608.22758