Lexical Perturbations Disrupt LLM Reasoning: An Empirical Study of Attention Diversion
Researchers studied how large language models (LLMs) perform when their input text is intentionally corrupted with keyboard noise, character swaps, or filler words. They found that these perturbations can significantly degrade the LLM's reasoning accuracy, especially on complex tasks. The study attributed this vulnerability to 'Attention Diversion', where the model pays too much attention to fragmented subwords resulting from the corruption. This phenomenon is difficult to ov
Researchers studied how large language models (LLMs) perform when their input text is intentionally corrupted with keyboard noise, character swaps, or filler words. They found that these perturbations can significantly degrade the LLM's reasoning accuracy, especially on complex tasks. The study attributed this vulnerability to 'Attention Diversion', where the model pays too much attention to fragmented subwords resulting from the corruption. This phenomenon is difficult to overcome because it affects both the content and attention allocation of the model.
---
Why it matters: This research matters to engineers working with LLMs, as it highlights a critical weakness in their robustness. Understanding how these models fail under realistic input conditions can inform the development of more resilient and reliable AI systems.
Source: https://arxiv.org/abs/2608.22140
This article was originally published at: https://arxiv.org/abs/2608.22140