Entity tracking emerges in sub-billion parameter language models and exceeds human performance in naturalistic narratives
Researchers evaluated how well both humans and large language models can track entities in naturalistic narratives. They found that human performance degrades with increasing narrative complexity, but not length. In contrast, large language models improve at tracking entities as they get larger, and even surpass human performance. This suggests that entity tracking, a key aspect of understanding language, is more easily achieved by large language models than previously though
Researchers evaluated how well both humans and large language models can track entities in naturalistic narratives. They found that human performance degrades with increasing narrative complexity, but not length. In contrast, large language models improve at tracking entities as they get larger, and even surpass human performance. This suggests that entity tracking, a key aspect of understanding language, is more easily achieved by large language models than previously thought.
---
Why it matters: This matters because it shows that large language models can perform complex tasks like entity tracking with ease, which could have significant implications for applications such as question answering and text summarization. It also highlights the need to re-evaluate how we evaluate these models in comparison to human performance.
Source: https://arxiv.org/abs/2608.18083
This article was originally published at: https://arxiv.org/abs/2608.18083