Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model
Researchers have found that large language models (LLMs) can be faithful and useful in simulating survey respondents, but only if they use a specific attention head. The study used representational similarity analysis to examine the internal workings of an LLM and found that four properties are dissociable: being readable, faithful, and causally used. While the model's answers were initially homogeneous, using a single attention head improved fidelity up to 70% in some cases.
Researchers have found that large language models (LLMs) can be faithful and useful in simulating survey respondents, but only if they use a specific attention head. The study used representational similarity analysis to examine the internal workings of an LLM and found that four properties are dissociable: being readable, faithful, and causally used. While the model's answers were initially homogeneous, using a single attention head improved fidelity up to 70% in some cases. However, causal use did not follow fidelity, and replacing the entire identity had little effect on predictions. The study suggests that treating these properties as one claim has kept the debate about LLMs' ability to simulate populations unresolved.
---
Why it matters: This research matters because it sheds light on how large language models can be used to simulate survey respondents and improve their fidelity, which is crucial for applications in social sciences and policy-making. Understanding the dissociable properties of these models can help researchers and developers create more accurate and useful tools for population simulation.
Source: https://arxiv.org/abs/2608.18768
This article was originally published at: https://arxiv.org/abs/2608.18768