Researchers analyzed language models to see if they still hold biases towards certain groups, even when their outputs appear neutral. They found that internal representations of user competence are influenced by demographic attributes like gender and socioeconomic status. This means that even if a model's outputs don't show bias, its underlying associations may still exist. The study used a causal framework to decompose occupational bias into two measurement points: the model's representation of user expertise and its observable outputs. --- Why it matters: This matters because it highlights potential failure modes in language models that behavioral metrics alone can't detect. Engineers and researchers need to consider how internal representations can influence downstream behavior, even if outputs appear neutral. Source: https://arxiv.org/abs/2608.20347