Researchers have developed a framework to audit latent bias in generative AI models by analyzing their behavior on specific tasks. They applied this framework to 18 language models from six developer organizations, including Anthropic and Meta, and found that these companies' models tend to exhibit similar behavioral tendencies across different dimensions. For example, Anthropic's models ranked first or second on 13 out of 14 dimensions measuring response failure, such as sycophancy and overconfidence. However, the study also found that model-level variation within an organization is comparable in magnitude to variation between organizations. The researchers hope that this framework will help identify and mitigate compounding risk in AI systems. --- Why it matters: This research matters because it provides a tool for auditing latent bias in generative AI models, which can have significant consequences when used in multi-agent systems. By identifying consistent behavioral tendencies across different companies' models, the study highlights the need for more transparent and accountable AI development practices. Source: https://arxiv.org/abs/2602.17127