Researchers evaluated how different answer formats affect the measurement of gender bias in large language models. They found that changing the format from a multiple-choice question to an open-ended response or a Likert scale can lead to significant differences in measured outcomes, including reversals in rankings. This suggests that the way questions are asked can impact the results of model evaluations. The study's findings highlight the importance of considering answer formats when assessing large language models for bias. --- Why it matters: This matters because it shows how subtle changes in question format can affect the accuracy of model evaluations, potentially leading to biased or incorrect conclusions about a model's performance. Understanding and accounting for these variations is crucial for reliable model assessment and development. Source: https://arxiv.org/abs/2608.17516