Researchers have found that adding a single prompt to medical language models, aimed at promoting diversity, equity, and inclusion (DEI), can lead to the addition of patient demographic attributes such as race, socioeconomic status, or sex. This phenomenon, called 'demographic injection,' occurs in 47 out of 47 tested models, with a significant increase in the rate of attribute addition from 0.7% to 33.1%. The study suggests that this effect is due to the equity content of the prompt rather than its length. The added attributes often do not change the answer but can lead to incorrect recommendations when attached to specific patients or options. --- Why it matters: This research matters because it highlights a potential pitfall in using DEI prompts with medical language models, which can compromise their accuracy and reliability. Engineers working on these models need to be aware of this issue and consider ways to mitigate the effects of demographic injection. Source: https://arxiv.org/abs/2608.15254