Researchers have developed a benchmark called HealMed to evaluate the performance of large language models in medicine across multiple languages. The benchmark contains 9,000 examples from nine languages and three task formats, and was reviewed by 23 physicians over two years. Performance declined most in low-resource languages, but proprietary models were more consistent across languages than open-source or medically specialized ones. Translation quality also affects evaluation results. --- Why it matters: This matters to researchers because it provides a standardized way to evaluate the performance of language models in medicine across different languages and cultures. It can help identify which models are most robust and reliable for use in real-world medical applications. Source: https://arxiv.org/abs/2608.19981