Researchers have created a new framework to evaluate language models for use in Germany's public sector. The framework, called M"OVE, considers three factors not typically evaluated by existing benchmarks: energy consumption, provider transparency, and knowledge of German politics. The study found that no single model excels across all these dimensions, highlighting the need for more nuanced evaluation methods. This is particularly important for public institutions, which require models that meet specific governance needs. --- Why it matters: This research matters to engineers working on language models because it highlights the limitations of existing benchmarks and emphasizes the importance of context-specific evaluations. By considering factors like energy consumption and provider transparency, researchers can develop more effective models tailored to the needs of public sector institutions. Source: https://arxiv.org/abs/2608.17827