Researchers have developed a benchmark called MecEng to evaluate the capabilities of Large Language Models (LLMs) in mechanical engineering and spatial geometry. The benchmark consists of 84 tasks that require LLMs to create simulation models from text descriptions. The results show that current LLMs perform well on simple tasks, but struggle with more complex ones. The study also explores how different factors such as model size and release date affect the performance of LLMs in mechanical engineering tasks. --- Why it matters: This research is important for engineers who work with LLMs to generate simulation models, as it provides a systematic evaluation of their capabilities and limitations. Understanding these limitations can help improve the development of more accurate and reliable LLMs for mechanical engineering applications. Source: https://arxiv.org/abs/2608.14615