Researchers have developed a benchmark to evaluate the investment logic of large language models. The benchmark, called extsc{InvestLogicBench}, contains real-world decisions from 151 investors and assesses how well these models can reason about investments. The results show that while the models are good at making profitable decisions, they often lack grounding in the underlying events and reasoning. This highlights a limitation of current evaluation methods, which focus on outcome-only metrics rather than the decision-making process itself. The authors argue that this benchmark is not only relevant to finance but also has broader implications for personalized, consequential agents. --- Why it matters: This research matters because it exposes a weakness in the way large language models are currently evaluated. By focusing solely on outcomes, these models can appear more competent than they actually are. This has significant implications for applications that require decision-making under uncertainty, such as finance, healthcare, and transportation. Source: https://arxiv.org/abs/2608.06108