Researchers have introduced InsufficiencyBench, a new benchmark for evaluating the performance of large language models (LLMs) in providing legal advice on underspecified user queries. Unlike existing benchmarks, which assume users provide all necessary information, InsufficiencyBench tests whether LLMs can recognize when a query lacks crucial details and refrain from making premature conclusions. The authors created 202 benchmark items covering six legal domains and 24 US jurisdictions, annotated by practicing attorneys. Evaluations of ten frontier models found that none performed well in identifying missing elements or providing qualified responses to deficient queries. --- Why it matters: This matters because it highlights the limitations of current LLMs in handling real-world scenarios where users often omit crucial information. Improving performance on InsufficiencyBench could lead to more accurate and reliable legal advice from AI systems. Source: https://arxiv.org/abs/2608.20220