Researchers have created DiagFlowBench, a dataset designed to test how language models handle unexpected inputs in diagnostic conversations. The dataset consists of 50 industrial flowcharts converted into multi-turn conversations that include both compliant and out-of-scope utterances. This is important because current benchmarks often don't prioritize handling such inputs, which can lead to hallucinations or providing incorrect but plausible advice. --- Why it matters: This matters for engineers working on language models used in maintenance operations, as it highlights a vulnerability in grounding systems that can lead to poor performance when faced with unexpected inputs. By evaluating the robustness of these systems, researchers can improve their ability to handle real-world scenarios. Source: https://arxiv.org/abs/2606.17904