Evaluating Multimodal LLMs across Text and Audio Modalities for Accessible Disaster Assistance
Researchers evaluated four state-of-the-art multimodal large language models (MM-LLMs) on their ability to provide consistent responses across text and audio modalities. The study found that none of the models achieved reliable consistency across modalities, with performance gaps being more pronounced for individuals with access needs, such as those who are hard of hearing or have dementia. This modality-dependent inequity undermines the humanitarian value of these systems.
Researchers evaluated four state-of-the-art multimodal large language models (MM-LLMs) on their ability to provide consistent responses across text and audio modalities. The study found that none of the models achieved reliable consistency across modalities, with performance gaps being more pronounced for individuals with access needs, such as those who are hard of hearing or have dementia. This modality-dependent inequity undermines the humanitarian value of these systems.
---
Why it matters: This research matters because it highlights a critical limitation in current AI tools for disaster risk communication, which can exacerbate existing inequalities. Engineers and researchers working on AI-powered emergency infrastructure need to address this issue to ensure that their systems are equitable and trustworthy.
Source: https://arxiv.org/abs/2608.14651
This article was originally published at: https://arxiv.org/abs/2608.14651