Listening or Reading? Evaluating Speech Awareness in Chain-of-Thought Speech-to-Text Translation
Researchers have evaluated the effectiveness of Chain-of-Thought (CoT) speech-to-text translation systems in using both spoken words and their transcriptions to improve accuracy. They found that CoT systems primarily rely on transcripts rather than leveraging speech, which challenges previous assumptions about their advantages. To address this issue, the authors suggest simple training interventions such as incorporating direct speech-to-text data or injecting noisy transcrip
Researchers have evaluated the effectiveness of Chain-of-Thought (CoT) speech-to-text translation systems in using both spoken words and their transcriptions to improve accuracy. They found that CoT systems primarily rely on transcripts rather than leveraging speech, which challenges previous assumptions about their advantages. To address this issue, the authors suggest simple training interventions such as incorporating direct speech-to-text data or injecting noisy transcripts.
---
Why it matters: This study is important for AI researchers because it highlights the limitations of current CoT systems and suggests that new architectures are needed to effectively integrate acoustic information into translation. This could lead to improved accuracy in speech-to-text applications.
Source: https://arxiv.org/abs/2510.03115
This article was originally published at: https://arxiv.org/abs/2510.03115