Researchers have introduced a new framework called EVA, which aims to evaluate the performance of voice agents in a more comprehensive and nuanced way. The current evaluation methods for voice assistants often focus on specific tasks or metrics, but may not capture the full range of human interaction with these systems. EVA takes into account multiple aspects of voice agent performance, including task-oriented dialogue, non-task-oriented conversation, and user experience. This framework is designed to provide a more accurate assessment of voice agents' capabilities and limitations. --- Why it matters: Engineers working on voice assistants will find this framework useful because it provides a more detailed understanding of how their systems are performing in real-world scenarios. EVA's evaluation metrics can help identify areas for improvement, leading to better user experiences and more effective interactions with voice agents. Source: https://huggingface.co/blog/ServiceNow-AI/eva