Eot-bench: Open benchmark suite for end-of-turn detection in voice AI (github.com)

Eot-bench has been announced as an innovative open benchmark suite aimed at improving end-of-turn (EoT) detection in voice AI applications. This groundbreaking initiative addresses a longstanding challenge in voice interactions—determining when a user has finished speaking. By providing a standardized public dataset of real human-to-agent conversations in 14 languages, Eot-bench allows developers to benchmark their EoT models in a consistent environment. The evaluation focuses on genuine interaction scenarios rather than isolated audio clips, enabling more relevant assessments of how quickly and accurately AI agents can respond during pauses in conversation. The significance of Eot-bench lies in its potential to unify the fragmented landscape of EoT detection research, promoting reproducibility and progress in the field. With its comprehensive evaluation methodology that considers both false-cutoff rates and latency, Eot-bench facilitates meaningful comparisons between different voice agent models. The benchmark demonstrates that the LiveKit Turn Detector v1 outperforms its competitors across multiple languages, emphasizing the importance of balancing prompt responses with avoiding interruptions. The open availability of Eot-bench and its accompanying dataset empowers researchers and developers to refine their models, ultimately leading to more fluid and human-like interactions in voice AI systems.