The Artificial Analysis Speech to Speech Index (artificialanalysis.ai)

The Artificial Analysis Speech to Speech Index was unveiled, establishing a comprehensive metric for evaluating native Speech to Speech models. This innovative index combines three key assessment areas: Speech Reasoning, Conversational Dynamics, and Agentic Performance, using datasets like Big Bench Audio, Full Duplex Bench, and 𝜏-Voice. Model performance is benchmarked, with OpenAI's GPT-Realtime-2 leading the pack at 77.2%, followed closely by other major players like @xAI Grok Voice Think Fast 1.0 at 75.7%. Notably, GPT-Realtime-2 excels in handling conversational dynamics while Grok Voice leads in agentic performance metrics, indicating a divergence in capabilities among top models. In addition to quality assessments, the index highlights key performance metrics such as speed and cost, where Deepslate Opal stands out for its rapid response time of 0.44 seconds, making it the fastest model evaluated. Cost-wise, the Gemini 3.1 Flash Live Preview (Minimal) model is the most affordable at $1.50 per hour of input audio. This announcement signifies a pivotal advancement for the AI/ML community, as it provides crucial insights into model capabilities, helping developers and researchers identify strengths and weaknesses in Speech to Speech technologies, and fostering ongoing innovation in this rapidly evolving field.