PAVO-Bench – 50K voice turns and an 85K-param router for ASR→LLM→TTS (github.com)

Researchers at the University of Pennsylvania and Google have announced PAVO-Bench, a new framework designed for optimizing voice-assistant pipelines in real-time applications by utilizing a demand-conditioned inference routing system. This innovative architecture employs an 85,041-parameter meta-controller that leverages multi-objective Proximal Policy Optimization (PPO) to efficiently direct automatic speech recognition (ASR), large language model (LLM), and text-to-speech (TTS) calls to either cloud or edge configurations. The framework was rigorously tested across 50,000 voice turns on NVIDIA A100 and H100 GPUs, as well as Apple M3 devices. The significance of PAVO-Bench lies in its ability to address the inter-stage dependencies traditionally overlooked in voice-stack systems. The empirical findings reveal that choices made during the ASR stage critically impact the LLM's performance, establishing quality constraints that the new meta-controller can navigate. By dynamically routing requests, the system achieves remarkable improvements, including a 34% reduction in median latency and a 71% decrease in energy consumption per turn. Additionally, it significantly lowers the coherence-failure rate from 7.1% to 0.9%, showcasing a more efficient and effective use of resources. This represents a pivotal advance for developers in the AI/ML community, promising enhancements in real-time voice applications and paving the way for more innovative deployments in the future.