Show HN: The First Open Source Diffusion ASR Audio Model 15x Faster Than Whisper (arxiv.org)

A groundbreaking advancement in automatic speech recognition (ASR) has been announced with the introduction of the first open-source diffusion ASR model, which operates 15 times faster than the well-known Whisper model. Developed by Harsha Vardhan Khurdula, this audio-native interface, called DiffusionGemma, utilizes a unique discrete diffusion language model that simultaneously generates entire transcripts in just a few denoising steps. The architecture features a 26B mixture-of-experts design, employing a frozen Whisper encoder for acoustic feature extraction and low-rank adapters to integrate the audio data effectively. This innovation is significant for the AI/ML community as it challenges the traditional autoregressive decoding paradigms that dominate ASR systems, potentially leading to much faster and more efficient speech-to-text applications. With a reported word error rate of 6.6% on the LibriSpeech test-clean dataset, and the capability to handle multiple languages (e.g., English, Hindi, and Mandarin) with a single trained adapter, the model exemplifies a shift towards parallel processing in speech recognition. By successfully addressing gradient flow issues during training through a novel connectionist temporal classification loss, this model paves the way for more powerful and nimble ASR solutions that could enhance user experiences across various applications.