Cohere Transcribe hits 1.46% WER clean, 2.5x worse than claimed on real audio (inferencebench.io)

Cohere has launched its speech-to-text model, Cohere Transcribe 03-2026, boasting impressive accuracy on controlled datasets but revealing significant discrepancies in real-world scenarios. While the model achieved a low Word Error Rate (WER) of 1.46% on LibriSpeech-clean, it fell well short of Cohere's claimed 5.42% average WER across multiple datasets, hitting 13.65% overall, with particularly poor performance on AMI meeting datasets ranging from 17.74% to 29.17%. The findings suggest that the model excels in quiet environments but struggles with challenging audio conditions, such as meetings and far-field recordings. The significance of this release lies in its competitive pricing and performance against established alternatives like OpenAI's Whisper. At around $0.21 per hour for self-hosted solutions, it presents an attractive proposition for high-volume users. However, the unverified multilingual capabilities and the stark difference between reported and actual performance raise concerns for potential buyers in diverse audio contexts. The robustness of the model to non-speech audio is a commendable feature, but until the accuracy gaps are thoroughly addressed, users should approach its marketing claims cautiously, particularly for critical applications like meeting transcription.