C inference for Qwen3-ASR 0.6B and 1.7B transcriptions models (github.com)

A new C implementation of the inference pipeline for Qwen3-ASR speech-to-text models (0.6B and 1.7B) has been announced, featuring zero external dependencies aside from the C standard library and a BLAS implementation, allowing it to run efficiently even on low-end hardware. This implementation supports both normal and streaming transcription modes: normal processes audio in one go while streaming handles input in small 2-second chunks, optimizing for real-time speech processing. Key features include automatic language detection, a system prompt for biasing outputs, and memory-mapped weights for rapid loading. Notably, the implementation omits support for Apple's MPS to maintain broader accessibility on standard Linux servers, although it still runs well on Apple hardware. The significance of this release lies in its focus on accessibility and performance optimization for CPU-based inference, which can greatly benefit the AI/ML community, particularly in real-time applications like transcription services. The model's flexibility in processing large audio files and providing immediate token streaming enhances user experience while minimizing latency. Moreover, with the ability to handle silence skipping and manage token emission based on chunk processing, the implementation promises improved accuracy and efficiency for various transcription tasks. This opens up new possibilities for deploying speech-to-text technology in resource-constrained environments, making it a valuable tool for developers and researchers in the field.