Offline streaming speech recognition on iOS with Nvidia Nemotron 3.5 and Core ML (github.com)

NVIDIA has announced a proof of concept (PoC) for on-device, offline, streaming speech recognition on iOS using its Nemotron-3.5-ASR Streaming model via CoreML. This innovative integration allows real-time transcription from live microphone input as well as the transcription of pre-recorded audio files on iPhone and iPad devices. The implementation has been verified on physical hardware, requiring iOS 17 and Xcode 16+ to function, and is designed to provide seamless multilingual support with a model capable of processing inputs in various languages, including English, Mandarin, Japanese, and Korean. This development is significant for the AI/ML community as it demonstrates the growing capabilities of on-device machine learning, enabling fast, real-time speech recognition without relying on cloud processing. The detailed pipeline includes sophisticated audio processing features such as a 16 kHz mono sample rate and an architecture that retains essential state information for seamless transcription across audio chunks. By leveraging Core ML and the Apple Neural Engine (ANE), this approach marks a significant advancement in making advanced AI features more accessible and efficient on mobile platforms, paving the way for further innovations in real-time AI applications.