Offline streaming speech recognition on iOS with Nvidia Nemotron 3.5 and Core ML (github.com)

A Proof of Concept (PoC) for offline streaming speech recognition on iOS has been developed using NVIDIA's Nemotron-3.5-ASR model alongside Core ML, enabling real-time transcriptions on devices like the iPhone 15 Pro. The system supports both live microphone streaming and offline file transcription, utilizing the full capabilities of Apple's hardware for optimized performance. Users can run the application in Xcode 16+ with iOS 17 or later, requiring specific permission for microphone access and separate downloads for model files. This advancement is significant for the AI/ML community as it demonstrates the potential for robust, on-device speech recognition that functions without an internet connection, which is crucial for privacy and efficiency in mobile applications. The application processes audio at 16 kHz and applies a tier-aligned chunk buffering strategy for real-time transcription, while also supporting multiple languages. Performance is recorded through benchmark instrumentation, providing insights into processing latency and efficiency, although full transcription accuracy is yet to be evaluated. The PoC highlights the growing capabilities of AI/ML in mobile environments and paves the way for future enhancements in natural language processing and real-time communication applications.