Shipping local speech-to-text inside a Tauri app (writewithset.com)

A new offline notes application called "Set" has integrated a local speech-to-text feature utilizing OpenAI's Whisper model through a C/C++ port known as whisper.cpp. This innovative app retains all notes as Markdown files on the user's device, ensuring privacy and reducing latency. The implementation of speech recognition serves not just as a transcription tool; it allows users to capture their thoughts in real-time, facilitating a more organic writing process. The app ships with a 5-bit quantized version of the Whisper small model, which boasts an impressive word error rate of 3.9% on clean speech, making it a reliable choice for dictation. For the AI/ML community, this application represents a significant leap towards efficient, on-device processing of speech recognition, particularly suitable for environments without guaranteed internet access. The lightweight design requires only a 190 MB download and maxes out memory usage at around 350 MB on an M1 MacBook Pro, demonstrating promising performance across multiple operating systems. Notably, it manages to compete with larger models in terms of accuracy while offering substantial resource savings, making it an appealing option for developers and users focused on compact, privacy-respecting AI solutions. Future enhancements are planned, including support for streaming transcription and additional language benchmarks.