Show HN: I trained a 9M speech model to fix my Mandarin tones (simedw.com)

A developer has created a 9M-parameter deep learning model to enhance Mandarin pronunciation through a Computer-Assisted Pronunciation Training (CAPT) system. By utilizing approximately 300 hours of transcribed speech data, the model employs Conformer architecture trained with Connectionist Temporal Classification (CTC) loss, enabling detailed evaluation of how words are pronounced rather than just transcribing them. This innovative approach merges convolutional networks for short-term spectral features with transformers for capturing longer-range vocal patterns, crucial for mastering the tonal intricacies of Mandarin. The significance of this model lies in its potential applications in language learning, especially for individuals struggling with pronunciation due to the language's tonal nature. Unlike traditional Automatic Speech Recognition systems that might correct utterances based on context, this model focuses on providing precise feedback on pronouncing tones, marking improvements in language learning accuracy. The system is designed to run efficiently on-device, making it easily accessible through a web demo. With ongoing improvements informed by user feedback, including plans to incorporate more conversational datasets, this project exemplifies how AI can bridge gaps in language education by offering tailored, actionable pronunciation guidance.