Show HN: Classify mechanical faults using Contrastive Language-Audio Pretraining (github.com)

A new project dubbed "cardiag" has emerged, presenting an innovative end-to-end audio machine learning pipeline designed to classify mechanical faults from recordings. By scraping fault-sound clips from platforms like YouTube and TikTok, cardiag isolates mechanical sounds from background noise and embeddings are generated using a frozen Contrastive Language-Audio Pretraining (CLAP) model. The system functions as both a command-line interface and a live web application, providing triage assistance rather than definitive diagnoses. It informs users whether an issue is present, identifies the general area in the vehicle, and ranks potential faulty parts. The model is notably designed to express uncertainty when audio data is insufficient. For the AI/ML community, cardiag represents a significant step towards developing reliable audio-based diagnostic tools, particularly given the inherent challenges of diagnosing through low-quality phone recordings. The approach highlights the use of calibrated training and robust audio cleaning techniques, achieving an AUROC of 0.79 for fault detection in noisy environments, while reaching 0.93 AUROC in cleaner conditions. The project's methodology, adaptable for other audio datasets, warns of the system's limitations, emphasizing its role as a triage tool that offers transparency in its uncertainty, thereby laying groundwork for future advancements in audio diagnostics.