Show HN: Claude optimizes mel-band roformer to run 4x faster on Mac (github.com)

A new development in AI audio processing has emerged with the Claude project, which optimizes the Mel-Band RoFormer source separation model to achieve approximately four times real-time performance on an 8 GB M3 MacBook Air. This model utilizes the Neural Engine and GPU, demonstrating impressive efficiency without altering the unmodified public checkpoint of 228 million parameters. It significantly reduces processing time for audio clips—transforming a 45-minute TV episode's separation time from 42 minutes to just 11. This advancement is noteworthy for the AI/ML community as it showcases the potential of leveraging Apple Silicon's architecture for machine learning tasks, particularly in audio applications. Key technical innovations include moving rotary embeddings to run in fp32 and optimizing the model by utilizing custom Metal kernels, which collectively enhance processing speeds. Additionally, the findings reveal that quantization would not notably boost performance, as the majority of the weights reside in mask estimators that contribute minimally to processing time. This work not only benefits individual Mac users but also offers insights and optimizations that could inform future model developments in efficient AI-based audio processing.