Magenta RealTime 2: Open and Local Live Music Models (magenta.withgoogle.com)

Magenta has unveiled Magenta RealTime 2 (MRT2), a cutting-edge open model that enables real-time interactive music creation directly on laptops using Apple Silicon. This model distinguishes itself from traditional offline generative music systems by allowing users to control it via MIDI, audio, and text inputs with significantly lower latency. MRT2 boasts a 2.4B parameter architecture, offering high-quality music synthesis layered with new functionalities, including the integration of playable instruments and experiences that let users clone sounds and blend styles instantly. The significance of MRT2 lies in its ability to democratize AI in music, providing musicians with practical tools that enhance creativity rather than replace human input. The model operates efficiently via a C++ inference engine, utilizing the MLX framework to achieve real-time audio generation on standard hardware. Innovations such as frame-level autoregression and continuous multi-signal control substantially reduce the model's response time to about 200 ms, making the music generation process seamless. As Magenta continues to expand its capabilities, MRT2 represents a substantial leap forward in live music AI applications, inspiring further exploration and collaboration within the music technology community.