Show HN: Melodjinn v0.1 – Adapting DeepMind's Genie world model for music (www.matthieulc.com)

The Melodjinn v0.1 project has emerged as an innovative approach to music generation by adapting DeepMind’s unsupervised Genie framework, traditionally used for video, to navigate the complex landscape of music. By treating music as a dynamic world characterized by discrete actions—like notes played or production decisions—the project aims to create an interactive music generation experience. The implementation utilizes two models: the Latent Action Model (LAM), which identifies meaningful transitions in audio, and a Dynamics model that generates audio in real time, allowing users to steer the music generation dynamically. This development is significant for the AI/ML community as it addresses limitations in existing music generation tools, which often struggle with real-time interaction and intuitive user control. The LAM quantizes audio patches into discrete action tokens, guiding the Dynamics model to produce coherent musical outputs that reflect user input. Though initial results showed promise, challenges remain, especially in honing the model’s ability to capture higher-level musical structures rather than micro-events. As Melodjinn progresses, it holds the potential to reshape how audiences interact with music, making generative AI an exciting frontier in the creative arts.