Show HN: Magenta Real-Time Music Generation Locally on iPhone, Without the GPU (github.com)

Google DeepMind has revealed an impressive achievement with Magenta Real-Time Music Generation, allowing for live music creation directly on iPhone devices without relying on the GPU. The modified Magenta RealTime 2 model runs at 25 frames per second, generating 48 kHz stereo sound continuously for up to ten minutes on a 2020 iPhone 12 Pro without any dropouts or heating issues. This was accomplished by partitioning the model into three Core ML graphs optimized for the device's Neural Engine, which is essential for handling the demanding processing under thermal constraints. This development is significant for the AI/ML community as it demonstrates the potential for sophisticated machine learning applications to operate efficiently on mobile devices without the need for more power-hungry GPU resources. Key technical innovations include the use of a stateful temporal transformer with a near-instantaneous response time (14 ms under a 40 ms budget), deterministic output token generation, and a unique design that strategically routes processing to different components (CPU vs. Neural Engine) based on their strengths. The project's commitment to transparency in its data and performance metrics further enhances its credibility and invites further exploration of real-time AI applications on consumer hardware.