Accelerating Gemma 4: faster inference with multi-token prediction drafters (blog.google)

The AI community has taken a notable leap forward with the introduction of Multi-Token Prediction (MTP) drafters for the Gemma 4 model family. Just weeks after the successful launch of Gemma 4, which has seen over 60 million downloads, this new technology promises to enhance inference speed by up to 3x without compromising output quality. The MTP drafters leverage a speculative decoding architecture that allows multiple future tokens to be predicted in parallel, significantly mitigating the memory-bandwidth limitations that traditionally cause latency bottlenecks in large language model (LLM) inference. This advancement is especially impactful for developers, as it drastically improves the performance of applications requiring rapid responses, including coding assistants and mobile applications. By enabling faster token generation and verification, the MTP drafters allow developers to deploy complex AI models on consumer-grade hardware with minimal delay, enhancing user experience and battery life. The technical innovations not only optimize the utilization of existing compute resources but also introduce architectural enhancements that facilitate shared cache usage between models. This promises a brighter future for AI applications, unlocking new possibilities in real-time interaction and complex task execution.