Gemma 4 Technical Report [pdf] (arxiv.org)

The Gemma 4 Technical Report reveals the latest advancements in the Gemma model family, showcasing a new generation of open-weight, multimodal language models designed to enhance compute efficiency and reasoning capabilities. This model suite offers varying architectures, including dense and Mixture-of-Experts options, with parameter counts ranging from 2.3B to 31B. Notably, Gemma 4 introduces an innovative encoder-free architecture for its 12B model, allowing it to process raw audio and image data directly—a significant shift aimed at broadening the model's applicability. The significance of Gemma 4 lies in its performance improvements across a range of benchmarks, particularly in STEM and multimodal tasks, positioning it as a competitive alternative to larger, state-of-the-art models. Its ability to generate reasoning traces, paired with enhanced efficiency in inference speed and memory usage, marks a critical advancement for the AI/ML community. By addressing long-context capabilities and facilitating better interaction with data from varied inputs, Gemma 4 establishes a new standard for the next generation of language models, pushing the boundaries of what multimodal AI can achieve.