Nvidia Nemotron 3 Super (research.nvidia.com)

Nvidia has announced the release of its Nemotron 3 Super, a cutting-edge 12 billion active and 120 billion total parameter Mixture-of-Experts hybrid Mamba-Transformer model. As the latest addition to the Nemotron 3 series, it introduces significant advancements such as LatentMoE for enhanced accuracy, MTP layers for accelerated inference through native speculative decoding, and pretraining in NVFP4 format. This model boasts impressive performance metrics, achieving up to 2.2x and 7.5x higher inference throughput than competitors GPT-OSS-120B and Qwen3.5-122B, respectively, while also supporting context lengths of up to 1 million tokens. The significance of Nemotron 3 Super lies in its potential to push the boundaries of large language models (LLMs) with higher efficiencies and accuracy across diverse benchmarks. The model's open-source availability, including pre-trained, post-trained, and quantized checkpoints, as well as specialized datasets for training, provides invaluable resources for researchers and developers in the AI/ML community. This release not only fortifies Nvidia's position in the AI domain but also invites further exploration and innovation in applications that leverage longer context lengths and advanced model architectures.