End‑to‑End 4‑Bit Training

NVIDIA has announced a breakthrough in training large language models using end-to-end 4-bit training, significantly enhancing processing speeds while managing the associated accuracy trade-offs. By leveraging three different numerical formats—BF16, MXFP8, and a newly developed NVFP4 format—NVIDIA’s GB300 platform can perform calculations with NVFP4 up to 6 times faster and reduce memory utilization by storing values with 3.6 times fewer bits compared to BF16. This advancement is particularly beneficial for high-volume inference and tasks that require rapid computation, such as reinforcement learning and pretraining.

However, lower-precision formats introduce quantization errors that can affect model performance. To mitigate this, NVIDIA optimized the training strategy by implementing NVFP4 precision in the expert layers for forward passes and MXFP8 for backward passes, utilizing techniques like stochastic rounding to minimize bias. This tailored approach strikes a balance between speed and accuracy, enabling the training of Inkling 2 Small, a model that benefits from a simplified training process and measurable efficiency improvements—specifically, an 8% speedup in pretraining and a 20% boost in reinforcement learning tasks. This innovation represents a significant step toward maximizing the efficiency of large-scale AI model training while maintaining performance integrity.