NVFP4 vs. MXFP4 Decode Benchmark (cezarcocu.com)

Recent benchmarking of the NVFP4 and MXFP4 formats on NVIDIA's B200 GPU has revealed significant performance differences in production workloads, particularly for low-batch decoding scenarios. NVFP4 outperformed MXFP4, achieving up to an 8% faster decode throughput at smaller batch sizes, primarily due to superior kernel implementations rather than just the differences in bit representation. This is particularly relevant for developers working with dense large language models like Qwen3-32B, highlighting NVFP4's advantages in specific contexts. The analysis underscored that while NVFP4 excels in small-batch throughput, the performance gap diminishes as batch sizes increase, attributed to the varying efficiency of GEMM kernel implementations for larger workloads. Notably, NVFP4 shows a significantly lower negative log-likelihood after quantization from BF16, suggesting improved evaluation performance as well. For developers optimizing AI inference applications, these findings reinforce the need for thorough benchmarking within their specific production configurations to maximize performance and efficiency.