Can a 2.8T Model Run on a Single Node of Nvidia B300 X8? (blog.us.fixstars.com)

On July 27, 2026, Moonshot AI launched Kimi-K3, a groundbreaking 2.8 trillion parameter MoE (Mixture of Experts) model, touted as the largest open-weight model ever. In an impressive feat, Fixstars successfully deployed Kimi-K3 on a single-node NVIDIA B300 x8 configuration on the same day, marking a significant advancement in model accessibility and deployment capabilities. The model's architecture utilizes innovative techniques like Kimi Delta Attention and quantization-aware training, ensuring it operates efficiently even with its massive scale. Kimi-K3's launch is significant for the AI/ML community as it sets a new performance benchmark, closely rivaling proprietary models while remaining open access. The deployment revealed that despite the model's vast number of parameters, it achieved effective memory management, fitting within the NVIDIA B300 node’s capabilities. Benchmarks indicated impressive throughput and low latency, with practical limits for concurrent processing identified at around 30 requests. Kimi-K3 demonstrated notable performance in coding tasks, showcasing its ability to generate high-quality outputs efficiently, thus paving the way for future advancements and applications in AI-driven development and multimodal capabilities.