LoongForge End-to-End Optimization for GR00T N1.6 Delivers 2.3× Throughput (baidu-baige.github.io)

Baidu Baige's LoongForge has unveiled significant end-to-end system-level optimizations for the GR00T N1.6 Vision-Language-Action (VLA) model, achieving a remarkable 2.3× increase in training throughput and reducing the training cycle by 56.6%. The GR00T N1.6 model is pivotal for the development of humanoid robots, merging perception, understanding, and action, but its training has been hampered by IO stalls and communication overhead. LoongForge addresses these challenges through a series of sophisticated engineering optimizations that streamline data management, enhance communication efficiency, and refine training scheduling. The optimization process focuses on three core areas: an asynchronous data IO pipeline that reduces GPU idling, a fine-grained communication-computation overlap facilitated by the Megatron Distributed Optimizer that allows for early parameter fetching, and an advanced micro-batching strategy using CUDA Graph to minimize scheduling overhead. By transforming GPU utilization from intermittent waiting to continuous operation, these improvements significantly hasten the development cycle for researchers in the AI/ML community, paving the way for more efficient VLA training and quicker model iterations without altering the underlying architecture. This is a substantial leap in supporting the rapid evolution of embodied intelligence technology.