Power Up FSDP2 as a Flexible Training Back End for Miles (lmsys.org)

The SGLang RL Team and the Miles community have unveiled FSDP2, a new flexible training backend for the Miles framework, enhancing reinforcement learning (RL) capabilities. With this upgraded architecture, FSDP2 supports innovative models like Qwen3-Next and aligns with the Megatron framework, providing significant advantages in training stability and acceleration. This shift from traditional data replication to sharding allows for a more efficient distribution of model weights across multiple GPUs, thereby reducing the maintenance cost and facilitating the adaptation of complex architectures. The importance of FSDP2 lies in its ability to streamline the training process, enabling seamless integration with existing ecosystems such as HuggingFace and simplifying the development environment. Enhanced features include improved data packing strategies to mitigate computational waste and strict training-inference consistency to ensure numerical alignment in outputs. This not only accelerates model training but also optimizes performance, thereby addressing common challenges in RL workflows, including training-inference mismatches. Overall, this announcement marks an important milestone for developers in the AI/ML community by enhancing usability and flexibility in training large-scale models.