Training over a thousand LoRA adapters at once (osmosis.ai)

A groundbreaking development in the AI/ML community has emerged with the introduction of a multi-adapter LoRA training path that enables the simultaneous training of over a thousand LoRA adapters. By modifying the Megatron-Bridge combined with the Miles framework, researchers have successfully implemented a system that allows multiple LoRA adapters to share a single base model as a matrix, significantly enhancing computational efficiency. In a stress test with the Qwen3.6-35B-A3B model, the team trained 1,536 LoRA adapters concurrently, achieving a training step time of under three minutes. This innovation is particularly significant as it addresses the resource inefficiencies when training various policies in parallel, which can waste valuable VRAM due to repetitive deployment of the full base model alongside individual adapters. By allowing multiple LoRA adapters to share the same base model, the new framework not only streamlines the training process but also optimizes memory usage. This approach facilitates the scaling of reinforcement learning experiments, enabling researchers to conduct thousands of experiments more effectively. The system employs advanced techniques such as grouped matrix multiplication and token-wise routing to enhance performance, marking a noteworthy advancement in the field of distributed training for reinforcement learning models.