Training Reasoning Models in Simulation to Natively Use Real Robots (mit-han-lab.github.io)

🤖 AI Summary
A groundbreaking study has demonstrated the successful training of a 27-billion parameter reasoning model in simulation to operate real robots, employing a novel technique called Robot-Use Reinforcement Learning (RL). This method enables the model to observe its environment, reason through actions, and command robotic movements without requiring any predefined demonstrations or additional tools. Initially, the model struggled with robot tasks, achieving only 1.6% success, but after implementation of RL, it soared to a remarkable 74.4% on the LIBERO benchmark and succeeded in 60% of real robot trials. The model's ability to generalize across unfamiliar tasks and follow instructions it was not explicitly trained on underscores its potential for broader applications in robotics. Significantly, this research highlights the promising capability of reinforcement learning to teach robots complex skills in a simulation environment that translate seamlessly to real-world applications. The findings reveal that not only can learned skills carry over to new tasks and robots, but the model retains its general reasoning and coding abilities while showing no measurable decline across various performance metrics. By releasing the model and its training infrastructure, the research aims to stimulate further exploration of sim-to-real techniques within the AI/ML community, advocating for advancements in robot-use agents that leverage foundational model post-training to enhance their effectiveness across diverse tasks.
Loading comments...
loading comments...