GR00T 2: new World-Action Model (WAM) from Nvidia (research.nvidia.com)

Nvidia has announced GR00T 2, a groundbreaking foundation world-action model (WAM) designed to anticipate future actions of robots in conjunction with video predictions, marking a significant advancement in the AI/ML community. This model can learn physical tasks from human demonstrations, even when those tasks were never directly shown to the robot. For instance, utilizing multi-fingered robotic hands, GR00T 2 successfully completed complex tasks like assembling a toy truck and preparing breakfast, achieving impressive success rates even with minimal training data. Its ability to adapt and generalize to new tasks without prior robotic experience—including actions performed solely in human video—demonstrates a leap in how AI can learn from human interaction. The technical innovations in GR00T 2 include a diffusion transformer that predicts the interplay between video and robot actions, which enables robust zero-shot generalization across 50 tasks, achieving a 40% success rate initially and improving to 95% with just a few demonstrations. This model also features agentic control, allowing a reasoning agent to manage task planning and interaction with the robot through an API, which enhances task execution reliability. GR00T 2 Turbo, a faster variant, can even run on smaller robotic platforms, suggesting broad applicability in various robotics domains and potential for self-improvement through reinforcement learning. The implications of GR00T 2 extend well beyond mere task execution, as it paves the way for advanced human-robot collaboration and more intuitive robotic systems.