Nvidia Puts a 4B World Model on the Robot [Weekly Physical AI Roundup] (topicqueue.substack.com)

NVIDIA has unveiled Cosmos 3 Edge, a groundbreaking 4 billion-parameter world foundation model that can operate on-device, specifically designed for use in robotics. Typically, world models are too resource-intensive for real-time control and require cloud computing. However, Cosmos 3 Edge employs a Mixture-of-Transformers architecture, combining an autoregressive reasoner with a diffusion generator, enabling it to run at approximately 15 Hz on a Jetson Thor platform. This development is significant for the AI/ML community as it allows for real-time decision-making across various robotic applications, including humanoid movements and machine manipulation, without the latency associated with cloud processing. The open availability of its weights and post-training scripts encourages adaptation and experimentation within the research community. This release raises important questions about the efficiency of smaller models for direct policy execution, traditionally the domain of larger, server-based models. As the field shifts toward more compact and efficient AI tools, Cosmos 3 Edge could influence the design of future robotics systems, making advanced functionalities accessible on more widely-available hardware. The implications extend to broadening the usability of AI in real-world robotics, as researchers can now locally deploy sophisticated models without the need for extensive external resources.