OpenWAM: An Open Framework for Composable World-Action Models (openwam.stanford.edu)

OpenWAM, a new framework for composable world-action models, has been introduced to address the complexities of integrating prediction and control in robotics. This framework provides a unified platform that allows researchers to systematically compare different approaches to video and action generation, enabling the composition of independently trained components. The architecture leverages a Mixture-of-Transformers (MoT) approach, incorporating both a 5B video predictor and a 2B action expert, facilitating innovative interaction programs that can generate video and actions in various sequences, such as video-then-action or action-then-video. The significance of OpenWAM lies in its ability to improve robot learning through adaptive pretraining on extensive datasets of real and synthetic manipulations. With over 3.34 million trajectories and advanced mechanisms like causal attention, OpenWAM establishes a robust foundation for enhanced robot decision-making. The framework also introduces a local-context interface for dynamic modeling, allowing robots to predict actions based on supplied future trajectories without pre-start history, ultimately improving performance in complex tasks. This approach not only fine-tunes the learning process but also enhances the transferability of models across different tasks, driving forward the capabilities of AI in robotics.