Same Weights, Different Robot: A Deployment Safety View of VLA Policies (arxiv.org)

A recent study highlights a critical safety gap in the deployment of vision-language-action (VLA) policies for robotics, revealing that identical model weights can lead to variable physical actions depending on the execution context. The authors introduce the concept of executable policy specification, which emphasizes the inclusion of learned models, action representations, and metadata-related unnormalization requirements. This approach challenges the assumption that identical checkpoints guarantee the same operational performance, underscoring the risks in safety certification processes that overlook the variability introduced during the physical execution of robotic tasks. The significance of this research lies in its quantification of action-space semantic drift—a phenomenon where minor discrepancies in metadata can drastically affect a robot's success in completing tasks. In experiments using the LIBERO-Goal and LIBERO-Spatial datasets, altering metadata led to dramatic declines in task performance, demonstrating the importance of thorough checks for action-space metadata before deploying VLA policies. By providing a closed-form mismatch transform and an ExecSpec certificate to measure drift, the study advocates for a reevaluation of deployment safety protocols, supporting a more granular approach to ensure robotic systems operate reliably and safely in real-world environments.