The Ability of Frontier Models to Control Robots Is Rapidly Improving (www.lesswrong.com)

Recent advancements have been made in the ability of frontier AI models to control robotic systems, particularly in manipulating tasks essential for scientific research. Progress was evaluated across nine multimodal language models (LMs) controlling a bimanual robotic platform, showcasing their improving proficiency with manipulators. While the capacity for locomotion remains limited, emerging models like Opus 4.8 and Gemini Argon 4 demonstrate enhanced skills in piloting quadrupeds. The study found that despite improvements, successful task completions are still infrequent, with top-performing models achieving only 45-48% in task scores, indicating that many tasks result in partial rather than full success. This development is significant for the AI and machine learning community as it highlights the rapid improvement in robotic control, suggesting a doubling time for manipulation task performance of approximately 54 days. The research also raises concerns regarding potential misuse, as enhanced manipulation capabilities could allow individuals with malicious intent easier access to dangerous activities. The study is grounded in solid evaluation metrics that include task performance, execution rates, and inference costs, underscoring a competitive landscape where different models exhibit trade-offs between speed, efficiency, and operational costs.