AA-Robotics v0.1 (artificialanalysis.ai)

AA-Robotics has launched v0.1, a research preview aimed at evaluating the ability of frontier language models to understand, plan, and execute physical tasks via a 6-axis robot arm. In this initial phase, models such as GPT-6 Astra and others attempted eight tasks, ranging from pressing buttons to measuring objects. The results show that while these models can achieve task completion, they are significantly slower and more expensive than human performance, taking approximately 30 minutes per task compared to an average of 20 seconds for a human. The scores reflect their ability to manipulate objects effectively, with GPT-6 Astra leading with a score of 69. The significance of AA-Robotics v0.1 lies in its demonstration of progress in embodied AI, as language models begin to bridge language understanding with physical action. This research not only illustrates the potential for generalist agents in robotics but also highlights the current limitations in speed and efficiency that need to be overcome before these models can be considered for production. With advancements anticipated, this project could pave the way for more capable robots capable of performing complex tasks autonomously, ultimately driving the robotics field forward.