Computer Use Is 45x More Expensive Than Structured APIs (reflex.dev)

A recent benchmark study revealed that using vision agents to operate admin panels is 45 times more expensive than leveraging structured APIs. The comparison involved two AI agents—the vision agent, which interacts with the UI through screenshots and clicks, and the API agent, which communicates directly with the application's HTTP endpoints. While both agents performed the same task of managing customer orders and reviews, the API agent completed it in just 8 calls and significantly less time and input tokens than the vision agent, which struggled due to limitations in interpreting rendered pages. This finding is significant for the AI/ML community as it highlights the importance of structured APIs, particularly in internal tools where the engineering costs of developing and maintaining these APIs can be minimized. Vision agents may still have their place in navigating third-party applications or legacy systems, but for in-house solutions, the stark difference in efficiency and cost suggests a shift towards API-driven models. As the cost to generate API surfaces decreases, more teams may abandon vision agents in favor of more efficient and predictable API agents, thereby improving productivity and reducing resource consumption. The benchmark results and associated code are open source, inviting further exploration and validation by the community.