Ora benchmarks every major AI agent on Vercel – Customers (vercel.com)

Ora has launched a groundbreaking platform on Vercel that benchmarks all major AI agents, providing real-time assessments of their performance on live customer websites. This unique system runs agents—like Claude Code, ChatGPT, and eve—side by side to evaluate their effectiveness in executing tasks such as product sign-ups and integrations. By tracking metrics like cost, latency, and stepwise success in these workflows, Ora identifies where agent failures occur, which represents a significant advancement in understanding agent readiness across the web. Currently, they estimate that 99% of websites are not agent-ready, highlighting a vast opportunity for improvement. The integration of complex benchmarking technology built entirely on Vercel enables Ora to provide detailed insights into agent performance without requiring separate infrastructures. Notably, the platform not only benchmarks agents but also improves their operational capabilities; after initial tests of the eve framework revealed performance issues, Ora collaborated with Vercel to deliver timely fixes. This ability to refine agent technology based on empirical evidence marks a pivotal step forward for the AI/ML community, allowing developers to enhance their applications with informed, data-driven adjustments and fostering a more agent-ready web ecosystem.