Artificial Analysis Intelligence Index v5 (artificialanalysis.ai)

The launch of the Artificial Analysis Intelligence Index v5 marks a significant update in benchmarking AI models, featuring two new benchmarks and three substantial upgrades, primarily built on private datasets. Notably, for the first time, 60% of the Index evaluates models using private test sets, enhancing the realism of benchmarks. Key additions include FrontierCode 2.0, which assesses coding tasks based on repository maintainer approvals, and Terminal-Bench-Science, focusing on scientific research workflows. Upgraded benchmarks like HLE-Diamond streamline assessments of reasoning and knowledge, while the newly introduced Artificial Analysis Progress Index v1 tracks model capabilities over time on a continuous scale. This release is crucial for the AI/ML community as it illustrates the evolving landscape of AI capabilities, emphasizing the disparity between leading models and their open weights counterparts. Top performers include Claude Opus 5.5 and GPT-6 Astra, though open weights models, such as Kimi K3.1 Pro, still lag significantly. The Index reveals notable variances in cost per task, with GPT-6.1 Sol showcasing superior performance at a fraction of the expense of higher-scoring models. This expansion of the Index not only provides a more nuanced understanding of AI capabilities but also highlights areas for continued advancement within the field.