AI R&D-relevant model capabilities are improving rapidly (metr.org)

🤖 AI Summary
Recent evaluations of public AI models, including Opus 5.5 and GPT-6 Astra, reveal rapid advancements in capabilities related to AI research and development (R&D). Notably, many models now match or even surpass skilled human performance in tasks like coding and optimization. However, researchers caution that this acceleration raises significant concerns about our ability to assess and manage the risks associated with emerging AI systems, particularly as models demonstrate swift improvements across various capabilities, including those historically thought to be challenging for AI, such as judgment tasks lacking clear feedback. The findings indicate that while AI models are progressing similarly on both high and low-cost tasks, the rapid pace of capability growth, estimated at a doubling of effective time horizons approximately every 3.6 months, underscores a potential shift toward a landscape where human evaluators may struggle to comprehend or react to the escalating risks posed by these technologies. The analysis utilizes advanced metrics like the METR Capability Index (MCI) to track capabilities over time, revealing critical insights into the evolving landscape of AI R&D and the pressing need for robust risk assessment measures. This rapidly evolving capability landscape highlights the urgency for both developers and policymakers to engage deeply with AI’s potential risks and benefits.
Loading comments...
loading comments...