Terminal-Bench-Science: Evaluating AI agents on scientific research workflows (www.terminal-bench-science.ai)

Terminal-Bench-Science 0.1 has been launched as a new benchmarking initiative led by Stanford researchers to evaluate AI agents using authentic scientific research workflows developed by scientists themselves. This benchmark consists of 70 tasks across multiple scientific disciplines, including life, physical, Earth, mathematical, and engineering sciences, aiming to establish a relevant measure of AI capabilities that extends beyond traditional, simplified assessments. The leading model, Claude Opus 5, achieved a 30% resolution rate, which underscores the benchmark's challenging nature designed to assess AI performance in realistic scientific contexts. This initiative is significant for the AI/ML community as it creates a direct channel for scientists to express their needs and define the criteria for evaluating AI capabilities, facilitating the creation of more effective research assistants. The dynamic nature of Terminal-Bench-Science ensures it will evolve alongside advancements in AI, enabling ongoing refinement of tasks and evaluation processes. By establishing verifiable and complex tasks that reflect real scientific practice, this benchmark not only reveals the current limitations of AI agents but also fosters dialogue and collaboration between AI developers and scientific professionals, ultimately aiming to accelerate scientific discovery.