Book: The Emerging Science of Machine Learning Benchmarks (mlbenchmarks.org)

The book "The Emerging Science of Machine Learning Benchmarks" critically explores the role and implications of benchmarks in machine learning (ML). While benchmarks have undeniably driven progress in AI, particularly seen in the success of datasets like ImageNet, critics argue that reliance on static test sets fosters narrow research goals and inflates model performance through metric gaming. This raises ethical concerns as well, with benchmark practices potentially reinforcing biases and exploiting marginalized labor in data annotation. The author highlights the urgent need to understand the underlying principles of these benchmarks and suggests a shift toward a more scientifically grounded approach. In addressing the changing landscape of ML, especially with the advent of large language models (LLMs) that can handle diverse tasks, the book examines the complications of multi-task evaluation and the impact of 'performativity'—how deployed models influence future data. Compounding the problem, increasingly sophisticated models challenge traditional evaluation methods, often exceeding human evaluators' capabilities. The author argues for the establishment of a robust scientific framework for benchmarking to navigate these complexities, focusing on theoretical and empirical insights that can guide future practices in the rapidly evolving AI field.