The Measure of Intelligence (2019) (arxiv.org)

A recent paper, "On the Measure of Intelligence," explores the need for a robust framework to evaluate intelligence in artificial systems, a topic critical to the AI/ML community's advancement. The authors argue that conventional benchmarks, primarily based on task-specific skills—like playing games—fail to capture the essence of intelligence, which they define as skill-acquisition efficiency derived from Algorithmic Information Theory. Their criticism highlights the limitations of existing methods that are heavily influenced by prior knowledge, leading to misleading assessments of a system's true generalization capabilities. To address these shortcomings, the paper proposes a new benchmark, the Abstraction and Reasoning Corpus (ARC), designed to align with human-like general fluid intelligence. By focusing on a standardized set of innate priors, ARC aims to facilitate fair comparisons between human and AI systems, paving the way for a deeper understanding of general intelligence. This work not only challenges prevailing measures of AI proficiency but also sets the stage for future developments in creating more intelligent systems that reflect human-like reasoning and adaptability.