Insurmountable Hans (or the era of turbocharged goalpost moving) (cjauvin.github.io)

François Chollet's ARC benchmark, established to challenge AI capabilities without falling prey to simplistic "hacks," has seen renewed interest since the emergence of advanced AI models like ChatGPT. Initially, ARC posed significant difficulties for AI programs due to its focus on deep reasoning beyond mere pattern recognition. Despite initial struggles, recent progress in solving ARC—fueled by an increased incentive through the newly introduced ARC Prize—raises questions about what these advancements signify for the pursuit of Artificial General Intelligence (AGI). However, the accelerating success in tackling ARC presents a paradox: as AI systems improve, the criteria for what constitutes true intelligence seem to shift even faster. This phenomenon, described by the "Insurmountable Hans" principle, suggests that no matter how advanced AI becomes, there may always be anthropological resistance to fully recognizing machines as genuinely intelligent beings. This ongoing debate emphasizes that the quest for AGI remains complex and elusive, highlighting both technical advancements and the philosophical challenges that continue to shape the AI landscape.