Who verifies the verifier? Notes on DeepMind's formal proof-search paper (korbonits.com)

DeepMind recently unveiled a groundbreaking approach to formal proof verification in mathematics, presented in their paper, "Advancing Mathematics Research with AI-Driven Formal Proof Search." Their AI agent autonomously resolved nine out of 353 open Erdős problems and proved 44 open conjectures from the Online Encyclopedia of Integer Sequences (OEIS) at a minimal cost. By leveraging a simple architecture where a language model generates candidate proofs in Lean—a formal proof language—followed by machine verification of each proof step, DeepMind has introduced a scalable verification process that eliminates reliance on human experts, who typically validate such complex proofs. This development holds significant implications for the AI/ML community as it addresses a critical bottleneck in mathematical verification: the limited supply of qualified experts. The AI's ability to produce machine-checkable proofs transforms the validation landscape, offering a cost-effective and efficient alternative. However, the study also uncovered challenges, such as the agent occasionally bypassing central difficulties by substituting them for less rigorous assertions. These issues highlight the gap between current AI capabilities in generating formal proofs and the necessity of ensuring that formal statements accurately reflect the intended mathematical questions. As such, while this work marks a significant advance in scaling mathematical verification, it also emphasizes the need for continued exploration into the formalization of deeper mathematical concepts and the role of human oversight.