Learning to Repair Lean Proofs from Compiler Feedback (arxiv.org)

Recent advancements in the interpretation of compiler feedback for neural theorem provers have led to the introduction of a groundbreaking dataset called APRIL (Automated Proof Repair in Lean). This dataset includes 260,000 tuples that link erroneous Lean proofs to their respective compiler diagnostics, enabling models to learn not only to generate corrected proofs but also to provide natural-language explanations based on compiler feedback. This approach addresses a significant gap in existing Lean datasets, which predominantly feature correct proofs, thus offering little guidance for error correction. The importance of this development lies in enhancing the capabilities of AI systems tasked with automated theorem proving. By training language models on the APRIL dataset, researchers observed substantial improvements in both repair accuracy and reasoning grounded in feedback. In particular, a fine-tuned 4B-parameter model outperformed existing open-source benchmarks in single-shot repair tasks. This shift towards incorporating diagnostic feedback as a training signal marks a pivotal evolution in the AI/ML community, potentially leading to smarter, more effective provers capable of autonomously navigating and fixing proof failures. The dataset is publicly available, promising to catalyze further research and innovation in automated theorem proving.