The Mathematician's Assistant: Integrating AI into Research Practice (arxiv.org)

This paper surveys the state of publicly accessible LLMs for mathematical research (snapshot to Aug 2, 2025), highlighting both striking capabilities and systematic weaknesses. Using benchmarks like MathArena and the Open Proof Corpus, the authors show that recent systems (e.g., “AlphaEvolve,” “Gemini Deep Think”) can solve problems and rate proofs robustly, yet commonly fail at reliable self-critique: high final-answer accuracy often masks invalid or incomplete full proofs. The mismatch between surface correctness and formal validity, plus model-dependent variability, means current models are powerful assistants but unsafe as independent theorem provers. To address this, the authors propose a durable integration framework centered on the “augmented mathematician”: AI as a copilot guided by human expertise. Distilled into five guiding principles and seven concrete roles across the research lifecycle (from ideation to writing), the framework emphasizes iterative prompting, rigorous verification, and methodological controls rather than automation. For the AI/ML community this stresses new priorities—benchmarks and metrics that capture proof validity and calibration, mechanisms for model self-critique and uncertainty, and tooling that supports human-in-the-loop verification. Practically, successful adoption will require new researcher skills (strategic prompting, critical evaluation) and stronger evaluation suites to bridge the gap between impressive answers and provably correct mathematics.