Terence Tao:"An extended AI chat helped me answer a MathOverflow question" (mathstodon.xyz)

Fields Medalist Terence Tao reported using an extended conversational AI session to solve a MathOverflow question, and—crucially—he encountered no hallucinations or “AI nonsense.” He attributes this success to having a precise mental model of the needed tedious computations and guiding the model through a step‑by‑step conversational workflow: each elementary operation or transformation was explained and confirmed with the AI before moving on, and only at the end did he externally validate the AI’s numerical outputs with Python, which indeed satisfied the required constraints. The exchange highlights a practical pattern for trustworthy AI-assisted math and engineering work: human-in-the-loop, spec-driven dialogues where the expert decomposes tasks, verifies each step, and reserves programmatic checks for final validation. The community has already noticed parallels in software development—dubbed “Spec‑Driven Development”—and raised questions about whether chat customization (“custom instructions”) and workspace features can improve quality. The main technical implication is that structured conversational scaffolding plus selective external verification can mitigate hallucinations, but it depends on user expertise and clear procedural specs rather than being a stand-alone fix. This suggests product design opportunities for interfaces that better support iterative step confirmation, reproducible workspaces, and integrated validation tooling.