ChatGPT seemed to think on the fly when put through an Ancient Greek math puzzle (www.cam.ac.uk)

Researchers tested ChatGPT-4 on Plato’s 2,400-year-old “doubling the square” lesson — a geometry problem Socrates used to show a pupil that a square with double area has sides equal to the original square’s diagonal. Instead of immediately reproducing the classical geometric proof, ChatGPT repeatedly produced algebraic solutions (anachronistic to Plato’s era), resisted being led into the pupil’s original mistake, and only produced the geometric construction after sustained prompting. In a rectangle variant it stuck to algebra and even asserted (incorrectly) that no geometric doubling was possible via the diagonal; for a triangle it again defaulted to algebra but later gave a correct geometric answer after more coaxing. The study’s significance lies in what the chatbot’s behavior reveals about LLM “knowledge”: rather than clean retrieval, ChatGPT’s surface behavior blended memorized text with on‑the‑fly hypothesis-making and errors that resemble learner‑like reasoning. Because LLMs are trained on text (weak on diagrams), they can default to algebraic reasoning and invent plausible but false claims. The authors argue this creates a “Chat’s zone of proximal development” where prompting can elicit stronger solutions, and they warn educators that AI‑generated proofs cannot be assumed valid. Pedagogically, interacting with models to probe and critique their reasoning — not simply accepting answers — should become a core skill in math curricula.