AI models are using material from retracted scientific papers (www.technologyreview.com)

Several recent studies — confirmed by MIT Technology Review — show popular AI chatbots and research assistants sometimes base answers on retracted scientific papers without flagging their retraction status, risking misleading users who don’t click through sources. In one experiment, researchers led by Weikuan Gu queried GPT-4o about 21 retracted medical‑imaging papers; the model cited retracted work in five answers and warned only three times. A separate study using ChatGPT‑4o mini on 217 retracted or low‑quality papers found no mention of retractions. Tests of research tools (Elicit, Ai2 ScholarQA/Asta, Perplexity, Consensus) also cited multiple retracted papers — Ai2 and Consensus referenced 17–18 of the 21 papers in early tests — though some vendors (e.g., Consensus) reduced false citations after adding retraction feeds. Why this matters: researchers, students and the public increasingly use AI for literature review and medical queries, and governments (the U.S. NSF committed $75M for science AI) are investing in AI for research. Technical causes include stale training data, inconsistent publisher retraction labels (retracted, erratum, expression of concern), scattered preprint copies, and incomplete retraction databases (Retraction Watch is curated but not exhaustive). Practical fixes include integrating multiple retraction feeds, real‑time checks, surfacing peer reviews and PubPeer critiques alongside answers, and treating retraction metadata as a critical quality signal — but vendors and users still need to exercise skepticism and due diligence.