Are bad incentives to blame for AI hallucinations? (techcrunch.com)

OpenAI’s latest research paper tackles the persistent issue of hallucinations in large language models (LLMs) like GPT-5 and ChatGPT—where models generate plausible but factually incorrect information. Despite improvements over time, hallucinations remain an inherent challenge because LLMs are trained to predict the next word based on language patterns, not to verify factual accuracy. This limitation is highlighted by experiments where a chatbot confidently produced multiple, distinct wrong answers about one of the author’s personal details, illustrating how low-frequency, arbitrary facts are especially prone to errors. The paper argues that hallucinations are exacerbated by the evaluation methods used for these models. Current benchmarks primarily reward accuracy—getting answers exactly right—without penalizing confident mistakes or rewarding uncertainty. This setup encourages models to “guess” answers even when unsure, rather than admit ignorance. To counter this, OpenAI proposes redesigning evaluation frameworks akin to tests that penalize wrong answers or grant partial credit for uncertainty. By updating scoring metrics to discourage blind guessing and more heavily penalize confidently wrong answers, models may better learn to express appropriate uncertainty, reducing hallucinations. This insight shifts focus from just improving training data to aligning incentives during evaluation, a significant step toward more reliable AI systems.