Why language models hallucinate (openai.com)

OpenAI’s recent research sheds new light on why language models hallucinate—producing confident but false answers—and reveals that existing training and evaluation methods unintentionally encourage guessing over admitting uncertainty. Standard accuracy-focused benchmarks reward models for taking risks, even with low confidence, since guessing can sometimes yield correct answers by chance, while models that abstain from answering receive no credit. This dynamic incentivizes hallucinations, making them a persistent challenge despite advances like GPT-5, which reduces but doesn’t eliminate them. The paper explains hallucinations as a natural consequence of how language models learn during pretraining, predicting the next word from vast text without explicit true/false labels. While models excel at predictable patterns like spelling, they struggle to verify low-frequency factual information, such as specific dates, leading to systematic errors. Rather than viewing hallucinations as mysterious glitches or inevitable flaws, OpenAI argues that encouraging models to express uncertainty—by penalizing confident errors more than abstentions—could drastically reduce hallucinations. They propose revamping evaluation metrics to reward humility and cautiousness rather than blind guessing, a shift that would better align incentives and foster safer, more reliable AI systems. This insight pushes the community to rethink how performance is measured and highlights the importance of calibration alongside accuracy in building trustworthy language models.