LLM Hallucinations in the Wild (arxiv.org)

Recent research highlights a significant issue regarding hallucinations produced by large language models (LLMs), revealing a dramatic increase in non-existent scientific citations following their widespread adoption. An audit of 111 million references from 2.5 million papers across platforms like arXiv and PubMed Central uncovered an estimated 146,932 fabricated citations anticipated in 2025. This phenomenon is particularly pronounced in research areas experiencing rapid AI integration, within manuscripts reflecting AI-assisted writing styles, and among small, early-career author teams. The implications of these findings are profound for the AI/ML community as they underscore the risks posed by LLM-generated errors to both the reliability of scientific literature and the equitable distribution of academic recognition. Hallucination errors often disproportionately credit established male scholars, potentially perpetuating existing biases within scientific fields. Moreover, current moderation and publication processes are unable to fully capture these inaccuracies, indicating that the prevalence of hallucinated content may undermining the integrity of knowledge production in the face of advancing AI capabilities. Addressing this issue is essential to ensure the credibility and fairness of future scientific inquiry.