Top AI models have mostly stopped inventing citations, Stanford study (thinkpol.ca)

🤖 AI Summary
A recent Stanford study reveals significant changes in the behavior of large language models (LLMs) regarding citation accuracy. Researchers re-evaluated a 2023 citation test where AI models like ChatGPT significantly fabricated cited sources, and found that the top models, particularly OpenAI’s GPT-6.1 Sol, have drastically reduced this tendency, inventing no citations. Conversely, models such as Alibaba’s Qwen3.7-Flash still show high rates of fabricated references. The findings indicate that while some weaknesses in LLMs, such as inventing citations, have decreased, persistent issues related to biases and correctness in responses remain. This study is crucial for the AI/ML community as it emphasizes the rapid evolution of these models and the need for ongoing scrutiny of their capabilities. The researchers noted that almost half of the critical findings from previous studies have "expired" or diminished over time, suggesting that assessments of AI performance must be continuously updated. The implication for researchers and users is clear: the reliability of AI outputs can change quickly, underscoring the importance of taking a temporal approach to understanding and utilizing LLMs in academic and professional contexts.
Loading comments...
loading comments...