Researchers who use hallucinated references to face ArXiv ban (www.nature.com)

arXiv has announced new sanctions against researchers who submit papers containing "hallucinated references" generated by large language models (LLMs). Under the new policy, any researcher penalized for such violations will be barred from posting future manuscripts unless their work has been accepted by a reputable peer-reviewed venue. This decision, supported by many in the AI community, aims to combat the proliferation of misleading citations and ensure the integrity of research circulating on the platform. Thomas Dietterich, chair of arXiv’s computer science section, emphasized the need for researchers to critically evaluate LLM outputs, expressing concern over the lax verification practices that have emerged. Critics, however, argue that simply banning authors from arXiv does not address the underlying issues associated with irresponsible LLM utilization. They suggest that researchers will continue producing questionable work elsewhere, raising questions about the overall trustworthiness of AI-generated content in scientific research. Dietterich acknowledged that while penalties will deter some, the broader challenge of addressing reliance on flawed AI outputs remains. This move marks a significant shift in how academic platforms are responding to emerging challenges posed by generative AI, particularly in the computer science field, which sees a high volume of AI-related submissions.