Scientists decry conference's use of hidden prompts to snare AI peer reviews (www.thetransmitter.org)

Organizers of the upcoming NeurIPS conference are facing controversy for incorporating hidden prompts into papers submitted for peer review, aimed at identifying reviewers who may be using generative AI to generate their assessments. This measure is part of a broader policy that prohibits reviewers from uploading the papers they critique to AI chatbots, which is seen as a breach of confidentiality. The hidden prompts instruct AI systems to recognize specific phrases typical in review reports, and while some researchers argue this approach could damage trust within the academic community, others believe it enhances integrity by catching unauthorized AI use. The significance of this move lies in the escalating concerns over the use of AI in the peer review process, particularly as similar tactics have reportedly led to the identification of numerous reviewers misusing AI in another notable conference. Although the NeurIPS committee has communicated with reviewers about the existence of these hidden prompts to mitigate misunderstandings, experts like Sören Auer and Sara Atito have criticized the method as flawed, suggesting it oversimplifies the complexities of peer review. Conversely, advocates like Nihar Shah highlight its effectiveness, claiming to have successfully flagged hundreds of inappropriate AI-assisted reviews in past conferences, a sentiment echoed by many in the community who feel overwhelmed by inadequate review quality.