The Conversation We're Not Having About AI in Peer Review (cabird.com)

A recent essay highlights the integration of AI into the peer review process, emphasizing the urgent need for the academic community to establish guidelines for its use. The authors, two senior software engineering researchers, reflect on their independent development of AI-assisted reviewing practices, which led to clearer assessments and more constructive critiques. While AI does not replace their judgment, it helps clarify their thoughts, enhancing the quality of reviews despite not necessarily speeding up the process. The essay also addresses the tension surrounding AI's role, particularly after a controversy at a major conference where papers contained fabricated references, igniting concerns about integrity in scientific work. The significance of this discussion lies in the acknowledgment that AI's presence in peer review is inevitable, and the community must navigate its implications. The authors propose a set of initial guidelines based on their experiences, advocating for a "human-first" approach to AI usage that retains reviewer judgment while leveraging AI's strengths. By framing AI as a tool for enhancing the review process—such as formulating constructive feedback and exploring potential concerns—rather than an outsourcing mechanism, the essay aims to stimulate a thoughtful dialogue about best practices in integrating AI in academia.