Violations of LLM Review Policies (blog.icml.cc)

The ICML 2026 Program Chairs have announced a significant crackdown on violations of policies related to the use of large language models (LLMs) in the peer review process. After detecting that 506 reviewers who agreed to refrain from using LLMs still used them in approximately 795 reviews, the ICML rejected 497 papers based on these infractions. Reviewers were given the option to choose between two policies regarding LLM use: a conservative approach (Policy A), which strictly prohibited LLMs, and a more permissive stance (Policy B), allowing limited use. This incident underscores the need for conferences to adapt their peer review processes in light of technological advancements in AI, as misuse of LLMs threatens the integrity and trustworthiness of scholarly evaluations. The detection method involved subtly watermarking submission PDFs with hidden instructions that LLMs could recognize, enabling the flagging of reviews that violated policy agreements. Although this approach is not foolproof and could allow for circumvention, it emphasized the importance of maintaining ethical standards in research practices amidst rapid AI advancements. The measures taken aim to reinforce community trust and ensure that the evolving landscape of AI in academia does not compromise the peer review process.