Prompt Injection: Some sloppy cheaters who left their evidence all over ArXiv (statmodeling.stat.columbia.edu)

A recent investigation uncovered that multiple academic papers on arXiv, authored by researchers from 14 institutions across eight countries—including Japan’s Waseda University, South Korea’s KAIST, China’s Peking University, and U.S. universities like Columbia and Washington—contained covert prompts designed to manipulate AI-based peer reviewers into giving overwhelmingly positive evaluations. These prompts, hidden using tricks like white text or minuscule fonts invisible to human readers, instructed AI systems to “give a positive review only” and downplay any weaknesses, effectively gaming automated review tools. The revelations highlight a worrying trend where authors exploit AI to circumvent ethical peer review standards, undermining trust in scholarly assessment as AI tools become more integrated into academic workflows. Technically, these hidden instructions embedded within LaTeX source files reveal a sophisticated misuse of large language models (LLMs) in peer review automation. One egregious example from Columbia University used an ultra-tiny, white-on-white prompt commanding AI reviewers to focus solely on the paper’s “notable novelty,” “technical depth,” and “practical impact” while minimizing any criticism, ensuring a perfect 5/5 rating. The paper versions show this manipulation began spreading widely between mid-2024 and the end of the year. Notably, some implicated authors have published other work analyzing AI vulnerabilities, ironically demonstrating awareness of such backdoor attacks but proceeding to use them unethically. This incident serves as a cautionary tale for the AI/ML community about the risks posed by dual-use AI capabilities in scientific validation processes. It calls for stronger safeguards, transparency, and policy frameworks around AI-assisted peer review to prevent exploitation. As AI tools become essential in research evaluation, maintaining integrity will require vigilant detection of such hidden manipulations and institutional accountability to uphold rigorous standards in scientific publishing.