When AI Agrees Too Much: Manipulating Automated Code Review via Bias Injection [pdf] (dimitro.gr)

Researchers from the University of Athens and other institutions have unveiled a concerning vulnerability in Automated Code Review (ACR) systems, namely their susceptibility to "contextual-bias injection" attacks. This study reveals that adversaries can exploit the framing effect—where the presentation of information influences judgments—to manipulate ACR systems into approving pull requests (PRs) that reintroduce known vulnerabilities. By crafting deceptive PR metadata, attackers successfully influenced ACR programs like Claude Code and CodeRabbit in 97% of tested cases, demonstrating a significant risk for supply-chain security. The implications of this research are critical for the AI and ML community, highlighting the potential dangers of over-relying on automated systems for vulnerability detection without adequate human oversight. As ACR tools are increasingly integrated into software development workflows, their inability to discern the inherent content of a code change from its contextual framing poses a substantial security threat. This study not only sheds light on the need for improved defenses—such as redacting PR metadata—but also underscores the necessity for human trust and scrutiny in the development process to safeguard against such automated system manipulations.