Agentic Bug Reproduction for Effective Automated Program Repair at Google (arxiv.org)

Bug reports often lack the failing tests developers need to reproduce and fix defects. Google researchers adapted a state-of-the-art Bug Reproduction Test (BRT) generator and built an agentic alternative, BRT Agent, which uses a fine-tuned large language model for iterative code editing to create tests that fail when a bug is present and pass when it’s fixed. Evaluated on 80 real, human-reported bugs from Google’s internal tracker, BRT Agent produced plausible BRTs for 28% of cases versus 10% for the prior LIBRO approach, demonstrating substantial gains in real-world, large-scale, proprietary codebases. Beyond reproduction, the team integrated generated BRTs with Google’s Automated Program Repair (APR) pipeline and showed concrete downstream benefits: supplying BRTs increased the number of bugs with plausible APR fixes by 30%. They also propose Ensemble Pass Rate (EPR), a metric that uses generated BRTs to rank APR candidate patches; EPR correctly selects a plausible fix from 20 candidates about 70% of the time (top-1). The work highlights that agentic, LLM-driven code editing can materially improve both test generation and repair triage in industry settings, offering a practical path to speed up debugging and prioritize higher-quality automated patches.