AI-to-AI Code Reviews of GitHub Pull Requests (arxiv.org)

A recent study has highlighted the emerging phenomenon of AI-to-AI code reviews in the GitHub development ecosystem, where AI agents both create and evaluate pull requests (PRs). This research utilizes a substantial dataset from CodAGE, comprising 248,641 unique AI-generated PRs linked to AI reviews, revealing that AI reviewers are increasingly involved in assessing contributions from other AI agents. Notably, cross-product AI-to-AI reviews accounted for about 1.6% of PRs, but the volume surged dramatically within a single year, suggesting a rapid adoption of AI-mediated workflows in software engineering. The implications of this trend are significant for the AI/ML community, as it signifies a shift toward closed-loop systems in software development where AI not only assists in coding but also actively participates in its evaluation. The findings show variations in reviewer feedback based on the authoring AI's identity and the type of review, emphasizing that certain configurations yield different levels of critique. This evolution in automated code evaluation could lead to improved code quality and efficiency but also raises questions about the dependency on AI in critical decision-making processes in software development.