Models are worse at reviewing their own code (www.greptile.com)

In a recent exploration by Greptile's research team, it was found that AI models are less effective at reviewing their own code than they are at reviewing code authored by other models. Using two datasets of pull requests (PRs) authored by OpenAI's Codex and Anthropic's Claude Code, researchers discovered that each model found more bugs in the other's code compared to its own. This significant finding highlights potential limitations in self-review capabilities among AI models, as the types of bugs they tend to introduce often align with the categories they fail to detect during reviews. The research also indicated that different models adopt unique reviewing strategies—while Claude models focus on broad patterns and potential issues, GPT models emphasize depth and verification. This disparity results in what the team terms "Model Inversion," where the AI code review agent routes PRs to a different model depending on the author, thereby enhancing the likelihood of catching bugs. This innovative approach could redefine code review processes in AI/ML, making it essential for the community to recognize and address the inherent biases in AI’s self-assessment capabilities.