AI Review Loops Don't Always Stabilise (kevinmahoney.co.uk)

Recent discussions reveal that AI review loops, intended to enhance code quality, can inadvertently lead to instability rather than improvements. Developers may rely on a cycle where AI generates code, another AI reviews it, and then the original developer implements fixes, leading to a self-perpetuating process that lacks effective oversight. However, without proper guardrails, this method can spiral into inefficiency due to inconsistent AI evaluations, which may shift from one review to the next, fostering confusion and chaos rather than clarity. This phenomenon is significant for the AI/ML community as it highlights the challenges of relying solely on AI for software development tasks. Issues such as scope creep—where review prompts lead to unnecessary expansions of the project—and the introduction of new defects from false positives during reviews can severely undermine code integrity. A practical test using an AI model to generate and iterate on code reveals a troubling pattern: the defect count tends to increase with each review, emphasizing the need for structured implementations and proper human oversight in AI-assisted code review processes to achieve the desired outcomes.