Superhuman AI coding still makes software worse (www.williamangel.net)

Recent analysis reveals that while AI has advanced in coding capabilities, its impact on software quality may not be beneficial in the long run. Although AI can generate code more quickly than humans, this speed can lead to an increase in defects. The study highlights a mathematical relationship where higher change velocity, even with a lower defect rate, can still result in a greater number of bugs. For example, if AI generates changes at ten times the speed of human developers, it could lead to five times the number of defects, even if its defect rate is lower. This raises concerns about the sustainability of code quality in large and complex software systems. Moreover, issues like model collapse—where AI-generated code loses coherence due to the self-referential nature of synthetic data—and context limitations make scaling complex codebases challenging for both humans and AI. The report suggests re-evaluating development practices to prioritize quality over speed, drawing insights from DevOps and agile methodologies. For the foreseeable future, retaining human oversight in software development processes is critical to ensure stability and effectiveness, marking a cautious but necessary approach to integrating AI in coding tasks.