If coding is solved, what now?: Measuring the sloppiness of code (earendil.com)

Recent discussions surrounding large language models (LLMs) highlight a new concern: while these models excel at generating correct code, they often produce "sloppy" code characterized by unnecessary abstractions, duplication, and poor decision-making. This sloppiness becomes particularly problematic as projects scale, with massive lines of code (LOC) accumulated rapidly, which complicates human oversight and quality control. Aiming to quantify code quality, researchers at Earendil have begun exploring metrics to measure this sloppiness, revealing that existing evaluation methods, such as using LLMs as judges, are inadequate. In their research, they tested various measures, such as "verbosity" (the amount of duplicated content) and "erosion" (the concentration of complexity in large functions), finding that agent-generated code averaged significantly higher levels of both metrics compared to traditional human-written code. For instance, the verbosity in LLM-generated code was found to be about twice that of human-generated projects. This raises critical implications for the AI/ML community: reliance on automated coding agents may inadvertently lead to a decline in code quality. The findings underline the importance of developing more nuanced evaluation methods and signal a need for greater human oversight in the coding process to maintain standards in software development.