More Code ≠ Better Code: Claude Haiku 4.5 Wrote 62% More but Scored 16% Lower (codelens.ai)

🤖 AI Summary
Anthropic’s new Claude Haiku 4.5 was put through a real-world TypeScript refactor—adding exponential backoff, connection state, message queuing, heartbeats and metrics to a fragile WebSocket client—and produced strikingly poor bang-for-buck. Haiku 4.5 generated the most code (13,666 tokens) of eight models tested but scored 74.4/100, landing 7th; by contrast Claude Sonnet 4.5 wrote 8,425 tokens and scored 89.0/100. A dynamic judge (GPT-5) rated Haiku low on code quality (60/100) and correctness (65/100) due to duplicated methods, heavy boilerplate and mixed concerns, while awarding high completeness (90/100) for feature coverage. Haiku’s tokens-per-quality-point (184) was the worst among models, illustrating extreme verbosity with limited quality gain. The result highlights a practical risk in AI-assisted development: more generated code can be over‑engineering rather than improvement. Haiku’s exhaustive logging, metrics and abstraction layers increased attack surface, performance overhead, and maintainability issues—exactly the tradeoffs teams must weigh when accepting generator output. Top performers like GPT-5 and Sonnet hit better balances of conciseness and correctness. Caveat: this is a single-task, n=1 evaluation and directional rather than definitive; the author is running more tests to see if Haiku’s verbosity is a consistent pattern.
Loading comments...
loading comments...