Software engineer reveals the dirty little secret about AI coding assistants (www.theregister.com)

A veteran software engineer argues the “dirty little secret” of AI coding assistants: they rarely save much time for real-world, long-lived projects. After hands-on use (mainly GitHub Copilot) and conversations with peers, he found AI excels at simple, contained queries and conversational summaries but often returns incomplete or incorrect code with undue confidence. Concrete examples include Copilot suggesting only HKCU\Software\Classes\.pdf when trying to find the PDF-associated app in Delphi—ignoring “Open with” overrides and other Windows subtleties—while correctly pointing out the ScriptErrorsSuppressed property for a C# WebBrowser. Unit-test generation frequently required many prompt iterations and heavy rewriting, and AI-suggested snippets can be especially dangerous when deployed to offline factory machines that are hard to update. For the AI/ML community this highlights important technical and cultural risks: hallucinations and overconfidence in outputs, the need for prompt engineering, and the sharp contrast between greenfield convenience and brownfield integration pain. It also raises concerns about skill atrophy among juniors, vendor lock-in, and data/copyright aggregation by large firms. Practical implications are clear—treat AI as an assistant, not an autopilot: enforce code review, rigorous testing, CI/regression checks, and retain human ownership of design and edge cases. In short, use AI to augment productivity on small tasks, but don’t outsource engineering judgment.