Busting Legacy Code with AI Agents and Test Driven Development (yonatankra.com)

An author walked through using AI agents (GitHub Copilot in VS Code) to convert an old JavaScript repo’s telemetry-system into “evergreen” code by auto-generating a Jasmine test suite for TelemetryDiagnosticControls and its TelemetryClient interactions. With a single prompt the agent produced a comprehensive set of tests that exercise constructor behavior, read/write diagnostic info, and the checkTransmission flow (disconnect → connect with the diagnostic channel "*111#" → send TelemetryClient.diagnosticMessage() → receive and store diagnostic data), including retry logic (up to 3 connects) and exception behavior when offline. However, running the generated suite revealed five failing tests and several quality problems. The story highlights a practical lesson for the AI/ML community: agents accelerate test coverage but don’t replace thoughtful test design or human review. The AI produced brittle and incorrect tests—asserting against private fields (_telemetryClient, _diagnosticInfo), mismatched descriptions vs. expectations, and tight coupling to implementation details—creating “AI-generated legacy code.” Technical implications: use AI to bootstrap tests and uncover behavior, but enforce TDD best practices (test public interface, clear assertions, validate retry/exception flows), perform manual vetting, and refine prompts. Properly integrated, AI agents can dramatically speed legacy-code coverage; misused, they can propagate fragile, misleading tests that give false confidence.