Steer Agents with Socrates and Meeseeks (backnotprop.com)

In a recent exploration of coding agents, the author challenges the reliance on documentation-driven approaches, revealing that both LLM-generated and developer-written context files lead to minimal performance improvements while increasing costs. This experience underscores a vital insight: agents often struggle with "context rot"—the diminishing accuracy of documentation over time—which can complicate tasks rather than streamline them. The author recounts the struggle of maintaining documentation while finding that it detracted from actual development work. Instead, the author proposes a more effective method inspired by the Socratic technique, advocating for a model where agents are guided through a planning and discovery process tailored to each task. By allowing agents to explore the relevant codebase instead of relying on potentially outdated documentation, this "agentic discovery" method promotes real-time context understanding while limiting task duration. This shift to task-specific, ephemeral interactions encourages focused execution and efficient problem solving in AI/ML environments, ultimately enhancing the productivity and efficacy of coding agents.