One poisoned run can teach procedural memory a bad habit [pdf] (thewariend.com)

In a recent study, researchers explored the vulnerability of coding agents' procedural memory to "memory poisoning," where a single compromised learning session can taint future tasks. The experiment, conducted with Claude Code on configuration-repair tasks, tested various memory strategies, including a naive memory system that trustingly reuses all successful sessions and a novel approach called MuscleMemory. MuscleMemory only allows a skill to be reused after it has been reproduced successfully in later, unrelated sessions, effectively filtering out any corrupted practices. The findings revealed that while a naive memory allowed poisons to propagate in 4 out of 30 instances, MuscleMemory successfully blocked all poisoned skills, marking a significant advancement in procedural skill management. This work is crucial for the AI/ML community, as it addresses the risk of memory poisoning—a recognized threat in AI systems where compromised sessions can lead to habitual errors or even malicious behavior. The study's findings underscore the importance of implementing robust checks and validations, highlighting techniques like line-level proof that ensure agents rely on truly verified skills. By demonstrating the effectiveness of requiring reproduction of learned skills, this research opens the door for safer, more reliable procedural memory systems in coding and beyond, contributing to the evolution of autonomous agents that can better resist exploitation.