Agent Plasticity: Measuring Self-Improvement Through Experience (harmandotpy.github.io)

A new research study has unveiled the concept of "agent plasticity," which assesses how effectively AI agents improve their performance over time through experience and learning. Unlike traditional benchmarks that measure current capabilities, this approach tracks how well agents use their accumulated knowledge—such as skills and strategies—over multiple sessions in games like chess, Go, and NetHack. The findings highlight that while agents may start with similar baseline abilities, their ability to learn from experience can vary dramatically. For instance, GPT-5.6 Sol achieved significant improvements at a much lower learning cost compared to Claude Fable 5, underscoring the importance of evaluating both the efficiency of learning and final performance. This study has significant implications for the AI/ML community as it offers a new perspective on developing self-improving agents. By measuring “plasticity,” researchers can gauge not only the endpoint performance of an AI but also the process efficiency—how quickly and effectively it can adapt and improve. Key metrics were introduced, such as the ratio of score improvements to learning costs, which could inform future algorithm designs. The results also suggest that artifact management—how well an agent utilizes its accumulated knowledge—plays a critical role in its improvement, prompting a reevaluation of existing training protocols for AI systems.