RRSI: Regularized Recursive Self-Improvement of Agent Harnesses (github.com)

A new approach called Regularized Recursive Self-Improvement (RRSI) has been announced, aimed at enhancing the adaptability of Large Language Model (LLM) agents by evolving their harnesses—structures that include prompts, tools, and memory management—without succumbing to overfitting. Traditional methods often result in the harness memorizing training tasks, leading to diminished performance in varied contexts. RRSI introduces an open edit space where modifications can be made across diverse components, with a regularized search that guides the evolution process through an annealed budget and various screening mechanisms to prevent bias and ensure continual improvement. The significance of RRSI lies in its ability to create agents that are more robust and versatile across different domains, as evidenced by its application in three specific instances: a coding terminal agent (Terminal-Bench 2.1), a document-work agent (Harvey LAB), and an engineering-design agent (EngDesign). By allowing each candidate harness to be drafted and evaluated independently in its own worktree, RRSI fosters a systematic and traceable evolution process, producing verifiable enhancements in performance metrics across tasks. Technical innovations such as a noise-adjusted evaluation framework and a methodical pruning system outline a path for effective harness evolution, potentially transforming how AI agents adapt and learn in real-world scenarios.