Improving Agents by Bootstrapping Experientially-Learned Environmental Knowledge (arxiv.org)

Researchers have introduced BREW (Bootstrapping expeRientially-learned Environmental knoWledge), a groundbreaking framework designed to enhance the capabilities of Large Language Model (LLM)-based agents. Traditionally, these agents have struggled to learn from past interactions, requiring them to rediscover solutions in each session. BREW addresses this challenge by converting agents' interaction histories into a structured knowledge base of "recipes" that detail the steps to take in various scenarios. This knowledge base enables the agents to learn from experience by utilizing past successes to inform future tasks. The significance of BREW lies in its innovative approach to knowledge accumulation and task execution. Leveraging techniques like Expand-and-Gather Monte Carlo Tree Search (EG-MCTS) and hindsight relabeling, BREW improves both the accuracy of task execution and the efficiency of the agents, achieving notable performance gains of 10-20% in task success rates and reduced execution steps on benchmarks like OSWorld and SpreadSheetBench. The modular and transparent nature of the knowledge base not only enhances agent optimization but also opens avenues for greater controllability and adaptability, marking a substantial leap forward in the development of intelligent agents that can learn and improve through experience.