Better Agent Leagues and Decision Supervision from Branched Rollouts (gertlabs.com)

Researchers have developed a novel system called VERDICT (Verifiable-Environment Rollouts at Decision Intervals for Credit over Trajectories), which enhances the evaluation of agentic decisions in complex games by utilizing branched rollouts. This approach allows agents to evaluate multiple candidate decisions at critical points during gameplay by simulating complete sessions for each candidate. The system aims to optimize the quality of training data while managing computational costs effectively, significantly outperforming traditional linear decision-making methods. In rigorous testing involving environments like Tribal Dominion, the branched sessions consistently delivered superior performance, yielding higher Elo ratings and more reliable programs. The significance of VERDICT lies in its ability to produce robust decision-making labels without the need for expensive human annotations, generating a rich dataset of 10,574 scored candidate labels for future training. By using the Monte Carlo Tree Search-inspired approach, VERDICT efficiently ranks and selects decisions based on their rollout scores, leading to higher performance across various game scenarios. The findings not only promise improved league creation from agentic sessions but also demonstrate that branched rollouts provide a more consistent quality baseline for decision-making systems in AI/ML applications.