Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Agents (arxiv.org)

Meta has unveiled a novel approach to enhance long-horizon autonomous research through a system called MIRA (Meta-reasoning for Iterative Research Agents). This innovative architecture streamlines the decision-making process by separating the tasks of research allocation and execution. MIRA features an outer-loop meta-reasoner that harnesses insights from a continuous research record to decide what to investigate next, while an inner-loop executor performs these tasks. By optimizing how resources are allocated without relying on policy training, MIRA significantly boosts long-horizon inference, making it effective in complex domains like theorem proving and neural architecture exploration. The implications of MIRA for the AI/ML community are profound, especially in reinforcement learning (RL). The introduction of MIRA-AC, a generative actor-critic model, enables the system to forecast potential returns and select subsequent investigations in a more targeted manner. This approach allows for efficient long-horizon RL without the burdensome need to directly optimize lengthy execution traces. By leveraging cross-environment pretraining for better adaptability, MIRA-AC demonstrates considerable performance improvements in various research environments, paving the way for more efficient autonomous agents capable of self-directed exploration and problem-solving in complex scenarios.