Discovering state-of-the-art reinforcement learning algorithms (www.nature.com)

Researchers demonstrated that machines can automatically discover a state-of-the-art reinforcement learning (RL) update rule by meta-learning from the cumulative experiences of a population of agents across many complex environments. Instead of hand-crafting update formulas, the team trained a meta-network to produce the rule that governs how an agent’s policy and value predictions are updated. In large-scale experiments the discovered rule outperformed all existing hand-designed rules on the Atari benchmark and beat several state-of-the-art RL algorithms on challenging benchmarks the rule had not seen during discovery. Supplementary material details the meta-learned rule design, meta-network architecture, and meta-optimization pipeline. This work is significant because it shows that powerful, general-purpose RL algorithms can emerge through automated discovery rather than human design, suggesting a new path for algorithm development that leverages cross-task experience and population-based meta-training. Key technical implications: the object of meta-learning was the learning rule itself (not just hyperparameters or model weights), the discovery process used large-scale, multi-environment experience aggregation, and the resulting rule demonstrated zero-shot generalization to unseen tasks. The result points to scalable, data-driven algorithm synthesis for advanced AI, though it also implies substantial compute and engineering in the meta-optimization stage.