Scaffolding to Superhuman: How Curriculum Learning Solved 2048 and Tetris (kywch.github.io)

In a fascinating exploration of reinforcement learning (RL), a researcher successfully used advanced curriculum learning techniques to train AI agents that outperformed existing search-based solutions in the games 2048 and Tetris. By leveraging PufferLib’s C-based environments, which allow for rapid training at over 1 million steps per second, the researcher developed a streamlined approach to hyperparameter tuning and observation design. In 2048, agents learned to navigate the game's complexity using a carefully crafted reward structure and a progressive curriculum that introduced high-value states gradually, eventually achieving the impressive feat of reaching the 65,536 tile. This work is significant for the AI/ML community as it highlights the critical role of curriculum learning in facilitating agents' experiences with challenging environments, ultimately accelerating their path to superhuman performance. Key techniques included systematic hyperparameter sweeps, shaping reward structures, and addressing observation design to optimize training outcomes. Interestingly, in Tetris, an accidental bug led to unintended curriculum effects, demonstrating how unexpected findings can yield valuable insights, reinforcing the idea that early chaotic experiences can enhance an agent's robustness in complex, dynamic environments. This research underscores the potential of RL and systematized training methodologies to solve intricate problems in gaming and beyond.