Sokoban Speedrun for RL (github.com)

Recent advancements in reinforcement learning (RL) have led to record-breaking speedruns for solving the classic puzzle game Sokoban using two distinct approaches: a large language model (LLM) track and a non-LLM track. In the LLM track, researchers fine-tuned the Qwen3-4B-Instruct-2507 model, achieving a significant increase in performance from 57% to over 80% in held-out pass@1 metrics on an 8xH100 setup. Meanwhile, the non-LLM track focused on training an agent from scratch on a single H100 GPU, with the top scoring submissions showcasing creative optimizations and novel RL strategies. These achievements are noteworthy for the AI/ML community as they push the boundaries of problem-solving in complex environments. With competitive records being established—such as a time of 19:20 in the LLM track and 12:38 in the non-LLM track—these dynamic efforts highlight the effectiveness of varied training strategies and RL algorithms. The emphasis on strict rules against using Sokoban-specific shortcuts underscores the pursuit of generalizable AI techniques. As details on training configurations and evaluations emerge, they will not only inform future research but also inspire innovations in RL methodologies.