Beating Warcraft III's insane AI with self-play reinforcement learning on one PC (warlock.ai)

An advanced AI agent has achieved remarkable success in playing Warcraft III, winning nine out of ten matches against the game's built-in "insane" difficulty AI. Achieving this feat from a single desktop setup—contrary to the extensive resources used by predecessors like DeepMind’s AlphaStar and OpenAI Five—demonstrates the effectiveness of self-play reinforcement learning. The agent operates under the game's own rules, making thousands of decisions within each match, challenging traditional limits of AI performance in complex real-time strategy games. Significantly, this project exemplifies innovative technical solutions to overcome the absence of a dedicated API or headless mode for Warcraft III. By running multiple game copies simultaneously and employing a tailored Python interface to drive the game, the developer was able to accelerate gameplay, gaining roughly 730 hours of experience per hour of real time. The learning process involved behavioral cloning from the built-in AI, self-improvement through self-play, and a finely tuned reward system that incorporates not just wins, but also intermediate rewards for material gains. This milestone not only advances understanding in reinforcement learning but also sets a precedent for future AI applications in competitive gaming.