Tiny World Model (paul.mou.dev)

A developer has successfully implemented a "tiny world model" based on Ha & Schmidhuber's 2018 World Models paper, applying it to a clone of Chrome's Dino game. This model consists of three main components: a variational autoencoder (VAE) that compresses game frames, a recurrent world model using LSTM to predict future states, and a two-layer actor that interacts within the model. The controller for the game was trained using Group Relative Policy Optimization (GRPO), allowing it to learn directly from the simulated environment before transferring its learned strategy to the actual game. This approach emphasizes the importance of independent metrics to ensure system reliability, ultimately revealing significant issues when superficial metrics were mistaken for meaningful performance indicators. The significance of this development lies in its innovative application of world models to generate and manage interactions within simpler game environments. The method allows for the prediction of various possible future states, enhancing the model's understanding of stochastic environments like the Dino game, which continuously spawns obstacles. The technical focus on using a VAE for latent representation and incorporating a mixture-density network to predict future states serves as a bridge toward more complex applications in AI. As advancements in world models continue, this tiny version demonstrates foundational concepts that will pave the way for increasingly sophisticated and interactive AI systems.