Training a local model to play browser games on the fly (ironbee.ai)

A new AI model named Laya, consisting of 322 million parameters, has been successfully trained to play browser games like Chrome Dino and Flappy Bird in real-time without any modifications to the game code. Laya makes decisions every 25-35 milliseconds based on a set of rules generated by a large language model (LLM). This LLM analyzes the state of the game, which is transformed into a JSON format, allowing Laya to determine optimal actions, such as when to jump or duck, by learning from both successful and failed attempts. The significant advancement here lies in the model's ability to learn dynamically from its performance, adapting its strategy for improved outcomes. This breakthrough is noteworthy for the AI/ML community as it demonstrates the potential of combining LLMs with real-time decision-making in gaming environments. The ability for Laya to independently derive game logic and improve its gameplay over time—without external APIs or code changes—highlights the model's generalization capabilities. Additionally, the structural design, utilizing a frozen game clock and various techniques to accurately interpret the game's state, showcases innovative approaches in AI training methodologies and sets the stage for further advancements in AI's autonomous operation within complex tasks.