The Next Evolution of AI Is Learning From Your Dodgy Gaming Skills (www.wired.com)

A British startup, Worldmodeldata, aims to revolutionize AI training by leveraging data extracted from video games to enhance world models—AI systems designed to navigate physical environments. As large language models (LLMs) face limitations in real-world applications due to their text-only training, researchers, including prominent figures like Fei-Fei Li and Yann LeCun, are shifting focus towards combining visual and action data for more advanced AI. Worldmodeldata seeks to curate and organize controller input data from various video games, which could address the current shortage of training data essential for the development of world models. The significance of this initiative lies in its potential to provide vast, diverse training datasets that capture nuanced scenarios—often referred to as "corner cases"—which are crucial for the success of AI in tasks like autonomous driving or robotic manipulation. By licensing nearly 1 million hours of gameplay data from popular studios, Worldmodeldata posits that video game environments can serve as an abundant resource for training AI to understand complex physical interactions. However, some experts, like Ming-Yu Liu from Nvidia, express caution about using video game data for tasks requiring fine motor control, highlighting concerns over the realism of in-game physics. As the AI community explores various avenues to refine world models, Worldmodeldata's innovative approach could pave the way for significant advancements in AI capabilities.