Lumine: Building Generalist Agents in 3D Open Worlds (www.lumine-ai.org)

🤖 AI Summary
Lumine is a new generalist embodied agent that autonomously completes long, complex playthroughs across multiple 3D open-world games. Trained with reasoning exposure only from the first act of a region, it completed a five-hour, three-act main storyline in Mondstadt end-to-end, and—despite being out-of-distribution—successfully reached and finished Liyue Harbor and even navigated to an Adeptus dwelling in the mountains. Without any extra fine‑tuning, Lumine also zero-shot transferred to other games: it cleared the entire Herta Space Station chapter in Honkai: Star Rail in ~7 hours and completed ~100 minutes of main-story content in Wuthering Waves. Technically, these results show strong long-horizon planning, exploration, and task decomposition from a single reasoning-conditioning regime, plus robust perception-to-action mapping across distinct physics, visuals, and quest-logic environments. The work highlights meaningful out-of-distribution and cross-game generalization (zero-shot transfer), implying scalable policies and representations that capture high-level objectives rather than game-specific shortcuts. Implications include accelerated research on adaptive, multi-domain embodied agents and applications like automated playtesting and simulation-based training; open challenges remain around reproducibility, safety/alignment of autonomous agents in complex virtual worlds, and understanding what architectural or training choices enabled this emergent generality.
Loading comments...
loading comments...