WorldGen: From Text to Traversable and Interactive 3D Worlds (arxiv.org)

A groundbreaking system called WorldGen has been introduced, capable of automatically generating expansive, interactive 3D worlds from simple text prompts. This innovative approach harnesses large language models (LLMs) for scene layout reasoning, alongside procedural generation and diffusion-based 3D techniques, to create fully textured and navigable environments. The modular design of WorldGen empowers users with precise control over layout, scale, and style, allowing for the rapid development of coherent virtual spaces without requiring advanced 3D modeling skills. This development is significant for the AI/ML community as it democratizes the process of 3D world-building, making it accessible to a wider range of creators—from game developers to educators and social simulation designers. By merging creative intent with functional functionalities, WorldGen essentially paves the way for the next generation of generative AI in gaming and immersive environments, enhancing user experiences and expanding the potential applications of AI in virtual design. The ability to create visually rich and geometrically consistent worlds in real-time could revolutionize industries reliant on 3D visualization and simulation, setting a new standard in interactive content creation.