World-in-World: World Models in a Closed-Loop World (arxiv.org)

Researchers released World-in-World, an open platform for evaluating generative world models (WMs) in true closed-loop agent–environment interactions rather than the usual open-loop visual prediction tests. The project standardizes an action API and an online planning strategy and supplies four rigorously designed closed-loop environments with task success as the primary metric. It also reports the first data-scaling law for WMs in embodied settings, enabling apples-to-apples comparisons of how models improve as they see more action–observation data. The study yields three practical surprises for the AI/ML community: (1) high visual fidelity alone does not ensure agents succeed—controllability and accurate action-conditioned dynamics matter more; (2) fine-tuning or scaling WMs with action–observation data after pretraining delivers bigger closed-loop gains than merely upgrading pretrained video generators; and (3) allocating more inference-time compute (e.g., deeper/planned rollouts or heavier planners) substantially boosts performance. Practically, the work suggests benchmarks and model design should prioritize action-conditioned dynamics and inference/plan capacity over raw video realism, and it provides a shared platform for testing those hypotheses.