Video-generative world models
The generative branch treats world modeling as video prediction: if a model can generate coherent futures of the visual world, it must have internalized something about how the world works. OpenAI’s Sora (2024) made the claim explicit by calling video generation models “world simulators.”
The verified critique of (original) Sora
The survey literature (Ding et al.) holds that original Sora fails as a full world simulator on two specific grounds:
- No action conditioning, no causal reasoning — it passively generates plausible sequences without predicting how actions would alter events.
- Inconsistent physics — it does not reliably reproduce physical laws: object behavior under forces, fluid dynamics, light interactions.
The physics finding is independently corroborated twice: by Bytedance’s “How Far is Video Generation from World Model: A Physical Law Perspective” (phyworld), and by DeepMind’s Physics-IQ benchmark, which found original Sora the most visually plausible model tested yet bottom-ranked on physical understanding — visual quality and physical understanding are dissociated properties.
Scope matters: this critique applies to original Sora. Sora 2 (late 2025) shows improved-but-imperfect physics (~70% physical-commonsense success in one benchmark) and remains non-action-conditioned.
Genie: the interactive turn
Genie (Bruce et al., DeepMind, arXiv:2402.15391, Feb 2024) fixed the action-conditioning gap from an unexpected direction — it learned actions without ever seeing them. Trained purely on unlabeled internet video:
- Stage 1: a VQ-VAE video tokenizer.
- Stage 2: with the tokenizer frozen, a latent action model co-trained with a MaskGIT-based, frame-level autoregressive latent dynamics model.
The result is the first generative interactive environment trained unsupervised: the learned latent actions are not physical actions, but they have consistent effects across inputs — “akin to learning the buttons on a new controller.” A user can press those buttons and steer the generation.
Genie 2 and Genie 3: from research artifact to product
Genie 2 (announced Dec 4, 2024) scaled the idea to a “large-scale foundation world model” generating playable 3D environments: prompt it with a single image (Imagen 3 output or a real photo), then a human or agent supplies a keyboard-and-mouse action at each timestep and the model simulates the next observation. DeepMind reported worlds consistent for up to one minute — with most published examples lasting 10–20 seconds, and real-time human play requiring a distilled variant at reduced quality. It was never publicly released.
Genie 3 (announced Aug 5, 2025) is DeepMind’s first world model allowing real-time interaction at full quality: navigable dynamic worlds from a text prompt at 24 fps, 720p, remaining largely consistent for a few minutes with visual memory extending back ~1 minute. (All figures are DeepMind self-reports — access was gated, so no independent benchmarks exist.) Availability tells the productization story: a limited research preview for a small cohort at announcement, then from January 29, 2026 powering “Project Genie” on Google Labs for US Google AI Ultra subscribers. Still no public API as of mid-2026.
Genie 3’s reach now extends beyond games: Waymo’s February 2026 Waymo World Model for driving simulation is built on it (see driving).
Veo-as-world-simulator (“zero-shot reasoner”) claims and SIMA 2: verification in progress.
← Latent prediction: the JEPA family3D & spatial world models →