Driving world models
Autonomous driving was one of the first commercial domains to bet on world models: a learned simulator can generate the rare, dangerous scenarios (near-misses, weather, erratic pedestrians) that real fleets almost never capture — conditioned on the ego-vehicle’s actions.
The GAIA lineage (Wayve)
GAIA-1 (arXiv:2309.17080, Sep 2023) framed driving world modeling as next-token prediction: ~9.4B parameters total — a 6.5B autoregressive transformer world model, a 2.6B video diffusion decoder, and a 0.3B image tokenizer (scaled up from a 1B version three months earlier). It takes video, text, and action inputs and generates realistic driving video (288×512) with fine-grained control over ego-behavior and scene features. Trained on ~4,700 hours of London driving; the world model took 15 days on 64 A100s — a useful calibration point for what “industrial scale” meant in 2023.
GAIA-2 (arXiv:2503.20523, Mar 2025) switched paradigms: a latent diffusion world model unifying multi-agent interaction, fine-grained control, and multi-camera consistency. It conditions on structured inputs — ego dynamics (speed, steering curvature), other agents as 3D bounding boxes, environment (weather, time of day), and road semantics (lanes, speed limits, crossings, traffic lights) — and generates spatiotemporally consistent video across up to 5 cameras at 448×960, trained on UK/US/Germany data. (Wayve’s own tech report, not peer-reviewed; “high-resolution” is the authors’ framing — 448×960 is modest by video-generation standards.)
The open ecosystem (OpenDriveLab)
Vista (arXiv:2405.17398, NeurIPS 2024) is the branch’s genuinely open model: weights on Hugging Face since June 2024, entire repo Apache-2.0, fine-tuned from Stable Video Diffusion. It supports multi-modal action conditioning (steering, speeds, commands, trajectories, goal points), long-horizon rollouts via latent replacement, and — author-demonstrated, not independently replicated — use as a generalizable reward function that evaluates actions without ground-truth labels.
Its companion line GenAD/OpenDV (arXiv:2403.09630, CVPR 2024 Highlight) contributed the data: OpenDV, 1,700+ hours of real-world YouTube driving across 244+ cities in 40 countries (~300× nuScenes’ video hours; full OpenDV-2K is 2,059 hours) — billed as the largest public driving video dataset as of March 2024 (NVIDIA’s Alpamayo dataset now contests the superlative).
The convergence event: Waymo × Genie 3
On February 6, 2026 Waymo announced the Waymo World Model: a generative simulation model built on Google DeepMind’s Genie 3, distinguished from video-only driving world models by generating multi-sensor output — camera and lidar — matched to Waymo’s hardware suite. It is an internal simulation tool, not a product or open model. The significance for the taxonomy: a general-purpose interactive world model (trained on 2D video) transferred into a safety-critical, 3D-sensor domain — the strongest evidence yet that the branches of this survey are merging.