Agents & robotics: world models for control
This is the oldest branch — the model-based RL tradition where the world model exists to be planned in. The agent learns a dynamics model from experience, then trains or searches a policy inside the model instead of the expensive (or dangerous) real environment.
DreamerV3 — the canonical agent world model
DreamerV3 (Hafner, Pasukonis, Ba, Lillicrap) learns an RSSM world model from experience and trains an actor-critic policy from trajectories imagined inside the world model. (Precisely: the actor is trained purely in imagination; the critic keeps a small auxiliary replay-buffer loss, weight 0.3.)
The headline result: it masters 150+ tasks across Atari, DeepMind Control, Crafter, and Minecraft — including collecting diamonds in Minecraft from scratch with no human data — using a single fixed set of hyperparameters, outperforming specialized methods. Published in Nature, April 2, 2025 (Mastering diverse control tasks through world models, Nature 640:647–653, DOI 10.1038/s41586-025-08744-2; arXiv:2301.04104). Official MIT-licensed code: danijar/dreamerv3.
As of mid-2026 it remains the standard general model-based RL baseline, with only suite-specific exceptions (TD-MPC2, EfficientZero on particular budgets). MuZero sits in the same lineage — a learned dynamics model used for planning, without being given the rules.
Cosmos 3 — the convergence point
NVIDIA Cosmos 3 (released May 31, 2026) is what happens when the video-generative and control branches merge. The family — five open-weight models in the nvidia/cosmos3 collection:
| Model | Params | Role |
|---|---|---|
| Cosmos3-Nano | 16B | omnimodal world model, small |
| Cosmos3-Super | 64B (~133 GB BF16) | omnimodal world model, flagship |
| Cosmos3-Super-Text2Image | 64B | text→image specialist |
| Cosmos3-Super-Image2Video | 64B | image→video specialist |
| Cosmos3-Nano-Policy-DROID | 16B | vision-language robot policy |
Cosmos3-Super is a Mixture-of-Transformers — a 32B reasoner tower plus a 32B generator tower (the source of stray “32B” figures in press coverage). It is omnimodal in both directions: conditioned on text (≤4096 tokens), images, video (≤5 frames), audio (≤0.5 s), and action trajectories (16–400 frames); generating text, images, video, audio, and actions.
Licensing is notable: NVIDIA moved Cosmos from its bespoke Open Model License to OpenMDW-1.1 (openmdw.ai), the Linux Foundation’s permissive Open Model, Data & Weights license — unrestricted commercial use, no output restrictions, with a “Built on NVIDIA Cosmos” attribution requirement. Technical report: Cosmos 3: Omnimodal World Models for Physical AI (2026-06-22).
V-JEPA 2-AC (robot control via latent planning) belongs to this branch too — see latent prediction. UniSim, 1X’s world models, and Physical Intelligence: verification in progress.