Get Started

World Models & Video Diffusion

A video model that predicts the next seconds of a scene is a world simulator. Condition that prediction on actions and you have a learned game engine.

75 minutesLearn + BuildPythonMembers

Learning Objectives

  • Explain the difference between a pure video generation model (Sora 2) and an action-conditioned world model (Genie 3, DreamerV3)
  • Describe a video DiT: spatio-temporal patches, 3D position encoding, joint attention across (T, H, W) tokens
  • Trace how a world model plugs into robotics: VLM plans → video model simulates → inverse dynamics emits actions
  • Pick between Sora 2, Genie 3, Runway GWM-1 Worlds, Wan-Video, and HunyuanVideo for a given use case (creative video, interactive sim, autonomous-driving synthesis)