IFM releases PAN 2 world-model weights and training code

PAN 2 gives researchers an open video simulator for exploring what might happen after an action. The Institute of Foundation Models at MBZUAI announced the release on November 10, with model weights plus training and inference code available. Its practical attraction is the ability to inspect and adapt the simulation pipeline. Its biggest constraint is reliability: a plausible-looking outcome can still skip the action that was supposed to cause it. IFM’s announcement sets out the release.

The launch date should not be confused with the first appearance of every artifact. The repository’s citation metadata says October, and the vLLM recipe carries a November 5 update date. November 10 is the dated public announcement used here.

How PAN 2 turns an action into video

A language model first describes the next state of a scene, including timed events, actors and camera movement. A 29 billion parameter diffusion model supplies the visual details and motion. This division lets an agent propose a plan, inspect its simulated outcome and then reconsider what to do. The technical explanation calls the approach Generative Latent Prediction.

One BF16 diffusion checkpoint handles text-to-video and image-to-video. It uses a frozen Qwen3.5-9B text encoder and a Wan 2.2 VAE from their respective repositories. The model card specifies 704 by 1248 output at 24 frames per second, with clips up to 20 seconds.

IFM’s demonstration page groups examples around action timing, material behavior and camera movement. Treat these as selected vendor examples. They offer useful ideas for a test set, but watching a successful clip does not establish how often the same request fails.

The prompt format is part of the system

The inference documentation expects structured captions describing the scene and its timeline. Prompt rewriting is a separate step: the generator does not automatically turn an ordinary instruction into that representation. IFM supplies an offline rewriter and a prompt-writing skill for coding agents. A local instruction-following model can perform the rewrite.

That separation is useful for debugging. Save the original request, the rewritten caption and the resulting video. If an action disappears, inspect whether the reasoner omitted it or the simulator ignored it. Changing the prompt writer and the video model together would make that diagnosis harder.

Hardware and running cost need a separate budget

The vLLM serving recipe reports roughly 119 seconds to generate a 13-second clip on four B300 GPUs. That is a specific serving configuration, not a real-time promise. For its longest 481-frame workload, the recipe lists a 107.4 GiB peak on one GPU with at least 141 GB of memory. A 96 GB RTX PRO 6000 configuration uses layerwise offloading and reports an 84.4 GiB peak, with substantial host memory also required.

The native repository launcher takes a different approach: each GPU worker holds the complete diffusion model and processes its own prompts. Adding workers raises throughput rather than automatically splitting one model across smaller cards. Follow the recipe for the inference stack you actually choose.

There is no universal dollar-per-video price in these materials. A useful estimate needs your GPU rental rate, measured wall time, prompt-rewriting cost and retry rate. Record cold starts and decoding as well as denoising. A successful first run establishes that the pipeline executes, not that the setup is affordable for thousands of simulations.

What the Apache 2.0 release includes

IFM licenses its own code and weights under Apache 2.0. The repository’s third-party notice separately identifies Wan 2.2, Qwen3.5-9B and Diffusers as Apache 2.0 components. Preserve their applicable notices and review the complete terms before redistribution. These statements do not establish rights to arbitrary footage used in a new training run.

The data preparation guide describes bringing local video latents and caption CSVs, with optional image archives. It explains how to encode and normalize that material before training. This is a documented training workflow; it should not be described as a release of the original training corpus. No downloadable original corpus was verified for this report.

The useful benchmark is where the plan breaks

IFM reports 75.1% completion of individual steps and 34.0% completion of entire plans. Those measurements test different things: a system can execute many isolated actions while rarely preserving every step in sequence. The reported Physics-IQ setting also uses Claude Opus 5.5 as the conceptual reasoner, so it is a pipeline result rather than a score for the diffusion weights alone.

The model card notes prompt sensitivity, cross-clip drift and coarse text control. It also says the study has not measured how faithfully the system simulates plans that should fail. That gap matters when an agent must reject a dangerous or impossible plan rather than simply illustrate a successful one.

Our Cosmos 3 simulation guide covers a different integration decision: choosing observations and controls for an existing simulation workflow. PAN 2’s distinct question is whether a language-described plan survives the handoff into generated video. For a pilot, hold the starting scene fixed, change one action, and score intermediate consequences as well as the final image. Include deliberately impossible sequences. ByteForward has not run PAN 2 or independently reproduced IFM’s results.

ByteForward conceptual illustration using the official PAN 2 wordmark. This is editorial artwork, not model output.