Wan Team, Alibaba (2025) — ARXIV-2503.20314
Key takeaways
- It discloses a recipe. Weights, code, data pipeline, VAE, training schedule, parallelism and inference tricks are all published for a model that its authors’ own benchmark and human preference put ahead of the closed systems it names. It is the video counterpart of an open language model report.
- Backbone. A DiT with cross-attention to umT5 text embeddings, full spatio-temporal self-attention over latent patches, and a rectified-flow velocity loss with logit-normal timesteps, following LIT-449. One timestep MLP is shared across all blocks, with a learned bias per block, instead of a separate adaLN MLP in every block. This saves about 25% of parameters.
- Wan-VAE. A 127M 3D causal VAE compresses 4× in time and 8×8 in space into 16 channels, and the first frame is compressed spatially only, so images and videos share one latent. GroupNorm is replaced with RMSNorm to keep it causal, which lets a chunked feature cache encode and decode arbitrarily long video in bounded memory. It is inflated from a 2D image VAE rather than trained from scratch. Treating the first frame on its own follows MAGVIT-v2 (LIT-623), which the report cites for it. Wan keeps that causal, first-frame-separate design and replaces MAGVIT-v2’s lookup-free quantized tokens with a continuous 16-channel latent.
- The curriculum. It starts with 256px text-to-image pre-training, then joint image and video training at 192px, 480px and 720px. The reason given is systems, not quality. At 81 frames of 720p, throughput and memory force small batches, and the report attributes training instability to the resulting spikes in gradient variance.
- Attention dominates at video sequence lengths. Sequences run to hundreds of thousands of tokens, and at 1M tokens attention is up to 95% of step time. The response is FSDP for parameters and a 2D context parallelism, Ring Attention outside and Ulysses inside, in place of tensor parallelism. At 256K tokens on 16 GPUs this cuts communication overhead from over 10% to under 1%. Activation offloading is preferred over checkpointing because compute grows quadratically in sequence length while memory grows linearly, so transfers can hide behind compute.
- Image-to-video is a channel-concatenation recipe. The conditioning frame is padded with zero frames and VAE-encoded. It is concatenated on the channel axis with the noise latent and a binary mask of which frames are given, and a zero-initialized projection absorbs the extra channels. The same mask trains image-to-video, continuation, first-and-last-frame interpolation and random-frame interpolation as one task. A CLIP image embedding enters through decoupled cross-attention in fine-tuning only. Training clips are kept only when the first frame’s SigLIP features are close to the mean of the rest. Early runs failed to learn stable image-driven generation without that filter.
- Where the hedges are. Per DP-010:
- The abstract says the 14B model “demonstrates the scaling laws of video generation with respect to both data and model size”. Nothing in the body plots either axis. The evidence is two model sizes.
- All three ablations run on the 1.3B model, on text-to-image, and two of them are judged mainly by training loss. The VAE ablation (Table 5) and the Qwen-VL arm of the text-encoder ablation (Table 6) report FID. On Table 6, Qwen-VL-7B’s second-last layer scores 42.91 against umT5’s 43.01. umT5 was kept for its size, not because it generated better.
- “Minimal contamination (< 10%) by generated images can significantly degrade the performance” is asserted with no experiment.
- The human-preference comparisons use anonymized opponents. The named comparisons are Wan-Bench, the authors’ own, and VBench, a public leaderboard. On image-to-video matching, Wan loses to CN-TopA (−4.2%, Table 7).
Standing in the anthology
As a teacher. DMAD (LIT-770) distils Wan2.1 at 1.3B and 14B into four-step students and reports VBench totals of 84.70 and 85.15 against the teacher’s 83.02 and 83.59 as it retested them.
rCM (LIT-783) distils the same two Wan2.1 models by score-regularized continuous-time consistency, to one, two and four steps. VBench at 1.3B is 84.43 at four steps, 84.09 at two and 82.65 at one, against the teacher’s 83.02; at 14B it is 84.92, 85.05 and 83.02 against 83.58. One-step video is blurry, and the 1.3B student runs at 14.6 frames per second against the teacher’s 0.72.
This is the report LIT-218 said the record was missing. LIT-218’s note ends by asking for “a frontier video training report with a disclosed recipe — the video equivalent of what LIT-139 or LIT-131 are for language”. Before this note, the record held the backbone (LIT-448) and the objective (LIT-449), but no video model built from them.
It is filed as a source and has not been mined yet. Candidate practices, each of which would need its evidence checked against this text first:
- Pre-train a video diffusion model on images at low resolution, then raise resolution before adding length. The report justifies this by throughput, and it has no ablation. The image-first half is now SOTA-386, sourced from the earlier controlled studies. Wan is adoption of it, not evidence. The resolution-then-length ordering is still unfiled.
- Condition image-to-video by channel-concatenating a masked latent through a zero-initialized projection, and filter clips by how close the first frame is to the rest. This is the report’s most transferable engineering.
- Use context parallelism rather than tensor parallelism once attention dominates. This bears on the record’s distributed-training practices, and the 10%→1% figure is measured.
It is adoption evidence for SOTA-187, not a source for it: Wan trains in a learned latent and does not test that choice against anything else (DP-005). Its VAE ablation compares two latents (reconstruction loss against diffusion loss), not latent against pixel.
What it was measured against. In the report’s own tables, Wan 14B scores 86.22 on VBench (Table 4), against 84.28 for Sora (LIT-652), 83.24 for HunyuanVideo (LIT-620) and 82.17 for CogVideoX’s 1.5 release, CogVideoX1.5-5B (LIT-622). On Wan-Bench (Table 2) it scores 0.724 against Sora’s 0.700 and HunyuanVideo’s 0.673, though it trails Sora on large motion, 0.415 against 0.482. The VAE comparison (Fig. 7: PSNR against speed, 200 clips of 25 frames at 720×720) includes HunyuanVideo’s and CogVideoX’s VAEs at the same 4×8×8, 16-channel setting and Step-Video’s (LIT-624) at 8×16×16 with 64 channels. Only the claim that Wan-VAE reconstructs 2.5× faster than HunyuanVideo’s VAE is given as a number, and the rest is plotted. Later, Self Forcing (LIT-629) used Wan2.1-1.3B as both base and baseline. The base runs at 0.78 fps with 103 s latency and scores 84.26 on VBench. The distilled model, with a 14B teacher, runs at 17.0 fps and scores 84.31, and its user study reports the distilled model preferred over Wan2.1.
Tags. generative-modeling comes first because the report’s subject is
the generator. vision-and-graphics is there because this is a visual
foundation model. representation-and-encoding covers Wan-VAE, a learned
latent designed for its compression ratio and causality. data-pipeline
covers a third of the report, including the motion-quality tiers.
distributed-optimization covers the parallelism and memory analysis.
multimodal-learning covers the text-encoder ablation and the image branch
of image-to-video. The FP8 attention work (§4.4.3) and the diffusion cache
(§4.4.2) are real but each takes about a page, so they are left as candidates
for numerics-and-precision and inference-optimization if that material
is ever mined. No new topic was needed: the vocabulary already names both
the generator and the modality.
Read in full as NOTE-333.