Huang et al., Adobe Research and UT Austin (2025) — ARXIV-2506.08009
Key takeaways
-
The idea is exposure bias, fixed at training time. A teacher-forced model trains on clean context and runs on its own output. The mismatch compounds with every autoregressive step. Self Forcing runs the rollout during training, with the KV cache built from its own outputs, and applies a sequence-level distribution-matching loss to the result (§3).
-
The controlled comparison. All rows share the Wan2.1-1.3B base. The 4-step rows also share ODE initialization, prompts and the DMD teacher. The many-step rows are 50×2-step models fine-tuned on 70k 14B-generated videos (§4). VBench totals, chunk-wise / frame-wise (Table 2):
Method Chunk-wise Frame-wise Self Forcing + DMD 84.31 84.26 Diffusion forcing + DMD (CausVid replicated) 82.76 80.56 Teacher forcing + DMD 82.32 78.12 Many-step teacher forcing 83.58 80.34 Many-step diffusion forcing 82.95 77.24 The baselines degrade when moved from chunk-wise to frame-wise (more autoregressive steps), and Self Forcing does not. The advantage holds with DMD, SiD and GAN objectives (84.31, 84.07, 83.88).
-
The cost is affordable. Backprop is truncated to one randomly chosen denoising step and the KV cache is detached. Per-iteration time is comparable to teacher forcing, and DMD converges in about 1.5h on 64 H100s (Fig. 6, App. A).
-
Speed. Chunk-wise: 17.0 fps, 0.69s first-frame latency. Frame-wise: 8.9 fps, 0.45s. Base Wan2.1-1.3B: 0.78 fps, 103s. All measured on one H100 at 832×480 (Table 1).
-
A rolling KV cache needs matching training. Without masking the first chunk during training, the rolling cache produces “severe flickering artifacts” (§3.4, App. B, qualitative).
Where the hedges are
Per DP-010:
- “Matching or even surpassing” slower models (abstract) compares a student distilled from Wan2.1-14B against the 1.3B base: 84.31 against 84.26 on VBench, within noise. The body says “slightly better visual quality” (§4). The SiD variant, which uses the 1.3B model as its score teacher, scores 84.07, below the base.
- Exposure bias is inferred, not measured. Every number is a VBench score on about 5s clips. The mechanism is argued from the chunk-to-frame degradation pattern. There is no quality-over-time curve, although CausVid has one.
- It works only inside the training window. “Quality degradation remains observable when generating videos substantially longer than those seen during training” (§5).
- “Real-time” is defined in §4 as throughput above playback rate and latency under a perceptual threshold. The frame-wise variant, at 8.9 fps against 16 fps playback, does not meet it.
- The argument against noised context is asserted. Methods that noise the context at inference, Diffusion Forcing among them, “sacrifice temporal consistency… and do not fundamentally resolve the exposure bias problem” (§1). No experiment tests this. The Diffusion Forcing baselines in Table 2 use clean context. The paper states this as the general AR inference setting (§1, Fig. 2c), though not per row in the ablation.
Standing in the anthology
This was the record’s first document on exposure bias as a named problem, although the term predates video by a decade. It was named by MIXER (LIT-647), for RNNs trained with cross-entropy. The two neighbouring RNN papers now filed with it, Scheduled Sampling (LIT-649) and Professor Forcing (LIT-642), address the same train/test mismatch and never use the term. The paper says the method is “general and can be applied to other sequence domains, especially where the data is continuous” (§5). That is asserted.
What it builds on. The recipe is CausVid’s (LIT-631): ODE initialization, a few-step block-causal student and DMD. What changes is where the student’s context comes from. CausVid trains on noised ground-truth context, diffusion forcing, which Table 2 replicates as “DF + DMD”. Self Forcing trains on its own rollouts. On the same Wan-1.3B base at the same 17.0 fps, the official CausVid scores 81.20 on VBench against 84.31 (Table 1). The base is Wan2.1-1.3B (LIT-619), whose weights initialize the student. For DMD the teacher is Wan2.1-14B. Wan’s causal VAE also shapes the rolling cache: it encodes the first latent frame without temporal compression, so training has to mask that chunk (§3.4).
What it is compared against. Wan2.1-1.3B (LIT-619) is the main comparison as well as the base: 84.26 on VBench at 0.78 fps and 103 s latency, against 84.31 at 17.0 fps and 0.69 s (Table 1). In the user study of 1,003 MovieGenBench prompts, one rater each, Self Forcing was preferred over Wan2.1 62.7% of the time. LTX-Video (LIT-618) is the other diffusion baseline, chosen as “known for efficiency”. Measured here on one H100, it reaches 8.98 fps with 13.5 s first-frame latency and scores 80.00 (Table 1). The paper names LTX-Video as a model that claims real time “based solely on throughput”, which is why its own definition adds latency. The diffusion-forcing rows of Table 2 apply Diffusion Forcing’s (LIT-554) per-frame independent noise levels, moved from that paper’s convolutional RNN to the Wan2.1-1.3B transformer. They score 82.95 / 77.24 many-step and 82.76 / 80.56 with DMD, against Self Forcing’s 84.31 / 84.26. They run with clean context, not the noised history Diffusion Forcing recommends for long rollouts.
It bears on SOTA-333, which recommends per-token independent noise in training plus noised history at rollout. The two halves come out differently:
- Training half: tested, with a split result. Table 2 is a controlled transformer-scale comparison by another group. In the many-step rows, diffusion forcing loses to teacher forcing (82.95 against 83.58 chunk-wise, 77.24 against 80.34 frame-wise). After DMD distillation it wins (82.76 against 82.32, 80.56 against 78.12). Both lose to Self Forcing.
- Inference half: not tested. The noised-history rollout appears only in an uncited sentence. This paper contests it without measuring it.
Read in full as NOTE-335.