Source
Huang et al. (2025), LIT-629 — Self Forcing, Table 2.
The claim
A causal generator trained with teacher forcing sees clean context in training and its own imperfect output at inference. The gap compounds with every autoregressive step. Close it in training. Run the rollout during training, build the KV cache from the model’s own outputs, and apply a loss to the generated clip as a whole.
The recommendation is Self Forcing’s (LIT-629). The paper proposes training the few-step causal model on its own autoregressive rollouts and scoring the generated clip as a whole with a distribution-level loss. It compares this against the alternatives on Wan2.1-1.3B, in two matched pairs (Table 2, VBench total, chunk-wise / frame-wise):
| Training | Chunk-wise | Frame-wise |
|---|---|---|
| Self Forcing + DMD | 84.31 | 84.26 |
| Diffusion forcing + DMD | 82.76 | 80.56 |
| Teacher forcing + DMD | 82.32 | 78.12 |
| Many-step teacher forcing | 83.58 | 80.34 |
| Many-step diffusion forcing | 82.95 | 77.24 |
The pattern is the claim. Every baseline loses quality moving from chunk-wise to frame-wise generation, which means more autoregressive steps. Self Forcing does not. The advantage holds with three objectives: DMD 84.31, SiD 84.07, GAN 83.88.
It is affordable. Backprop is truncated to one randomly chosen denoising step and the KV cache is detached. Per-iteration cost is comparable to teacher forcing, and DMD converges in about 1.5 hours on 64 H100s (Fig. 6, App. A).
Conditions
- Measured on short clips. Every number is a VBench score on clips about 5 seconds long. “Quality degradation remains observable when generating videos substantially longer than those seen during training” (§5). The practice fixes the gap inside the training window, and does not claim to remove it beyond.
- The mechanism is inferred. Exposure bias is argued from the chunk-to-frame degradation pattern. No quality-over-time curve is shown.
- The table is two setups. The many-step rows are 50×2-step models fine-tuned on 70K videos generated by Wan2.1-14B. The distilled rows are 4-step students. Compare within a pair.
- The distilled teacher is larger than the base. The DMD rows use Wan2.1-14B as the score teacher. The SiD variant, with the 1.3B model as teacher, scores 84.07, still above every baseline.
- Relation to SOTA-333. That practice recommends per-token noise in
training and noised history at rollout. Self Forcing argues against the
rollout half without testing it, and SOTA-333 records it as
contested_by. The two are alternative answers to the same train-test gap. Nothing has compared them head to head. - Beyond video, the claim is asserted. The paper says the method “can be applied to other sequence domains, especially where the data is continuous” (§5).