Chen, Marti Monso, Du, Simchowitz, Tedrake, Sitzmann, MIT and TU Munich (2024; NeurIPS 2024) — ARXIV-2407.01392
Key takeaways
- Noise as partial masking. Zero noise is an unmasked token and full noise a masked one. Training with an independent noise level per token teaches the model every conditional between clean history and noisy future. The paper proves the objective bounds the likelihood of all training subsequences
- Sampling is a 2D schedule over sequence position and noise level, chosen at inference without retraining. Examples are autoregressive (history clean), full-sequence (all equal) and “zig-zag” (near future cleaner than far future)
- Stable long rollouts: condition on history that is treated as
slightly noisy (
0 < k ≪ K), so small prediction errors look like training-distribution noise. On Minecraft and DMLab, rollouts stay coherent to 1,000 frames where a teacher-forced next-frame diffusion model and a causal full-sequence model diverge (Figure 3, qualitative) - Planning (D4RL Maze2D, Table 1): average reward 141.7 (single task) and 146.2 (multi-task), against Diffuser’s 119.5 and 129.4. Diffuser needs a hand-coded PD controller and falls to 8.7 and 20.6 when executing its own actions. Monte Carlo Guidance adds about 12 points on single-task
- A real-robot task needing memory: 80% success, where a memoryless diffusion policy fails. 76% with occluded or distracted observations, against 48% for a next-frame baseline
Standing in the anthology
Filed from #163 (“diffusion forcing”), and sources SOTA-333 for
the video-rollout use.
What travelled and what was measured. The claim that it “stably rolls out past the training horizon” is the one people cite. In this paper it is Figure 3 and a project-page video, with no FVD or other number in the main text, on a small RNN and two game datasets. The measured results are the planning table and the robot success rates. The authors list transformer-scale implementation and scaling as open.
Used as a baseline by Self Forcing. Self Forcing (LIT-629) trains diffusion-forcing baselines on a Wan2.1-1.3B transformer and scores them on VBench (its Table 2). Many-step diffusion forcing scores 82.95 chunk-wise and 77.24 frame-wise, slightly below teacher forcing’s 83.58 and 80.34. Four-step diffusion forcing with DMD scores 82.76 and 80.56, and Self Forcing’s training on its own rollouts 84.31 and 84.26. Those baselines are run with clean context at inference, the setting Self Forcing describes in its §1, not the slightly noisy history this paper recommends for long rollouts, so the configuration that carries this paper’s stability claim is not the one that was tested.
Read — NOTE-298.