few-step-generation.
5 of 378 NOTE documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| NOTE-043 v2 | DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling | Notices that the diffusion ODE is semi-linear, solves its linear part analytically, and applies a numerical method only to the intractable neural-network integral. Training-free, 10–20 function evaluations, 4.70 FID at 10 NFE on CIFAR-10, and DDIM turns out to be exactly the first-order case. | Read |
| NOTE-048 | Progressive Distillation for Fast Sampling of Diffusion Models | Repeatedly halves a diffusion sampler’s step count by distilling two teacher steps into one student step, from 8192 down to 4 at FID 3.0 on CIFAR-10, for no more total compute than training the original model. Also where v-prediction comes from — and the reason for it is that ε-prediction’s implied x̂ blows up as signal-to-noise goes to zero. | Read |
| NOTE-053 v2 | Consistency Models | Trains a model to map any point on a diffusion trajectory directly to its origin, so one network evaluation generates a sample — FID 3.55 on CIFAR-10 and 6.20 on ImageNet-64 in one step. Trainable either by distilling a diffusion model or from scratch, and the from-scratch mode is what makes it a model family rather than a distillation trick. | Read |
| NOTE-335 | Self Forcing | Training a few-step autoregressive video model on its own rollouts, with a distribution-matching loss on the whole clip, beats teacher- and diffusion-forced training in matched few-step comparisons (VBench 84.31 against 82.76 and 82.32 chunk-wise), and does not drop when moved to frame-wise generation, where the baselines do. It runs at 17 fps with 0.69s latency on one H100, and its parity with the slower 1.3B base rests on a 14B teacher: the SiD variant, with a 1.3B teacher, scores 84.07 against the base’s 84.26. | Read |
| NOTE-343 v2 | CausVid | Distilling a bidirectional video DiT into a 4-step block-causal student with distribution matching gives a streaming generator: 1.3s to the first frame and 9.4 fps, against 219.2s and 0.6 fps for the teacher. At equal initialization a bidirectional teacher makes a better causal student than a causal one (VBench 94.7 / 64.4 / 30.1 against 91.9 / 61.7 / 28.2). The causal teacher is also the weaker model, so the paper’s error-accumulation explanation is not isolated. | Read |