Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Few-step generation — getting a sample in one or a few network evaluations — progressive, distribution-matching and adversarial distillation, consistency, flow-map, shortcut and mean-flow models, and fast ODE solvers. A subtopic, worn beside generative-modeling or inference-optimization.
17 of 737 LIT documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| LIT-067 v3 | Progressive Distillation for Fast Sampling of Diffusion Models | Salimans et al. (2022), ARXIV-2202.00512. Faster sampling through distillation. | Active |
| LIT-076 v3 | DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps | Lu et al. (2022), ARXIV-2206.00927. Fast sampling for diffusion models. | Active |
| LIT-093 v3 | Consistency Models | Song et al. (2023), ARXIV-2303.01469. Single-step generation. | Active |
| LIT-629 v3 | Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion | Huang et al., Adobe Research and UT Austin (2025), ARXIV-2506.08009. Self Forcing trains an autoregressive video model on its own rollouts, conditioning on self-generated context and scoring the whole clip with a distribution-level loss. On Wan2.1-1.3B it beats teacher-forced and diffusion-forced training under every objective tried, and holds quality where they degrade frame by frame. It reaches 17 fps at 0.69s latency on one H100. The comparison against slower models is confounded by a 14B teacher. | Active |
| LIT-631 v3 | From Slow Bidirectional to Fast Autoregressive Video Diffusion Models | Yin et al., MIT and Adobe (2024), ARXIV-2412.07772. CausVid distills a bidirectional video DiT into a 4-step block-causal student with distribution matching, cutting first-frame latency from 219s to 1.3s at 9.4 fps. The controlled result is that a bidirectional teacher makes a better causal student than a causal one. The authors attribute this to error accumulation, but the causal teacher is also the weaker model, and the ablation does not separate the two. The paper does not claim real time, and it reports that the student flickers more and is less diverse. | Active |
| LIT-643 | One-step Diffusion with Distribution Matching Distillation | Yin et al., MIT and Adobe (2023), ARXIV-2311.18828. DMD distills a diffusion model into a one-step generator by descending an approximate KL(p_fake ‖ p_real), whose gradient is the difference of a frozen “real” score and an online “fake” score. It adds an LPIPS regression loss onto precomputed teacher ODE outputs. It reaches 2.62 FID on ImageNet-64 and 11.49 on zero-shot COCO-30k. The ablation that makes the regression loss look necessary ran its no-regression arm at a different learning rate, and its mode-collapse evidence is qualitative. | Active |
| LIT-646 | Improved Distribution Matching Distillation for Fast Image Synthesis | Yin et al., MIT and Adobe (2024), ARXIV-2405.14867. DMD2 drops DMD’s regression loss. It stabilizes training by updating the fake score five times per generator step, adds a GAN loss on real images, and trains multi-step students on their own simulated inputs. The ImageNet ablation is clean: removing the loss costs 2.62 → 3.48 FID, and the update rule restores 2.61. The student is slightly less diverse than its teacher (LPIPS diversity 0.61 against 0.64). The paper does not attribute this to the reverse KL. | Active |
| LIT-770 v2 | DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation | Yu et al., Texas A&M and ByteDance (2026), ARXIV-2610.02188. DMAD replaces DMD’s fake-score critic with a discriminator and trains the student on a linear loss on its logit. At the unrestricted discriminator optimum that gradient equals DMD’s idealized one (App. A, checked); nothing bounds it off the optimum, where training runs with one discriminator step per student step. One-step ImageNet-64 FID is 1.24 against DMD2’s 1.28, at 570K iterations, and 1.04 with an ImageNet-pretrained discriminator scored only by FID. Four-step SDXL reaches 14.47 FID at lower CLIP. Wan2.1 VBench margins are 0.1–0.3 points. No diversity metric and no seeds are reported. | Active |
| LIT-775 | Inductive Moment Matching | Zhou, Ermon and Song, Luma AI and Stanford (2025), ARXIV-2503.07565. A one-network, from-scratch few-step model that maps the interpolant’s marginal at t to the one at s < t. It is trained by matching, with MMD over groups of samples, the distributions reached from t and from a nearby r. Consistency models are its one-sample, first-moment case, and that case collapses on ImageNet. ImageNet-256 FID is 8.05 at one step and 1.99 at eight, with guidance at sampling time, so 16 evaluations. Training is 1.2M steps at batch 4,096. CIFAR-10 2-step 1.98. One run per row. | Active |
| LIT-783 | Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency | Zheng et al., Tsinghua and NVIDIA (2025), ARXIV-2510.08431. sCM made to run on 14B text-to-image and text-to-video models, with a FlashAttention-2 JVP kernel under FSDP and context parallelism. Pure sCM loses fine detail and temporal coherence there. Adding a DMD loss at weight 0.01 (rCM) fixes it. Four-step Wan2.1 VBench is 84.43 at 1.3B against a re-implemented DMD2’s 84.56, and 84.92 at 14B; GenEval tracks the teacher at 1–4 steps. The diversity advantage over DMD2 is shown in samples only. One run per row. | Active |
| LIT-784 | Mean Flows for One-step Generative Modeling | Geng, Deng, Bai, Kolter and He, CMU and MIT (2025), ARXIV-2505.13447. Train a network for the average velocity u(z, r, t) over an interval, using the identity u = v − (t − r)·du/dt with du/dt from a Jacobian-vector product and the target under stop-gradient. With r = t it is flow matching. From scratch on ImageNet-256 with DiT-XL/2 and guidance folded into the target, one-step FID is 3.43 at 240 epochs and two-step 2.20 at 1,000. The best one-step baselines it beats, Shortcut 10.60 and IMM 7.77, are copied rows at other budgets. Behind iCT on CIFAR-10. One run per row. | Active |
| LIT-786 | Improved Techniques for Training Consistency Models | Song and Dhariwal, OpenAI (2023), ARXIV-2310.14189. Consistency training from random weights, without a teacher or LPIPS. The teacher network takes no EMA, because with one the limiting objective carries no information about the data. A Pseudo-Huber loss replaces LPIPS, whose ImageNet features can leak into FID. The step count doubles on a schedule, and noise levels are drawn log-normally. One-step FID is 2.51 on CIFAR-10 and 3.25 on ImageNet-64, two-step 2.24 and 2.77, beating consistency distillation. Ablations are CIFAR-10 curves, one run each. | Active |
| LIT-787 | One Step Diffusion via Shortcut Models | Frans, Hafner, Levine and Abbeel, UC Berkeley (2024), ARXIV-2410.12557. A flow-matching network also conditioned on step size d, trained in one run: the flow-matching loss at d = 0, and a target for step 2d built from two of its own d-steps. On DiT-B at matched compute, one model gives 6.9 / 13.8 / 20.5 FID at 128 / 4 / 1 steps on CelebA-HQ-256 and 15.5 / 28.3 / 40.3 on ImageNet-256. Progressive distillation is better at one step (14.8 and 35.6) and cannot sample at many. The re-run consistency baselines are far below their published numbers. DiT-XL reaches 10.6 in one step. One run per cell. | Active |
| LIT-790 | Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models | Lu and Song, OpenAI (2024), ARXIV-2410.11081. Continuous-time consistency models had been unstable. A trigonometric flow parameterization, identity time input with positional embeddings, normalized tangents, learned loss weights and a Flash Attention JVP make them train to 1.5B parameters. Continuous time beats every discrete step count tried. Two-step FID is 2.06 on CIFAR-10, 1.48 on ImageNet-64 and 1.88 on ImageNet-512, against teachers at 2.01, 1.33 and 1.73. Every model, “consistency training” included, starts from the pretrained diffusion model. | Active |
| LIT-791 | Improving the Training of Rectified Flows | Lee, Lin and Fanti, CMU and Microsoft Research (2024), ARXIV-2405.20320. One reflow round is enough, argued informally, and the 2-rectified flow then trains better with a U-shaped timestep distribution and an LPIPS-Huber loss. Started from EDM, with no separate distillation stage, one-step FID is 3.07 on CIFAR-10 and 4.31 on ImageNet-64. Its own table lists DMD at 2.62 on ImageNet-64. There is no distillation-alone arm from the same teacher, and many-step quality is not compared with the pre-reflow model. | Active |
| LIT-793 | Flow map matching with stochastic interpolants: A mathematical framework for consistency models | Boffi, Albergo and Vanden-Eijnden, Courant Institute (2024), ARXIV-2406.07507. Learn the two-time flow map X_{s,t} of a probability-flow ODE. Distillation from a velocity field can use a Lagrangian loss (new) or an Eulerian one, which is the continuous-time limit of consistency distillation; both bound a Wasserstein distance. Consistency training’s update is shown not to descend any loss. On CIFAR-10 the Lagrangian loss distils far better than the Eulerian: FID 7.13 against 48.32 at two steps, teacher 5.53. Direct training on ImageNet-32 gives 16.90 FID at four steps. Small models; no one-step image table. | Active |
| LIT-796 | InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation | Liu, Zhang, Ma, Peng and Liu, UT Austin and Helixon (2023), ARXIV-2309.06380. Reflow Stable Diffusion once, then distil the reflowed model to one step with an LPIPS regression. Against distilling SD 1.4 directly, at the same 100K steps and 3.2M generated pairs, reflow first gives one-step FID 31.0 against 40.9 on COCO-2017-5k and 20.0 against 34.6 on COCO-2014-30k. The scaled model, InstaFlow-0.9B, scores 23.4 and 13.10. The teacher is a diffusion model, not a rectified flow, and the distiller is a regression, not distribution matching. One run per row. | Active |