Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Few-step generation — getting a sample in one or a few network evaluations — progressive, distribution-matching and adversarial distillation, consistency, flow-map, shortcut and mean-flow models, and fast ODE solvers. A subtopic, worn beside generative-modeling or inference-optimization.
8 of 449 SOTA documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| SOTA-206 v3 | Keep a multi-step sampling option in a few-step generative model | Song et al. (2023), LIT-093. A one-step model that cannot spend more compute has no quality dial: whatever it produces is what you get. Consistency models trade compute for quality at inference without retraining, and that property is worth preserving by design. | Active |
| SOTA-289 v2 | When the sampling budget is small, prefer uniform-state discrete diffusion with consistency distillation — masked diffusion cannot revise what it has already emitted | Proposed | |
| SOTA-392 v3 | To get a one-step sampler from a rectified flow, distil it; treat reflow as an optional extra pass, and keep the pre-reflow model for many steps | Liu, Gong and Liu (2022), LIT-636. On CIFAR-10, one-step FID is 378 from the base model, 6.18 after distillation alone, 12.21 after one reflow, and 4.85 after reflow and distillation (Table 1a). Distillation does most of the work. Reflow adds a further gain at the cost of a second training pass, and worsens many-step quality from 2.58 to 3.36. InstaFlow (LIT-796) contests it at text-to-image scale, starting from a diffusion model rather than a rectified flow. | Proposed |
| SOTA-394 v3 | When distilling a causal few-step video generator, take the teacher bidirectional, not causal | Yin et al. (2024), LIT-631. A block-causal 4-step student distilled from a bidirectional teacher beats one distilled from a causal teacher, at the same initialization, data and step count (Table 4). The causal teacher is also the weaker model before distillation, so the result shows the arrangement works and does not isolate why. | Proposed |
| SOTA-443 | Down-weight large residuals in consistency and MeanFlow training: use a Pseudo-Huber loss or an adaptive inverse-error weight, not plain squared L2 | Song and Dhariwal (2023), LIT-786, and Geng et al. (2025), LIT-784. When the regression target is the model’s own output under stop-gradient, squared L2 trains worse than a loss that limits the pull of large errors. iCT’s Pseudo-Huber beats squared L2 on CIFAR-10 (Fig. 2, curves). MeanFlow’s adaptive weight 1/(‖Δ‖² + c)^p takes one-step ImageNet-256 FID from 79.75 at p = 0 to 61.06 at p = 1 (Table 1e). Two groups, small settings, one run per arm. | Proposed |
| SOTA-446 | In consistency training, take the target from the current weights under stop-gradient, not from an EMA teacher | Song and Dhariwal (2023), LIT-786, §3.2. Consistency Models trained the student against an EMA of itself. iCT shows that on a point-mass data distribution the limiting objective with any teacher other than the student carries no information about the data (Prop. 1), and that setting the teacher’s EMA rate to zero improves CIFAR-10 FID with both LPIPS and squared L2 (Fig. 2a, curves). Keep an EMA of the weights for evaluation. Do not use it for the target. | Proposed |
| SOTA-447 | When distilling a large image or video model by continuous-time consistency, add a small DMD distribution-matching term | Zheng et al. (2025), LIT-783. Pure sCM distillation of 2B–14B text-to-image and text-to-video teachers loses small text, texture and temporal coherence. Adding DMD’s loss at weight 0.01, on the student’s own 1–4-step rollouts with a fake-score network updated 5–10 times per step, fixes it in samples. Four-step Wan2.1-1.3B VBench is 84.43 against a re-implemented DMD2’s 84.56. sCM’s own Fig. 7 (LIT-790) is why the consistency term stays primary: distribution matching alone loses recall. | Proposed |
| SOTA-448 | Without a pretrained teacher, train a one-step generator as a MeanFlow average-velocity model rather than by consistency training or a shortcut model | Geng et al. (2025), LIT-784. Train a network for the average velocity over an interval, regressed onto v − (t − r)·du/dt under stop-gradient, with du/dt from a Jacobian-vector product. From scratch on ImageNet-256 with DiT-XL/2, one-step FID is 3.43 after 240 epochs, against 10.60 for a shortcut model and 34.24 for iCT as Inductive Moment Matching re-ran it (Table 2). Those rows are copied, at other budgets. No arm compares distilling a teacher. | Proposed |