Lu et al. (2022) — ARXIV-2206.00927
Key takeaways
- Notices the diffusion ODE is semi-linear — a closed-form linear part plus the neural network — and solves the linear part analytically, leaving a change of variables to log-SNR that reduces the rest to an exponentially weighted integral of the noise-prediction model. Prior fast samplers handed the whole ODE to a black-box solver and paid for approximating the analytic half
- 4.70 FID at 10 NFE, 2.87 at 20 on CIFAR-10; 4–16× over previous training-free samplers. First-, second- and third-order versions with convergence-order guarantees, plus uniform-in-λ and adaptive step schedules
- DDIM is exactly DPM-Solver-1 (§4.1) — the first-order case. That retroactively explains DDIM and bounds it as the least accurate member of a family
- Proposition 3.1: the exact solution is invariant to the noise schedule between endpoints. The schedule does not change the solution being approximated, only where the steps land
- Works on pretrained discrete-time models by feeding non-integer times, which the authors report works and hypothesise is due to smooth positional time embeddings — a hypothesis holding up half the paper
Standing in the anthology
Read — NOTE-043.
The primary source of SOTA-203 — sample with a higher-order ODE solver on weights you already trained. Filed by the #121 audit; NOTE-043 recorded that nothing was sourced here, which was that pass’s policy rather than a finding.
The reading sharpens how SOTA-188’s source should be read. LIT-075’s
reading records EDM’s sampler as Heun’s second-order method on a ρ = 7 grid
between σ_min = 0.002 and σ_max = 80, with those grid parameters tuned.
Proposition 3.1 says the exact solution does not depend on the schedule between
endpoints — so EDM’s ρ chooses where approximation error lands, not what is
being approximated. That is a sharper reading than EDM offers of itself, and
the record now has both papers in view.
The bound on that family came from its own authors. DPM-Solver++ (LIT-676) reran this paper’s solvers under classifier guidance at scale 8.0 on ImageNet 256×256: at 10 function evaluations DPM-Solver-2 scored FID 114.62 and DPM-Solver-3 164.74, against 13.04 for DDIM, so the first-order member this paper calls the least accurate was the best of the three there. Its multistep second-order DPM-Solver++(2M) reached 9.10 at 20 evaluations.