Source
Zhang, Zhang, Zardini, Amin and Ozdaglar (2026), LIT-490 §§3–6 — read as NOTE-239.
What it explains
| practice | what it says to do | what this says is going on |
|---|---|---|
| SOTA-301 | apply the objective gradient before the denoiser, not after | you are not adding a prior to an optimizer, you are running an optimizer — the denoiser is the projection step, so the ordering is not a preference but the difference between projected and unprojected gradient descent |
The account
A diffusion sampler already computes D(x) = E[x₀ | x] at every reverse
step, via Tweedie–Miyasawa from the score. If the clean data lie on or near a
structured set C, that expectation is pulled toward C — which is what a
projection does.
The paper makes the analogy quantitative in three cases and the cases were
chosen to span what “feasible set” usually means: a linear subspace
(latent structure), a compact convex set (explicit constraints, possibly
nonsmooth), and a compact smooth submanifold (the generic
low-dimensional-data story). In each it bounds the gap between D and the
corresponding projector.
The bounds and what follows from them are LIT-490’s. It proves the denoiser-to-projection bounds in all three settings for an exact Stein posterior-mean denoiser under deterministic DDIM, then derives finite-time rates for its gradient-then-denoise guidance (DCG). Its trajectory-tracking experiments show the failure the account predicts: post-denoising guidance plans that track the reference closely and lose much of that advantage once rolled out through the true dynamics.
The consequence is a reclassification rather than a new algorithm. With
D read as an inexact projection, the guided reverse process is a
time-varying inexact projected-gradient method, and the standard machinery
applies: geometric contraction for strongly convex objectives, a best-iterate
rate for smooth nonconvex ones. What had been a heuristic nudge acquires the
guarantees of an optimization method — and, more usefully, acquires an
explanation of when it fails, namely when the correction step is placed
where it cannot correct.
Why Proposed
Because the proofs assume the denoiser is exact. They are about
E[x₀ | x], and what anyone runs is a network trained to approximate it. The
paper is straightforward about this: projection-error terms can absorb
learned-score error, and a full treatment of misspecification is left to
future work. That gap is not a technicality — the whole claim is that this
particular operator has a geometric property, and the operator in the theorem
is not the operator in the code.
Because the geometry is assumed to be represented faithfully. The feasible set is whatever the learned support says it is. If the model has mass off the true feasible set, the “projection” projects onto the wrong thing, and nothing in the analysis notices.
Because the setting is narrow. Deterministic DDIM, variance-exploding diffusion, an exponentially decaying noise schedule. The paper says other samplers require tracking their own relaxation and denoising errors, which is an honest way of saying the result does not transfer as written to most deployed configurations.
Because the supporting evidence is indirect. The experiments show the practice works and that the alternative fails in the way this account predicts. They do not measure the projection property itself.
What it does not say
It does not say the denoiser is a projection. It bounds the distance between them under stated conditions. The word “approximate” is carrying weight, and how much depends on the noise level, the geometry’s curvature and the denoiser’s error — all of which the bounds expose and none of which is estimated for a real model here.
It does not license reading every denoiser as a constraint mechanism. The argument needs the clean data to actually concentrate on a structured set. For a diffusion model over natural images the “feasible set” is not a manifold anyone has characterized, and the theory says nothing about that case even though the intuition is often extended to it.
It says nothing about whether the learned geometry is the one you want. Projection onto the training distribution’s support is a feasibility guarantee only to the extent that the training data were feasible — which is an assumption about the dataset, not a property of the method.