Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Generative modeling — diffusion, samplers, text-to-image, conditioning and control.
22 of 129 THEORY documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| THEORY-027 v2 | In continuous time the diffusion bound depends on the noise schedule only through its endpoints | Kingma et al. (2021), LIT-446 — integrate the diffusion variational bound over signal-to-noise ratio instead of over time and the schedule leaves the integrand, remaining only in the limits. The bound and the generative distribution therefore depend on the schedule through two numbers, and variance-preserving and variance-exploding specifications are the same model up to a rescaling of the latents. | Active |
| THEORY-046 | Uniform-state discrete diffusion is the argmax of a Gaussian diffusion, which is why Gaussian technique transfers to it and not to masked diffusion | Sahoo et al. (2025), LIT-479 — taking the argmax of a Gaussian diffusion’s latents carries its marginals onto those of a uniform-state discrete diffusion, under a reparameterization of the noise schedule, and the discretized process satisfies the defining ODE of a discrete diffusion. So it is one. Masked diffusion has no such preimage. | Active |
| THEORY-053 | The Stein posterior-mean denoiser acts as an approximate projection onto the learned data geometry | Zhang et al. (2026), LIT-490 — D(x) = E[x₀ | x] maps a noisy point toward the data support, and where the clean data lie on a linear subspace, inside a compact convex set, or on a compact smooth submanifold, the paper bounds its distance from the corresponding projection. The reverse process then reads as a time-varying inexact projected-gradient method rather than as sampling with a nudge. | Proposed |
| THEORY-054 | The regularizer that is intractable in data space is tractable in noise space, and bounds the data-space divergence | Eyring et al. (2025), LIT-491 — the KL to the base model needs Jacobian determinants through the generator and is intractable in data space. Posed over the input noise it reduces, under a Lipschitz condition, to an L2 penalty on the modification, and the data processing inequality makes that an upper bound on the data-space divergence. | Proposed |
| THEORY-079 | What makes a diffusion model generative is iterated restoration of any degradation, not Gaussian noise | Bansal et al. (2022), LIT-553 — the account cold diffusion was named for: the train-to-restore, sample-by-restore-and-redegrade procedure generates whatever the degradation is, so noise is incidental. Its own Table 5 has noiseless blur generation at FID 97.00 on CelebA against 23.11 with noise, and Warm Diffusion’s sweep degrades from 1.85 to 11.97 as noise is taken out. The procedure survives. The claim that noise is not what matters does not. | Rejected — its own paper’s unconditional numbers contradict it, and a blur-to-noise sweep shows generation degrading sharply as noise is removed |
| THEORY-080 v2 | A diffusion process needs noise to keep its intermediate states on the data manifold, and blur helps only while noise still dominates the bands it removes | Hsueh et al. (2025), LIT-555 — blur removes high frequencies deterministically, so many images share a blurry one and the deblurring target is their mean. Stepping toward it leaves the manifold unless noise has already made those images indistinguishable. Natural spectra fall as 1/f² against flat noise, so blur is safe in a band only once noise dominates it. That puts the useful ratio near 0.5 on CIFAR-10. The FID sweep fits the account, and the manifold departure itself is not measured. | Proposed |
| THEORY-086 v2 | A text document is better modelled as a mixture of several topics, each a distribution over words, than as a draw from a single topic | Blei, Ng and Jordan (2003), LIT-592 — exchangeability of a document’s words implies, by de Finetti, a mixture over a latent parameter. LDA takes that parameter to be a per-document distribution over topics. It has lower held-out perplexity than a mixture of unigrams, which forces one topic per document, on AP newswire and C. elegans abstracts, at every number of topics. The multi-topic structure and LDA’s Dirichlet smoothing are not separated, and perplexity is the only measure. | Proposed |
| THEORY-095 | No unbiased estimator of FID exists, so its plug-in bias can reverse a comparison while the sample variance stays small | Bińkowski et al. (2018), LIT-615 — the FID is a non-linear functional of the two distributions, and by the Bickel–Lehmann argument no estimator of it is unbiased for all distributions. The bias is therefore a property of the quantity, not of the plug-in formula, and it does not appear in the standard error: at d=2048 and 50,000 samples the estimator reversed a true ordering in all 100 trials, with standard deviations of 0.2 and 0.5 on a gap of 2.5. The non-existence proof needs a mixture family; what rescues it for practice is that Inception codes are not normal. | Active |
| THEORY-104 | A large guidance scale amplifies the model’s derivatives, which narrows a high-order solver’s convergence radius | Lu et al. (2022), LIT-676. Why the fast diffusion samplers invert under guidance. Classifier-free guidance scales the difference between conditional and unconditional predictions, which amplifies the model’s derivatives as well as its output; a k-th order solver is built from k-th order derivatives, so its convergence radius shrinks fastest for the largest k. At a fixed evaluation budget the higher-order method is then worse than the lower, which is what the measured FIDs show — 13.04, 114.62 and 164.74 at 10 evaluations for orders 1, 2 and 3. | Proposed |
| THEORY-106 v2 | Gaussian flow matching is a diffusion model; what separates the named recipes is loss weighting, network output and sampling schedule | Gao et al. (2024), LIT-678, on LIT-630. With a Gaussian source the flow-matching interpolant is a diffusion forward process with α_t = 1−t, σ_t = t. The conditional flow-matching loss is an ε-MSE under one particular weighting, and the flow-matching Euler sampler is DDIM. So a result labelled “flow matching against diffusion”, or “straight against curved path”, is a comparison of three things: the loss weighting, the network output and the sampling schedule. “Straight” describes the conditional path to one data point, not the marginal ODE a sampler integrates. | Active |
| THEORY-107 | A diffusion loss whose weighting falls monotonically with signal-to-noise is maximum likelihood on noise-augmented data; a non-monotone one is not a likelihood objective | Kingma and Gao (NeurIPS 2023), LIT-692. Every standard diffusion loss is ½∫ w(λ) E‖ε̂ − ε‖² dλ for an implied weighting. Integration by parts turns it into an expectation, over p_w = dw/dt, of the KL of the reverse process from time t. That is a distribution exactly when w decreases in log-SNR. The loss is then the expected ELBO of Gaussian-noise-augmented data, with the model told the noise level. v-MSE with a cosine schedule, flow matching on the OT path with uniform t, and InDI are monotone. ε-MSE with a cosine schedule, EDM, P2 and Min-SNR are not. | Active |
| THEORY-109 v3 | Classifier guidance may flatter classifier-based metrics because it steps along a classifier gradient, and guidance without a classifier is the control | Ho and Salimans (2022), LIT-693, raising the question about the method they replace. Classifier guidance mixes the score with an image classifier’s input gradient, which “can be interpreted as attempting to confuse an image classifier with a gradient-based adversarial attack” — and FID and Inception Score are both computed with an Inception classifier. So the metric gains may be partly adversarial rather than perceptual. Classifier-free guidance obtains the same trade with no classifier anywhere, which is the control, and it wins — but on the same classifier-based metrics. The antecedent (LIT-699) had already asserted the negative, in one clause of its introduction, with no test anywhere in the paper. SOTA-337 names the instrument that would settle it and nobody has run it on guided sampling. | Proposed |
| THEORY-110 | FID is not neutral between fidelity and coverage: it weights coverage more, so optimizing it walks toward variety at the cost of per-sample quality | Kynkäänniemi et al. (2019), LIT-703, measured on StyleGAN: “FID favors configurations with high recall over the ones with high precision”, and the best-recall configuration set a new state-of-the-art FID. The mechanism they give is that FID is a Wasserstein-2 distance in feature space, so low intrinsic variation yields low FID even when much of that variation is missed. LIT-699 is an independent instance in another family: ADM-G beats BigGAN-deep on FID at three resolutions while losing precision at all three and winning recall. So FID does not merely mix fidelity and coverage — it mixes them with a direction, and a model tuned on FID is tuned toward coverage. | Proposed |
| THEORY-112 | Classifier-free guidance improves image quality because its unconditional reference model is worse, not because it emphasizes the class, so guidance is adaptive truncation toward the better-fit density | Karras et al. (2024), LIT-721. Guidance adds a force toward higher log[p₁/p₀]. The reference p₀ is more spread out than p₁ — a harder task on a smaller training slice — so it falls off slower away from the data and the ratio’s gradient points inward, concentrating samples on the manifold. That, not class emphasis, is where CFG’s image-quality gain comes from; the class emphasis is the separate part that costs diversity. Tested with a negative control: degrade the two models incompatibly and guidance stops helping at any weight. | Proposed |
| THEORY-118 | Continuous-time consistency training is unstable through the time derivative in its tangent, and tangent normalization, which MeanFlow’s adaptive loss weight already is, controls it | Lu and Song (2024), LIT-790, §4.1–4.2. The continuous-time consistency gradient follows the tangent df/dt along the ODE. Of its parts, the authors find the time derivative of the network unstable, and derive why: EDM’s time transform contributes a factor 1/cos(t) that blows up at the noise end, and a large Fourier embedding scale oscillates. Normalizing the tangent caps the rest. Zheng et al. (LIT-783) find the same term fragile at 14B and in BF16, and state that MeanFlow’s adaptive weight with p = 1 is tangent normalization, which is checkable algebra. The localization is measured on CIFAR-10 gradient norms, and the fixes on one ImageNet-512 setting. | Proposed |
| THEORY-119 | Consistency, shortcut, MeanFlow and progressive-distillation models all estimate the two-time flow map of the probability-flow ODE, and differ in which identity of that map they train on | Boffi, Albergo and Vanden-Eijnden (2024), LIT-793. The object a few-step model learns is X_{s,t}, which carries any point of the ODE from time s to time t. It satisfies three identities: a Lagrangian equation in the end time, an Eulerian equation in the start time, and composition. Progressive distillation and shortcut models train on composition, consistency models and MeanFlow on the Eulerian equation, and flow map matching’s Lagrangian loss on the first. Consistency models are the one-time case. Inductive Moment Matching is the exception: it matches marginals, and its minimizer need not be the ODE’s map. Which identity trains best is measured once, on CIFAR-10, where the Lagrangian form won by a wide margin. | Active |
| THEORY-120 | A likelihood-trained normalizing flow samples well only when its training data carry Gaussian noise well above the quantization width, because its inverse must be well conditioned over the whole Gaussian latent | Zhai et al. (2024), LIT-794, §2.4 and §3.3. Trained on the uniform dequantization noise that likelihood evaluation uses, TarFlow could not sample: “constant numerical issues” and nothing sensible. Gaussian noise of standard deviation about 0.05, against the dequantization noise’s 0.002, gave its best samples. The authors offer two reasons and a hypothesis, and test none of them apart. STARFlow (LIT-785) keeps the noise, at 0.3 in latent space, on TarFlow’s word. One ablation from one group, and an account that is the authors’ own guess. | Proposed |
| THEORY-121 | Distribution-matching distillation loses coverage because it descends a reverse divergence on the student’s own samples, and a consistency term on teacher trajectories restores it | Lu and Song (2024), LIT-790, Fig. 7, and Zheng et al. (2025), LIT-783. DMD (LIT-643) descends KL(p_student ‖ p_teacher), evaluated on the student’s own samples. That direction penalizes mass where the teacher has none and is indifferent to a mode the student no longer produces. A consistency loss supervises points along the teacher’s trajectories that the student did not choose. On one backbone, sCM found distribution matching raising precision and losing recall as guidance rises, while consistency stayed near the teacher; read from the curves, the recall gap is already there at guidance 1. The account is rCM’s framing. That figure is its only measurement, from one group and one backbone, and DMD’s own authors give a different mechanism. | Proposed |
| THEORY-122 | A diffusion or flow sampler can be trained by a likelihood-ratio policy gradient only when each step is stochastic with a tractable density, and a same-marginal SDE built from the trained velocity supplies that without changing the marginals | Black et al. (2023), LIT-776, and Liu et al. (2025), LIT-779. A policy gradient needs the log-density of each action. Treated as one action, a diffusion model’s sample has only a variational bound. Treated as T actions, each DDPM step is an isotropic Gaussian with an exact density. A flow’s ODE step is deterministic, so it has no per-step density and explores nothing past the initial noise. The stochastic- interpolant identity (LIT-645) gives, for any noise level, an SDE with the ODE’s marginals, built from the trained velocity alone. Each of its steps is Gaussian, so the ratio and the KL to the reference are closed form. All of this is derivation. What is not shown is that it survives the ten-step discretization used for rollouts. | Active |
| THEORY-123 | A flow’s sampling ODE curves because the noise-data pairs it was trained on cross and the learned velocity averages them, so a coupling whose straight paths do not cross gives a straight flow | Pooladian et al. (2023), LIT-789, and Tong et al. (2023), LIT-798, with Rectified Flow, LIT-636. Flow matching regresses onto the velocity of a straight line between a paired noise and data point, so the learned field at (x, t) is the average of the velocities of every pair whose line passes through x at t. Under independent pairing many lines cross there, and the average turns. A coupling with no crossings, the OT plan in the limit or the flow’s own map after reflow, leaves one line through each point and a straight flow. The mechanism is proved. That this is what limits few-step image quality is measured only at 64 px and below, and Lee et al. (LIT-791) argue, without proof, that one reflow is enough. | Active |
| THEORY-124 | Consistency training is a bootstrapped fixed-point iteration, not descent on a loss, so a target that lags the student swamps a data signal that enters only at the order of the step | Song and Dhariwal (2023), LIT-786, Prop. 1, and Boffi, Albergo and Vanden-Eijnden (2024), LIT-793, Prop. 3.12. Consistency training compares the model at one noise level with a target at the next, and the data enter only through that small step. With the target’s weights equal to the student’s under stop-gradient, the scaled update converges to a regression onto the data. With any other target, such as an EMA, the limiting objective does not depend on the data at all. Proved on a one-point distribution, where it refutes the argument that justified EMA targets. That it is why EMA targets hurt on real data rests on one CIFAR-10 curve. | Proposed |
| THEORY-125 | One affine autoregressive flow block makes every conditional a single Gaussian and cannot approximate an arbitrary density, while two blocks in opposite orders make every conditional but the last a Gaussian mixture and three make all of them one | Gu et al. (2025), LIT-785, §3.1, Prop. 1 and App. A.1. In one affine autoregressive block each coordinate is a mean plus a scale times fresh Gaussian noise, both functions of the coordinates before it, so every conditional is one Gaussian and no multimodal conditional can be represented. A second block in the opposite order feeds later latents into those parameters and turns each conditional into an infinite Gaussian mixture, except the last coordinate’s; a third restores that one. The one-block half is proved and is what TarFlow (LIT-794) measured, FID 267 at matched depth. The universality half is a sketch. | Proposed |