THEORY-107: A diffusion loss whose weighting falls monotonically with signal-to-noise is maximum likelihood on noise-augmented data; a non-monotone one is not a likelihood objective

Source

Kingma and Gao (2023), LIT-692, §4 and Appendix C.

The account

Write the loss as L_w = ½∫ w(λ) E‖ε̂ − ε‖² dλ. The derivative in t of the reverse-process KL from t is ½(dλ/dt)E‖ε − ε̂‖². Integrate by parts and, with w normalized to 1 at the noisy end:

L_w = E_{t ∼ p_w}[L(t; x)] + const, with p_w(t) = dw(λ_t)/dt,

plus a point mass w(λ_max) at t = 0. L(t; x) is the expected negative ELBO of z_t, the data with Gaussian noise added at level t. So:

• If w is monotone (decreasing in λ), p_w is a distribution. The objective is maximum likelihood on a noise-augmented family, with the model conditioned on the augmentation: DistAug.
• If not, p_w changes sign. The objective is a signed mixture of ELBOs and is not a likelihood objective.

The derivation is LIT-692’s Theorem 1, proved in its Appendix C. The sorting of recipes into monotone and not comes from the same paper’s Table 1, which derives the implied w(λ) of each common loss. That is how v-MSE with a cosine schedule and flow matching on the OT path both reduce to e^{−λ/2}, while EDM and Min-SNR do not.

What it does and does not say

• It is the ELBO of augmented data, not of the data. Clean data gets only the small mass w(λ_max).
• It does not say monotone weightings sample better. The source’s own 64×64 table shows parity: EDM-monotonic 1.43 against EDM 1.43. SD3’s non-monotone logit-normal won its 61-way sweep at scale. What the theorem gives is an interpretation, not a ranking.
• It carries THEORY-027 into the weighted case. Given w, the loss depends on the schedule only through its endpoints, and the schedule’s shape sets only the Monte Carlo variance. THEORY-027 is the unweighted special case, and it remains scoped to scalar schedules.
• No practice is explained by it. Plain rectified flow is monotone and SD3’s logit-normal is not. Neither is recommended because of this, so explains: is empty on purpose.