LIT-555: Warm Diffusion: Recipe for Blur-Noise Mixture Diffusion Models

Hsueh, Yen, Peng, Huang, National Yang Ming Chiao Tung University (2025) — ARXIV-2511.16904

Key takeaways

• One process spanning hot and cold: q(x | x₀) = N(V M_α Vᵀ x₀, β² I), a DCT-domain Gaussian blur of level α plus noise of level β. BNR = α/β moves from hot (0) to cold (∞)
• Divide and conquer: a denoiser predicts the blurry clean image and a deblurrer predicts the missing high-frequency residual. They share most of an EDM network with doubled output channels, so BNR does not change capacity
• The trade-off it names: raising BNR hands work from the denoiser to the deblurrer, which can exploit spectral dependency. But it shrinks the noise that covers the space between samples, so reverse steps from an ill-posed deblurring more often leave the data manifold
• Choosing BNR from spectra: natural images fall off as 1/f² and white noise is flat. BNR = 0.5 is where blur starts attenuating a band about when noise already dominates it
• Table 3 (CIFAR-10, NFE 35): BNR 0 / 0.1 / 0.3 / 0.5 / 0.65 / 1 / 2 / 10 give FID 1.97 / 1.97 / 1.90 / 1.85 / 1.91 / 2.01 / 2.57 / 11.97
• Blurring Diffusion’s schedule, re-implemented in this framework, needs many more steps to be competitive (Table 4), which the paper reads as a schedule sitting at high BNR

Standing in the anthology

Filed from #163 (“[theory] warm diffusion”). Its account of why noise matters is THEORY-080, which corrects Cold Diffusion’s (THEORY-079).

Small gain, clear shape. The improvement over EDM at the best BNR is 0.12 FID, reported as the best of three sampling rounds with no variance. What the sweep shows more robustly is the other side: going cold costs a lot (11.97 at BNR 10). That is the part the correction rests on.

What it takes from Cold Diffusion, and what it adds. Cold Diffusion (LIT-553) showed the restore-and-re-degrade procedure running on blur with no Gaussian noise at all, and its noiseless blur generation was far worse than noise diffusion. Warm Diffusion keeps blur as a degradation but puts it on one axis with noise, so Cold Diffusion becomes the BNR → ∞ end of a family whose other end is ordinary hot diffusion, and adds the two-head denoise/deblur split and an account of why the cold end leaves the data manifold.

The baseline is EDM (LIT-075). The network, training recipe and Heun sampler are EDM’s, and BNR = 0 is EDM, so Table 3 is a controlled comparison against it: FID 1.97 for EDM against 1.85 at BNR = 0.5 on unconditional CIFAR-10 at 35 steps. Table 1 reports the same direction on class-conditional CIFAR-10 (1.79 against 1.68) and FFHQ at 79 steps (2.53 against 2.29), all best of three sampling rounds.

A misattributed baseline. Table 1 lists Cold Diffusion (Blur) at FID 80.08 for unconditional CIFAR-10 generation. That number is Cold Diffusion’s CIFAR-10 deblurring FID (its Table 1), a conditional restoration result. Cold Diffusion reports unconditional generation only on CelebA and AFHQ.

Read — NOTE-297.