Source
Sahoo, Gokaslan, De Sa and Kuleshov (2023; NeurIPS 2024), LIT-677.
What to do
If the model is trained for likelihood (density estimation, compression),
use a multivariate noise schedule: a monotone polynomial in t for each
dimension, with endpoints fixed at the usual values. Condition it on an
auxiliary latent z inferred by an encoder from the clean input. The same
z conditions the denoiser, and at generation time z is drawn from a
learned prior. The bound gains a KL term for z.
Do not expect either half alone to help:
- A scalar schedule conditioned on the input “doesn’t offer any advantage over the scalar schedule used in VDM”.
- A multivariate schedule conditioned only on time “becomes comparable to that of VDM”.
- Conditioning directly on the image or on class labels does worse than VDM.
Why
For a scalar schedule the bound depends on the schedule only through its endpoints (THEORY-027). The one free degree of freedom is then shape, and shape cannot move the bound. A per-dimension schedule turns the bound into a line integral along a path in SNR-space, with many paths between the same endpoints, so the schedule has something to optimize. That this path-dependence holds in general is argued in the source and not proved. What is measured is that the latent-conditioned version moves the bound.
Evidence
At equal steps and architecture: 2.65 → 2.60 bits/dim on CIFAR-10 (10M steps), and 3.72 → 3.71 on ImageNet-32 (2M). The same 2.65 comes in 2M steps rather than 10M on CIFAR-10, and 1M rather than 2M on ImageNet-32. Moving a trained MuLAN denoiser back onto a scalar schedule returns exactly VDM’s 2.65, which ties the gain to the schedule rather than the denoiser.
The method and these numbers are Sahoo et al.’s, LIT-677, which introduced MuLAN on VDM’s own U-Net, settings and datasets (Table 1). The warning that either half alone does nothing comes from the same paper’s ablation, which is one run each on CIFAR-10 at 2.5M steps and shown only as a figure.
Conditions
Likelihood only, 32×32 only, VDM’s U-Net only. The source is explicit that sample quality is not its aim. Its FID is mixed, and worse on ImageNet-32 at 1M steps.
The equal-step gain is small. Most of the value is in steps to a given bound, which matters to anyone whose budget is the constraint.
It competes with SOTA-264 for the same objective. SOTA-264 takes the scalar setting and spends the shape on the loss estimator’s variance. This practice leaves the scalar setting and spends a multivariate shape on the bound. MuLAN measured against VDM’s learned scalar schedule, which is SOTA-264’s source setting. Choose by whether the added encoder, KL term and roughly 10% of extra parameters are worth it for your target.
Known implementations
- MuLAN, the source’s code.