Source
Ho and Salimans (2022), LIT-693 — ARXIV-2207.12598.
What to do
At training time. With probability p_uncond, replace the conditioning
signal with a null token before computing the loss. One network, one training
run, both scores. Use p_uncond = 0.1: the paper’s sweep puts the best
achievable FID at 1.55 for 0.1, against 1.62 for 0.2 and 1.91 for 0.5, so more
unconditional training costs and does not buy.
At sampling time. Combine the conditional and unconditional score estimates,
extrapolating away from the unconditional one by a weight w. There is no
classifier and no second model.
The recipe originates with LIT-693, which introduced it to replace Dhariwal
and Nichol’s classifier guidance: the same FID–IS trade without a second
classifier trained on noisy data. On ImageNet 128×128 it beats the
classifier-guided ADM-G on FID at w = 0.3, and at w = 4.0 beats
BigGAN-deep on both FID and IS.
Choosing w, which is the part that is usually got wrong
The weight is not a quality knob with a best setting. It is a position on a
trade, and the trade is steep. ImageNet 64×64 at p_uncond = 0.1, 50 000
samples per point:
w | FID ↓ | IS ↑ |
|---|---|---|
| 0.0 | 1.80 | 53.71 |
| 0.1 | 1.55 | 66.11 |
| 0.3 | 3.03 | 92.80 |
| 1.0 | 12.60 | 170.1 |
| 2.0 | 21.03 | 225.5 |
| 4.0 | 26.22 | 260.2 |
FID degrades about 17-fold across the range while IS nearly quintuples. So:
- want the best FID →
waround 0.1 to 0.3; - want the best IS, or visibly sharper individual images →
w ≥ 4; - there is no setting that is best at both, and the paper does not claim one.
On text-to-image with human judges, the trade runs against FID: Pick-a-Pic (LIT-782) found users usually prefer guidance 9 to 3, and across nine model–guidance settings on MS-COCO captions FID ranked models at Spearman −0.900 against expert preference. If the metric you keep is human preference, FID is the one you lose.
Never compare two models at different w. A guidance sweep can produce
almost any FID or IS you like from one checkpoint, so a comparison that does not
fix the weight is not a comparison. SOTA-307 says to search the scale per cell
and to report the search tolerance; this is the measurement that makes that
non-negotiable rather than fastidious.
Conditions and costs
The visible cost at high w is saturation. The paper reports that strongly
guided samples “display saturated colors” at w = 3.0. That is the excursion
SOTA-202 exists for — predictions leaving the training range — so at high
guidance the two practices are used together, and SOTA-410 is the sampler
constraint that makes the clamp available.
Conditional generation only — for this recipe, not for guidance. There has to be a condition to drop, so nothing in the training change above applies to an unconditional model.
SOTA-397 is the record’s holding on conditioning a
model trained without one, and SOTA-433 is guidance that needs no condition at all — it reaches unconditional EDM2-S from FID 11.67 to 3.86, a regime this practice excludes.
A capacity question the paper does not answer. One network now represents two
distributions. The sweep over p_uncond bounds how much unconditional training
is useful and says nothing about whether a larger model would prefer a different
split, or whether the conditional score is worse than it would have been from a
dedicated model.
EDM2 (LIT-714) answers the half of this that concerns the unconditional score, from the other design: it trains the unconditional model separately, and an XS model (125M parameters) guides its XXL conditional model (1.5B) as well as any larger unconditional model did — “using a larger unconditional model did not improve the results at all” — at almost half the sampling cost of guidance. One table, one conditional model, ImageNet-512. It says the unconditional score needs far less capacity than the conditional one; it does not say whether sharing one network, as recommended here, costs the conditional score anything.
The weight is not the only thing guidance moves. EDM2 also finds the best EMA length depends “very strongly” on the guidance weight, and that FID and FD_DINOv2 disagree on the weight itself (1.4 against 1.9 in EDM2’s convention, where 1 means no guidance — 0.4 against 0.9 on this practice’s scale). A weight swept at one EMA length and reported at another is not the sweep it claims to be; the EMA side is SOTA-428.
Diversity is what is being spent, and it is the class emphasis that spends it. The paper is explicit that raising the weight decreases sample variety and increases individual fidelity, and for this recipe that trade is real: plan for it if your application needs coverage, because the default high weights consumer tools ship are the wrong end of it.
What LIT-721 adds is that the trade is not a property of guidance. It argues the quality gain comes from the unconditional reference model being a worse fit — a harder task on a smaller training budget — and the diversity loss from the class emphasis, and it separates them by swapping the reference for a degraded copy
of the conditional model. THEORY-112 is the account. So the sentence to carry forward is not “guidance costs diversity” but “this reference model costs diversity”.
Known implementations
- Stable Diffusion, where the weight is user-facing and typically defaulted around 7.5.
- Imagen, which pairs it with the thresholding of SOTA-202.
- diffusers, as
guidance_scale.