Source
Ho and Salimans (2022), LIT-693, who raise it about their predecessor rather than about themselves.
The account
Classifier guidance improves a diffusion model’s FID and Inception Score by mixing the model’s score estimate with the input gradient of a classifier’s log probability. Both of those metrics are computed by passing samples through an Inception classifier.
Taking gradient steps that maximise a classifier’s confidence in a target label is, mechanically, what an untargeted-to-targeted adversarial attack does. The paper says so:
because classifier guidance mixes a score estimate with a classifier gradient during sampling, classifier-guided diffusion sampling can be interpreted as attempting to confuse an image classifier with a gradient-based adversarial attack. This raises the question of whether classifier guidance is successful at boosting classifier-based metrics such as FID and Inception score simply because it is adversarial against such classifiers.
If that were the whole story, the reported gains would be an artefact of the scoring apparatus rather than a fact about the images — the shape SOTA-200 exists to make readers check.
The control the paper builds. Classifier-free guidance obtains the same fidelity/diversity trade with no classifier in the pipeline at all: no classifier to attack, no classifier gradient to follow. That the trade survives its removal is evidence the trade is real.
What was actually shown, and what the control cannot settle
The trade does survive. Guidance weight from 0.1 to 4.0 on ImageNet 64×64 moves FID from 1.55 to 26.22 and IS from 66.11 to 260.2, with no classifier anywhere in that sweep, and the qualitative effect the paper reports — less variety, more per-sample fidelity — is what the metrics say. So the phenomenon is not an artefact of having a classifier in the loop.
The suspicion about the predecessor’s magnitude is not settled, though, and
the reason is that the comparison is still scored the same way. At w = 0.3
classifier-free guidance beats classifier-guided ADM-G on FID at 128×128. That is
one classifier-based metric preferring the method with no classifier gradient,
which is consistent with the adversarial reading having been wrong and equally
consistent with it having been right and the classifier-free method simply being
better. Beating a suspect measurement at its own game does not tell you whether
the measurement was suspect.
Hence the promote_when: the question needs a metric with no classifier in it.
What the antecedent says about it, which is less than it looks
LIT-699 was read after this account was filed, and it changes the shape of the dispute rather than the verdict. Dhariwal and Nichol got there first and did not measure it either. Their introduction says the gradient scale can be raised
by an order of magnitude without obtaining adversarial examples
citing Szegedy et al. (2013), and the word “adversarial” does not appear again anywhere in the paper — no test, no held-out classifier, no non-Inception distance. The nearest check is a memorization test on “a handful of samples”, run in InceptionV3 feature space, which is the same circularity one level down.
So the sequence is not “raised in 2022, already answered in 2021”. It is
asserted away in 2021 without evidence, raised in 2022, and measured by
neither. That is why this stays Proposed with both papers as sources: two
documents gesture at the question and the record has no measurement from either.
The record already owns the instrument. SOTA-337 says that when an ImageNet classifier is in the pipeline, FID’s ranking is not to be trusted, and names CLIP and SwAV Fréchet distances as the check; its source explicitly suspected classifiers “placed in the sampling loop”. That practice’s v1 title said “training”, which is why this connection was missed when this account was filed hours earlier the same day. The experiment is cheap, the instrument is named, and it has not been run.
Two further facts from the full text bear on the account from the antecedent’s side:
- The classifier being followed is not the classifier doing the scoring. It is the U-Net’s downsampling trunk with an attention pool, trained on noised ImageNet; FID and IS use Inception-V3. The mechanism would have to be transfer — a gradient that raises one ImageNet classifier’s confidence also moving another’s features — which is plausible, but is one step more than “attacking the scorer”.
- The paper observed the predicted shape once, at the low end. On an unconditional model at scale 1 “the classifier assigned reasonable probabilities (around 50%) to the desired classes”, and the samples “did not match the intended classes upon visual inspection”; FID got worse (26.21 → 33.03). A classifier satisfied by images that do not look like the class is what the account is about. Scaling up to 10 fixed it by eye and in FID, which is the case the account cannot yet distinguish from a real improvement.
What this does not say
Not that classifier guidance’s results were fake. The trade it reports is real in the sense that matters — classifier-free guidance reproduces it with no classifier anywhere. What is open is the magnitude attributable to the classifier gradient, and neither paper measured it.
Not a general claim about gradient-based guidance. SOTA-301 recommends applying an objective gradient before the denoiser when steering diffusion toward a task objective; nothing here bears on that, because the objective there is not the thing the evaluation is computed with. The concern is specifically that the network being differentiated and the network doing the scoring are both ImageNet classifiers — not the same network, as LIT-699 makes clear, but close enough for a gradient on one to plausibly move the other.
Not an argument for either method. SOTA-424 recommends classifier-free guidance on grounds that have nothing to do with this — one model instead of two, no noisy-data classifier to train — and would stand if this account were abandoned entirely.