Hoogeboom, Heek and Salimans, Google Research, Brain Team (January 2023; ICML 2023) — ARXIV-2301.11093. Read at v2: the main text, Tables 2–9 and Appendix C.
Key takeaways
-
The argument (§3.1). Average-pooling a d×d noised image by s×s halves the per-pixel noise at each halving step: the SNR at d/s is s² times the SNR at d. So a schedule designed at 32 or 64, like the cosine schedule, adds too little noise at high resolution. Global structure is fixed early, “a small time window to decide on the global structure”.
-
The fix. Choose a reference resolution. Define the schedule there, and multiply the SNR by (ref/d)², which is a log-SNR shift of 2·log(ref/d) (eq. 5).
-
The measurement (Table 2, §5.1). Same setup, and only the reference resolution varies:
resolution schedule FID train FID eval 128 cosine, unshifted 2.96 3.38 128 shifted to 64 2.41 3.03 128 shifted to 32 2.26 2.88 256 cosine, unshifted 7.65 6.87 256 shifted to 128 5.05 4.74 256 shifted to 64 3.94 3.89 256 shifted to 32 3.76 3.71 The gain grows with resolution, which is the direction the argument predicts. The authors recommend shifting to 64 because it “performed slightly better in early iterations”.
-
Interpolated schedules for guidance. A shifted schedule “can only tolerate little guidance” (Table 9). For text-to-image they interpolate in log-space between shift 32 and shift 256 (eq. 6).
-
The other three findings. Scale the U-Net only at 16×16. Add dropout, but not at the highest-resolution maps. Downsample at the input. A multiscale loss helps at 512 (FID 4.85 → 4.30 train) and slightly hurts at 256 (Table 6).
Where the hedges are
Per DP-010:
- The reference resolution is tuned, not derived. The argument gives a shift for any reference. Which reference is best is found empirically (32 or 64), and the two differ by 0.18 FID at 256, one run each.
- Single runs throughout. The 7.65 → 3.76 gap is far outside SOTA-307’s roughly 2% run-to-run floor. The 3.94 against 3.76 comparison is not.
- Pixel space, trained from scratch at each resolution. The schedule is set before training, not carried from a low-resolution stage. The mechanism is the same one SOTA-263 describes for finetuning, but the measurement is not on the pretrain-then-finetune path.
Standing in the anthology
This is where the resolution-dependent schedule shift comes from, together with Chen (2023), LIT-659, which it names as “concurrent and complementary”. SD3 (LIT-449) derives the shift again from a constant-image argument and states that it implies “a log-SNR shift … similar to (Hoogeboom et al., 2023)”. SOTA-263 named SD3 as its origin and said no ablation existed. Table 2 here is that ablation, published 14 months earlier.
EDM2 (LIT-714) compares against this paper’s ImageNet-512 result by quoting its published numbers, without a rerun (its Table 2). The U-ViT L here is listed at FID 3.54 unguided and 3.02 guided, at 2455M parameters and 256 stochastic sampling steps. EDM2-S, a 280M latent model sampled deterministically in 63 evaluations, scores 2.56 and 2.23. The comparison differs in pixel against latent space and in sampler as well as in architecture, so it ranks the systems and does not test the schedule shift.