Source
Podell et al. (2023), LIT-566 — SDXL. Read as NOTE-306.
The practice
When the training set mixes resolutions and the model trains at a fixed size:
- Do not filter out images below the training resolution. Doing so threw away 39% of SDXL’s pretraining data at 256², and in the ImageNet ablation discarding cost more than anything else (FID-5k 43.84 against 39.76 for simply keeping them)
- Do not just upsample them either. The model learns the upsampling blur as part of the distribution
- Condition on
(h_original, w_original), Fourier-embedded (LIT-550) and added to the timestep embedding, and set it to the target size at inference (36.53). The same idea extends to random-crop offsets, set to zero at inference to avoid cut-off subjects, though that part is shown only qualitatively
Micro-conditioning on original size and crop was introduced in the SDXL report, LIT-566, to let the model train on all of a mixed-resolution dataset without learning upsampling blur or cut-off subjects. Its one quantitative test of size conditioning is Table 2, a class-conditional ImageNet model at 512² scored by FID-5k: 43.84 discarding small images, 39.76 keeping them unconditioned, 36.53 keeping them with size conditioning.
Conditions
- Most of the measured gain is from keeping the data (43.84 → 39.76). The conditioning adds 3.2 FID-5k on top
- One run per arm, FID-5k, class-conditional ImageNet, not the text-to-image model the practice is used in