Lu, Wang, Huang, Wu, Liu, Ouyang and Bai (2024) — ARXIV-2402.12376. Read: §1–4, Tables 3–4 and the hyperparameter appendix.
Key takeaways
-
The pipeline (§3.2). DiT “resizes and crops the images to a fixed resolution 256×256”. FiT does not crop and does not upsample small images. It resizes only images above HW ≤ 256² down to that bound, keeping aspect ratio, and pads each sequence to L_max = 256 tokens, the same as DiT’s fixed length. Padding tokens are masked in attention and loss.
-
The pipeline alone (Table 3, DiT-B → Config A). Same architecture, 400K steps, batch 256, no guidance:
eval size DiT-B fixed FID / sFID Config A flexible FID / sFID 256×256 44.83 / 8.49 43.34 / 11.11 160×320 91.32 / 66.66 50.51 / 10.36 224×448 109.1 / 110.71 52.55 / 16.05 -
The rest of FiT. Switching to 2D RoPE and SwiGLU brings 256² FID to 36.36 (Table 3). Training-free RoPE extrapolation methods (VisionNTK, VisionYaRN) extend it to unseen shapes (Table 4).
Where the hedges are
Per DP-010:
- At the square output everyone evaluates, the gain is small and mixed. −1.49 FID (3%) with a worse sFID, from one run each. SOTA-307’s floor is about 2%.
- The non-square gains measure coverage as much as quality. The fixed model trained only on 256² squares. The flexible one trained on the native shapes, which the table labels in-distribution at 160×320.
- Two changes are bundled. “No crop” and “no upsampling” change together. Small images are no longer enlarged, so the flexible model also sees fewer tokens on average per image.
Standing in the anthology
It extends DiT (LIT-448): the backbone is DiT’s transformer over latent patches, kept at DiT-B/2 for the controlled comparison, with the same 256-token sequence length. What it changes first is the data pipeline, from DiT’s resize-and-crop to 256×256 to padded, masked sequences at native aspect ratio (Table 3, DiT-B → Config A), and then the blocks, with 2D RoPE and SwiGLU, which take square FID from 43.34 to 36.36.
This is the closest generative analogue to NaViT (LIT-657) that the record holds, and it bears on SOTA-398. That practice measures no generative model. FiT shows that for a diffusion transformer, aspect-preserving training costs nothing at the square output and is required for non-square output. It does not show that preserving aspect ratio improves square generation, which is what isolating aspect ratio would need.