anthology-of-the-sota

LIT-073: Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding

2 min read

Properties7
StatusActive
Version2
Tagsgenerative-modeling
Compared againstLIT-785 — STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
Cited as “introduced by” bySOTA-202 — Clamp the prediction to the training range at every step when sampling from a model's own output
Cited as “paper” byNOTE-040 — Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Cited as “source” bySOTA-202 — Clamp the prediction to the training range at every step when sampling from a model's own output

Graph View

Saharia et al. (2022) — ARXIV-2205.11487

Key takeaways

  • The finding is an allocation result, not an architecture one: “scaling the size of the frozen text encoder improves sample quality significantly more than scaling the size of image diffusion model”, and frozen language models trained only on text are surprisingly effective text encoders for image generation
  • Diagnoses why high guidance weights wreck images. At each step the x̂₀ prediction must lie in the training range [−1, 1]; high guidance pushes it out; and because the model is applied iteratively to its own output, the excursion compounds — “produces unnatural images and sometimes even diverges”
  • Dynamic thresholding is the fix: take s as a percentile of |x̂₀|, and if s > 1 clip to [−s, s] and divide by s, pushing saturated pixels inward at every step rather than letting saturation accumulate
  • The benchmark determines the conclusion. T5-XXL and CLIP text encoders perform similarly on MS-COCO; human raters prefer T5-XXL on DrawBench for both alignment and fidelity. DrawBench exists because COCO could not resolve it
  • Zero-shot COCO FID 7.27 without ever training on COCO, over GLIDE and DALL·E 2

Standing in the anthology

Read — NOTE-040. Nothing is sourced here.

The transferable finding is the thresholding rule, generalised: **when a model is applied to its own output, bound the output to the training range at every

step.** The record has SOTA-035 (clip gradients) and SOTA-158 (bound activations in low precision) as separate practices; this is a third instance of the same instrument in a third place. Iterative self-application appears in sampling, in autoregressive decoding and in agentic loops, and the record states the principle nowhere.

The allocation finding is the second gap: the record’s allocation practices are about the model being trained, never about what it is conditioned on.

STARFlow (LIT-785) quotes Imagen’s zero-shot COCO FID-30K of 7.3 as a text-to-image reference. Its 3.8B latent normalizing flow, trained on about 700M text-image pairs, scores 9.1, behind Imagen. The row is copied, not re-run.

Backlinks