anthology-of-the-sota

LIT-560: Analyzing and Improving the Image Quality of StyleGAN

2 min read

Properties8
StatusActive
Tagsgenerative-modeling, model-architecture, vision-and-graphics
ExtendsLIT-561 — A Style-Based Generator Architecture for Generative Adversarial Networks
Extended byLIT-559 — Alias-Free Generative Adversarial Networks
Compared againstLIT-699 — Diffusion Models Beat GANs on Image Synthesis
Cited as “introduced by” bySOTA-334 — In a style-modulated generator, demodulate the convolution weights instead of instance-normalizing the activations, SOTA-336 — Train high-resolution GANs at a fixed topology with output skips in the generator and a residual discriminator, not by progressive growing
Cited as “paper” byNOTE-301 — StyleGAN2
Cited as “source” bySOTA-334 — In a style-modulated generator, demodulate the convolution weights instead of instance-normalizing the activations, SOTA-336 — Train high-resolution GANs at a fixed topology with output skips in the generator and a residual discriminator, not by progressive growing

Graph View

Karras, Laine, Aittala, Hellsten, Lehtinen, Aila, NVIDIA and Aalto (2019) — ARXIV-1912.04958

Key takeaways

  • Droplets come from AdaIN. Instance normalization discards feature magnitudes, so the generator smuggles scale past it with a localized spike. Removing normalization removes the droplets. Weight demodulation, scaling each output channel’s weights by 1/√Σ w'² after modulation, restores the expected unit variance without normalizing the data
  • Phase artifacts come from progressive growing. Each resolution is briefly the output, so it learns maximum-frequency detail, which leaves details stuck to pixel positions. A fixed topology with output skips in the generator and a residual discriminator keeps the coarse-to-fine emphasis without growing
  • Table 1 (FFHQ 1024², FID / PPL): baseline 4.40 / 212 → demodulation 4.39 / 175 → lazy R1 4.38 / 158 → path-length reg 4.34 / 123 → no growing, new G and D 3.31 / 125 → larger networks 2.84 / 145. On LSUN Car, path-length regularization raises FID from 2.83 to 3.43
  • Table 2 (architecture grid, no growing): on FFHQ, output-skip G with residual D is 3.31, against 4.32 for the original feedforward pair
  • Lazy regularization: R1 computed every 16 minibatches, with no harm
  • Metrics are averaged over 10 evaluation seeds. Each configuration is one training run, selected at its best-FID snapshot

Standing in the anthology

Filed from #163 (StyleGAN/2/3, “progressive training”). It sources SOTA-336 (no progressive growing) and SOTA-334 (weight demodulation). It extends StyleGAN (LIT-561), and StyleGAN3 (LIT-559) extends it.

FID’s blind spot is part of the paper’s argument. It shows images with equal FID and precision/recall but visibly different quality, and uses PPL as the metric that tracks them. That is the reason path-length regularization exists, and also a caution against reading its FID trade-off on LSUN Car as the whole story.

The GAN that diffusion was measured against. Dhariwal and Nichol (LIT-699) use StyleGAN2 as the baseline on LSUN Horse and Cat at 256². Their diffusion model scores FID 2.57 against StyleGAN2’s 3.84 on Horse and 5.57 against 7.25 on Cat, with higher precision and recall on both, and their Table 10 puts the diffusion runs at similar or lower training compute (116 against 130 V100-days on Horse, 92 against 115 on Cat). Their architecture ablation also tests the 1/√2 residual scaling taken from both StyleGAN papers, and finds it makes FID slightly worse.

Read — NOTE-301.

Backlinks