anthology-of-the-sota

LIT-562: Projected GANs Converge Faster

2 min read

Properties5
StatusActive
Tagsgenerative-modeling, vision-and-graphics, training-optimization
Cited as “introduced by” bySOTA-338 — Train the GAN discriminator on frozen multi-scale pretrained features with fixed random channel and scale mixing
Cited as “paper” byNOTE-303 — Projected GAN
Cited as “source” bySOTA-338 — Train the GAN discriminator on frozen multi-scale pretrained features with fixed random channel and scale mixing

Graph View

Sauer, Chitta, Müller, Geiger, Tübingen, MPI-IS and Heidelberg (2021; NeurIPS 2021) — ARXIV-2111.01007

Key takeaways

  • Discriminate in a fixed feature space. Four discriminators, one per scale (64² to 8²) of a frozen pretrained network, each a small spectrally normalized conv net. Their logits are summed
  • Random mixing makes the deep features usable. Without it the discriminator ignores parts of the deeper, semantic features. Fixed Kaiming-initialized 1×1 convolutions (cross-channel mixing, CCM) and 3×3 convolutions with upsampling (cross-scale mixing, CSM) help, and all four discriminators then work best together
  • Compact feature networks win. EfficientNet-Lite1 gives FID 1.65 on LSUN-Church. Larger and more ImageNet-accurate networks are worse (ViT at 12.38). ImageNet accuracy does not predict FID, and CLIP-R50 slightly beats R50, which the paper takes as evidence ImageNet features are not required
  • Speed and scores (Table 3, 256²): LSUN-Church FID 1.59 against StyleGAN2-ADA’s 5.85 at 10M images. Bedroom 1.52 against GANformer’s 6.15. The previous best FID on Church, StyleGAN2’s 3.39 at 88M images, is reached after 1.1M, under 3 hours against 5 days
  • Data efficiency: large gains on small datasets (AnimalFace-Dog 17.88 against 60.90)

Standing in the anthology

Filed from #163 (Projected GAN). Sources SOTA-338, which is Proposed and contested.

The measurement problem is in the design. The discriminator is trained on ImageNet features, and FID is computed in ImageNet-trained Inception features. The authors avoid Inception as the feature network and report KID, SwAV-FID and precision/recall in the appendix, which they say agree (not read here). Kynkäänniemi et al. (LIT-563) then trained Projected FastGAN with this paper’s code on FFHQ. At FID 5.28 against StyleGAN2’s 5.30, the CLIP-space distance was 4.67 against 2.76, and human raters preferred StyleGAN2. They cite this paper’s own human study as reaching the same conclusion on FFHQ. So “converges faster” is established in FID. How much of that is image quality is contested.

Read — NOTE-303.

Backlinks