Karras, Aittala, Laine, Härkönen, Hellsten, Lehtinen, Aila, NVIDIA and Aalto (2021) — ARXIV-2106.12423
Key takeaways
- The diagnosis: coarse features in a GAN control whether fine features exist but not where they go. Fine detail stays fixed in pixel coordinates. The cause is aliasing: boundary padding, pointwise nonlinearities and sloppy up- and downsampling all leak absolute-position information and high frequencies into the hierarchy
- The fix, from signal processing: treat features as continuous bandlimited signals. A pointwise nonlinearity commutes with translation and rotation in the continuous domain but creates frequencies beyond the band, so apply it at 2× temporary resolution and low-pass filter back down. Nonlinearities are then the only source of new frequencies, and their cutoff per layer controls how much detail each layer adds
- Figure 3 (FFHQ-U 256², FID / EQ-T / EQ-R in dB): StyleGAN2 5.14 / – / – → Fourier-feature input 4.79 / 16.2 → … → filtered nonlinearities 6.35 / 30.6 → non-critical sampling 4.78 / 43.9 → StyleGAN3-T 4.62 / 63.0 / 13.1 → StyleGAN3-R 4.50 / 66.7 / 40.5
- Cost: parameter ablations trade equivariance against time and memory. Upsampling by 4 instead of 2 raises EQ-T to 74 at 2.3× time
Standing in the anthology
Filed from #163 (StyleGAN/2/3, “equivariant representation”). It
sources SOTA-335 and extends StyleGAN2 (LIT-560). Its input is
a Fourier-feature mapping in the sense of LIT-550.
Equivariance here is an output property, measured. EQ-T and EQ-R are PSNRs between transforming the input and transforming the output. That is a different claim from THEORY-052’s about equivariant representations in generative priors, though the two agree on direction.
Read — NOTE-300.