Gal et al. (2022) — ARXIV-2208.01618
Key takeaways
- Personalizes a completely frozen text-to-image model by optimizing exactly
one thing — a new token embedding
S*inserted where tokens become vectors. No weights change; the model’s vocabulary is expanded with a pseudo-word - Frames the result on the distortion/editability tradeoff borrowed from GAN inversion, claiming a point on the curve rather than an escape from it
- Reaches abstract concepts, not only objects: a shared style is found by
training on prompts of the form “A painting in the style of
S*” - A decomposition worth stealing: optimize a shared placeholder
S*and a per-image placeholderS_ijointly, with prompts “A photo ofS*withS_i”, so the model puts the shared concept in the shared code and relegates per-image incidentals like background toS_i
Standing in the anthology
Read — NOTE-034. Retagged to adaptation-and-tuning: nothing about the
generative model changes.
Nothing is sourced here; the record’s adaptation practices are LoRA-shaped and concern language models.
The finding with reach past the domain is the per-image decomposition: give nuisance variation its own parameters rather than hoping it averages out. That is a control variable, applied to learned embeddings, and the record states it nowhere.
Audited by #121; unsourced, and its claim is held from the other side. R1 — try the smallest intervention point first, because a frozen model’s input embedding space may already reach the concept — is the same argument SOTA-199 files from LIT-079, and LIT-091 makes it again for adapter sizing. Three instances of one idea, filed once. Recorded here so the next audit does not read the silence as an oversight.