Mou et al. (2023) — ARXIV-2302.08453
Key takeaways
- The framing is the contribution: the adapter “is not learning new generation abilities but learning an alignment between the condition information and internal knowledge in pre-trained T2I models” — which is why it can be small. Adding capability implies a large adapter; aligning to a latent one does not
- The spatial colour palette is a control signal designed for what it destroys: 64×64 bicubic downsampling wipes semantics and structure while keeping colour, then nearest-upsampling restores the size. The result cannot carry structure, so it can only control colour
- Adapters compose for multi-condition control, and transfer unchanged to any model fine-tuned from the same base — the property that made the ecosystem work
- The authors state the limitation: multi-adapter control “requires manual adjustment” of the guidance-feature combination; adaptive fusion is future work
Standing in the anthology
Read — NOTE-026. Retagged to adaptation-and-tuning.
The takeaway “zero-shot control” was wrong — the adapters are trained, per condition. What the paper actually reports is transfer to fine-tunes of the same base model, which is different and more specific.
Nothing is sourced here. The transferable idea is the colour palette’s design principle: build the input so it is incapable of expressing what you want excluded, rather than training the model to ignore it. The record’s data practices are about selection and mixing, never about constructing inputs with deliberately bounded content.
The disagreement with LIT-089 is worth keeping: ControlNet’s cloned encoder
implies the capability needs a large branch to reach, this implies it does not,
and neither paper runs the comparison.
Audited by #121; unsourced. R1 — ask whether you are adding a capability or aligning to one that already exists, because the answer sets the adapter’s size — is the sizing rule behind a practice the record holds without it: SOTA-051’s exact-zero argument and SOTA-184’s low-rank update both assume the capability is present. R3 (report whether composition requires manual weighting) is an honest-disclosure norm rather than a practice.