Source
Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf and Daniel Cohen-Or (2023), “Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models”, SIGGRAPH 2023, LIT-806, read in NOTE-639. The independent binding measurement is from T2I-CompBench++ (LIT-798, NOTE-610).
What was actually shown
The failure. “Catastrophic neglect”: one of the subjects a prompt names is absent (Figs. 2, 5). The paper shows it on Stable Diffusion v1.4 and reports no rate.
The account. The 16×16 cross-attention maps, averaged over layers and heads, are per-patch distributions over tokens, so one token can win everywhere and another nowhere. Layout is fixed in the early steps. A subject appears only if its token governs some patch early on.
The intervention. At each step t ≤ 25 of 50, the loss is L = max over subjects of (1 − max of G(A_t^s)), with G a 3×3 Gaussian (σ = 0.5) and the start-of-text token dropped; the latent takes a gradient step against L, with refinement to set thresholds at steps 0, 10 and 20. No weights change.
Results (276 prompts; Fig. 8, Tables 1–2). CLIP image–text similarity is higher than every baseline on all three subsets, and minimum-object similarity at least 7% above Stable Diffusion and StructureDiffusion. Caption similarity is 0.806, 0.830 and 0.811. In the preference study 90.70%, 77.64% and 77.16% chose it.
Ablations (Appendix B). Without smoothing, the loss is met by a fragment and the subject does not appear. Without refinement, neglected subjects stay missing. Updating after step 25 adds artefacts without semantic gain.
Binding, measured elsewhere. Re-implemented on Stable Diffusion v2, the method raises BLIP-VQA colour binding from 0.5065 to 0.6400 and texture binding from 0.4922 to 0.5963, with little change in shape binding or relations (LIT-798, Table XIII).
What could have come out otherwise. Raising a token’s attention could have produced a fragment or nothing; without the smoothing it does exactly that.
What this does not say
- How often neglect happens, or how much is removed. Every quantitative score in the paper is of overall prompt similarity or preference, not of per-subject presence.
- That attention causes the subject. The account is supported by an intervention that works, not by a test that attention predicts which subject goes missing.
- That presence fixes binding. The paper argues that the causal text encoder has already mixed the attribute into the subject’s token, and shows figures. The T2I-CompBench measurement shows colour and texture binding improving, not why.
- Anything about relations, negation or many subjects. Prompts are templated conjunctions of two subjects with colours; relations are out of scope and do not improve.
- That the cross-attention maps explain the image in general. The paper’s claim that they become faithful after correction is qualitative.
Connections
- Benchmarks of composition (THEORY-522) place this method among the binding-specific baselines; in T2I-CompBench it helps colour and texture binding most.
- Composable Diffusion (THEORY-405, from LIT-770). This paper reports, qualitatively, that its conjunction fuses the two subjects into one object.
- CLAIM-321 cites this paper for a conjunction failing by omission of a conjunct, before any misbinding. That reading is NOTE-639’s.
- Classifier-free guidance (LIT-839) is on throughout at scale 7.5.
Note of 2026-10-29: what holds it back
No per-subject presence rate is in the record. The paper scores CLIP and caption similarity and a preference study, which its promote_when rules out, and T2I-CompBench++ (LIT-798) measures binding, not presence. GenEval’s detector scores two-object presence (THEORY-522), but not for this method and not against early attention. Narrowing to what was measured would leave only the method’s scores, which are machine-learning practice and belong to the anthology (NOTE-639).