Source
Meng, Bau, Andonian and Belinkov (2022), NeurIPS 2022, arXiv:2202.05262, LIT-1536, read in full in NOTE-1166.
What was actually shown
Causal tracing. For each of 1000 prompts that GPT-2 XL completes correctly (mean clean probability 27.0%), Gaussian noise of three embedding standard deviations is added to the subject tokens (mean probability 8.47%), and the clean value of one state is restored. Averaged: ATE 18.6%; the strongest single states at the last subject token around layer 15; MLP outputs (windows of ten layers) peak there at AIE 6.6%, attention at 1.6%; attention peaks at the last token in late layers. Freezing MLPs at their corrupted values removes the effect of low-layer states at the last subject token, freezing attention does not. Similar traces in GPT-J, GPT-NeoX and smaller GPT-2s, and under other corruptions.
Editing. ROME computes k* (the MLP’s activation at the last subject token, averaged over random prefixes) and v* (optimized to make the layer’s output predict the new object, with a KL term preserving the subject’s other predictions), and inserts the pair by the rank-one least-squares update. On CounterFact (GPT-2 XL, 7,500 records): S = 89.2 against FT+L 66.9, FT 65.1, MEND 57.9, KE 52.2, KN 35.6; FT’s neighbourhood score is 40.4, FT+L’s paraphrase score 48.7. On GPT-J (2,000 records): 91.5 against FT+L 68.7. Swept over layers and tokens, ROME is best at the last subject token in middle layers; editing late attention reproduces the prompt but not paraphrases.
What could have come out otherwise. Effects could have been diffuse, or concentrated at the last token only; attention could have dominated the early site; edits at the traced site could have done no better than elsewhere; and some baseline could have matched ROME on both generalization and specificity.
What this does not say
- Not where facts are stored. That inference is THEORY-1011. Makelov et al. (LIT-968) show that a rank-one edit with the same output exists for any one-dimensional fact patch in an MLP, and that ROME edits are approximated by subspace patches over layers 20–35.
- Not that single layers matter. MLP and attention effects are for windows of ten layers; single-layer restorations were “generally negligible”.
- Not independent of the corruption. The effect is relative to a noised, off-distribution run.
- Not about other knowledge (logical, spatial, numerical), or many edits at once.
A number the paper gives two ways
§2.1 puts the strongest single-state effect at “AIE=8.7% at layer 15”; Appendix B.1’s figures for the same peak give 15.0% − 8.47% = 6.5%. The title uses neither.
Connections
- THEORY-282 (Makelov et al.’s illusion) bears on what these results license, not on the results.
- THEORY-980. The IOI circuit leaves MLPs out; here MLPs are the decisive site for recall. Both are GPT-2-family models read with patching-type interventions.
Note of 2026-10-27: two later readings, consistent with it
- Hase et al. (LIT-1578, THEORY-1024) find in GPT-J that the per-fact tracing effect does not predict where an edit of that fact works best. That leaves this entry standing: its “working best … in middle layers” is an average over facts, and Hase et al. find ROME best on average at layer 6, near where tracing peaks on average.
- MEMIT (LIT-1582, NOTE-1197) traces 501 facts in GPT-J and reproduces this entry’s GPT-2 XL pattern, including the gap when MLPs are severed, and adds a strong early attention effect at the last subject token. The measurements now have a second model; this entry’s title stays scoped to GPT-2 XL.