Source
Meng et al. (2022), LIT-1536, read in full in NOTE-1166. Hase et al. (2023), LIT-1578, read in full in NOTE-1194, is the second source: it measures what the conjecture predicts, though it does not establish it.
What is asserted
Two things, both from Meng et al. §2.3. Mid-layer MLPs act as linear associative memories from a subject key to an attribute value, and the middle range has no privileged layer for a given fact: “any fact could be equivalently stored in any one of the middle MLP layers”. The paper rests the second clause on Zhao et al.’s finding that BERT’s layers can be reordered.
It does not assert the claim THEORY-1011 was rejected for: that the middle layers where causal tracing peaks are where a given fact is stored.
What the record holds toward it
- Consistent with it: THEORY-1024. In GPT-J, on 652 CounterFact facts, ROME rewrites a fact at over 96% at every layer but the last, and the per-fact tracing effect adds at most 3.2 points of R² to edit success once the edit layer is known.
- Why that is not support: LIT-968 and THEORY-282. Any one-dimensional fact patch has a rank-one edit with the same output at that token, so an edit’s success at a layer shows that the layer can write the output, not that it held the fact. Edits that work everywhere are what distributed storage predicts, and also what “edits create the association where they are made” predicts.
- Nearest rival: THEORY-1026, Hase et al.’s account. Localizing and editing answer different questions, and “many layers could hold a fact and some happen to”. It is weaker than “any … equally”, and it rests on override, not storage.
- Not yet tested: no work in the record ablates candidate memories on clean inputs, layer by layer.
Why it stays Proposed
Every piece of evidence for it so far is an edit, and edits cannot tell storage from writing. The test it needs, a clean-run ablation layer by layer, has not been run in any work the record holds.
What would confirm or reject it
The promote_when above.