nucleation

LIT-1576: Transformer Feed-Forward Layers Are Key-Value Memories

1 min read

Properties5
StatusActive — read 2026-10-27 (NOTE-1200) in full, from the arXiv copy (v2, the EMNLP 2021 version); worth reading as the origin of the reading of a transformer's MLP as a key–value memory, with each hidden unit a key over input patterns and each output row a value over the vocabulary, and for its evidence in one 16-layer language model: keys whose top training prefixes share recognizable patterns, shallow below and semantic above, and upper-layer values that agree, weakly, with what follows those patterns (THEORY-1033). Read it knowing that the general reading is wider than the evidence (THEORY-1030), that the agreement it reports peaks at about 3.5%, that the patterns were labelled by one annotator per key, and that ROME and MEMIT took the matrix-level version of the idea, not this unit-level one.
Version2
Tagsrepresentation-learning, anthology-candidate
Cited as “paper” byNOTE-1200 — Geva, Schuster, Berant and Levy — Transformer feed-forward layers are key-value memories
Cited as “source” byTHEORY-1030 — In transformer language models, each hidden unit of a feed-forward layer acts as a key–value memory, its input weights detecting patterns in the prefix and its output weights promoting the tokens that follow them, and a layer's output composes many such memories while the residual stream refines the prediction from layer to layer, as Geva, Schuster, Berant and Levy propose, THEORY-1033 — In a 16-layer transformer language model trained on WikiText-103, the training prefixes that most activate a sampled feed-forward hidden unit share patterns that human annotators recognize, mostly shallow in layers 1–9 and semantic in layers 10–16; projected through the output embedding, a unit's output weights put the next token of its top prefix first in up to about 3.5% of units in the upper layers and almost none below; and a layer's top prediction differs from every unit's in at least about 68% of examples

Graph View

Geva, M., Schuster, R., Berant, J. and Levy, O. (2021). Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 5484–5495. arXiv:2012.14913. doi:10.18653/v1/2021.emnlp-main.446.

Read in full on 2026-10-27 (NOTE-1200). Two accounts were drawn from it: THEORY-1033 and THEORY-1030.

Why this is here

Registered and read at the owner’s request of 2026-10-27, to fill a gap the circuits readings of 2026-10-26 named. ROME (LIT-1536) takes from it the reading of MLPs as key–value memories on which its storage hypothesis (THEORY-1011) and its edit rest, and MEMIT (LIT-1582) cites it for the same. Without it the record held the hypothesis but not its source, and could not see that the source’s keys and values are individual hidden units and output rows, while ROME’s are activation vectors and the whole output matrix. It is also an early account of the residual stream as a prediction refined layer by layer, which the circuits framework (LIT-1534) states in its own terms.

Backlinks