nucleation

LIT-1540: Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

1 min read

Properties5
StatusActive — read 2026-10-26 (NOTE-1167) in full, from the arXiv copy; worth reading as the first end-to-end circuit for a natural behaviour in a pretrained language model, for path patching, for the backup heads that take over when the main ones are knocked out, and for its three tests of a circuit explanation, which its own circuit passes on average and fails under a greedy search (THEORY-980). Read it knowing that the behaviour is fifteen templates in one model, that MLPs are left out of the circuit though removing them breaks the task, that the dropout explanation of the backups is a hypothesis (THEORY-995), and that a few of its numbers and its greedy algorithm do not match their descriptions.
Version2
Tagsrepresentation-learning, causality, anthology-candidate
Cited as “paper” byNOTE-1167 — Wang, Variengien, Conmy, Shlegeris and Steinhardt — A circuit for indirect object identification in GPT-2 small
Cited as “source” byTHEORY-980 — In GPT-2 small on fifteen templates of indirect object identification, a circuit of 26 attention heads in seven classes, found by path patching, keeps 87% of the model's mean logit difference when every other head is mean-ablated; knocking out its three main name mover heads costs only 5%, because other heads take over, and knockout sets found by greedy search make the circuit and the full model differ by up to 87% of the logit difference, so the circuit is faithful on average and incomplete under knockouts, THEORY-995 — The attention heads in GPT-2 small that take over the indirect-object prediction when its main name mover heads are knocked out exist because the model was trained with dropout, which optimized it to be robust to disabled components, as Wang, Variengien, Conmy, Shlegeris and Steinhardt hypothesize

Graph View

Wang, K., Variengien, A., Conmy, A., Shlegeris, B. and Steinhardt, J. (2022). Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. arXiv:2211.00593.

Read in full on 2026-10-26 (NOTE-1167). Two accounts were drawn from it: THEORY-980 and THEORY-995.

Why this is here

Registered and read at the owner’s request of 2026-10-26, after an answer to “how’s the record on neural circuit theory?” found that the record held almost none of the core circuits literature. The record already relied on its circuit second-hand. The Mechanistic Interpretability Benchmark’s circuit track and its causal-variable IOI task are built on it (LIT-1550), and Makelov et al.’s interpretability illusion (LIT-968, THEORY-282) is shown on its MLP8, with the faithful residual direction written only by its S-inhibition heads. It is the record’s first full circuit for a natural behaviour in a pretrained model, as against the toy and attention-only models of LIT-1534 and LIT-1538.

Backlinks