Source
Wang et al. (2024), LIT-1535, read in full in NOTE-1188: §4.3, Figs. 5–7, and appendix B.6.
What is asserted
That in this model a circuit’s components are not new structure grown into idle heads. Heads that already did something, predicting simple n-grams, change to new roles as later circuits form, and the old roles either lapse or move to other heads.
What the paper shows toward it
- Roles at two checkpoints. Token-in-context classification at 8.5k and 50k steps (Fig. 9) puts heads 0:1, 0:4 and 0:7 among n-gram heads early and among previous-token and bracket-matching heads late. Head 0:0 goes from attending to whitespace to a standard multigram head.
- Information given up. With a trained one-layer model’s outputs as the data, the layer-0 multigram heads’ data-refined LLC falls from LM3 onward, which the authors read as their losing simple n-grams that a one-layer model can predict.
- A new dependency. Path patching shows head 1:5’s loss on nested brackets coming to depend on the layer-0 multigram heads at that time, and K-composition scores between the two layers turn around together.
- Roles moving. The “ago” n-gram is head 0:0’s at 8.5k steps and head 1:0’s at the end; the “more…th → an” comparative goes from 0:7 to 1:1.
Why it stays Proposed
The role changes rest on two snapshots in one run, classified by mean-ablation effects with manual inspection. “Repurposing” says more than “a head had one role and later another”: it says the later role was made from the earlier one. Nothing in the paper tests that, and judging that the “ago” n-gram “migrated” from one head to another is continuity by resemblance of the affected tokens, which CLAIM-329 says is not enough for continuity of the thing.
What would confirm or reject it
The promote_when above.
Connections
- QUESTION-275 asks whether a circuit “emerge[s] by acquiring new features, reorganizing existing features, or both”. This is the record’s best evidence for “reorganizing”, at the level of components.
- QUESTION-039, on whether acquisition is one kind of event: this would make repurposing one kind, beside growth.
- THEORY-279. Low-order structure first, then lost, is the same shape within a single transformer.
- LIT-1534. Elhage et al.’s mathematical framework is where the two-layer attention-only setting and the QK/OV reading of heads used here come from; its path expansion is THEORY-988.