nucleation

LIT-1552: Circuit Tracing: Revealing Computational Graphs in Language Models

1 min read

Properties6
StatusActive — read 2026-10-26 (NOTE-1185), main text in full and five appendices, from the Transformer Circuits page; worth reading for the attribution graph, a prompt's computation rewritten exactly in sparse features of a local replacement model (THEORY-979), for its own measurements of how far that replacement responds to perturbation like the model (THEORY-994), for the parallelogram constraint on one-step sums (THEORY-971), and for a limitations section that names where the method fails: attention, error nodes, inactive features. Read it knowing that the graphs explain the replacement, not the model, that intervening confirms a direct edge almost by construction, that the replacement matches the model's top token on about half of prompts, and that its faithfulness figures differ between the main text and the appendix.
Version2
Tagsrepresentation-learning, causality, anthology-candidate
ExtendsLIT-1554 — Transcoders Find Interpretable LLM Feature Circuits
Cited as “paper” byNOTE-1185 — Ameisen et al. — Circuit tracing: revealing computational graphs in language models
Cited as “source” byTHEORY-971 — A unit that thresholds a pre-activation equal to a term depending only on one operand plus a term depending only on the other cannot be active on exactly the pairs with a given sum when that sum has two decompositions: if it is active on (x, y) and (z, w), it is active on (x, w) or on (z, y); so a sum computed from separately represented operands needs an earlier unit that combines them, as Ameisen et al. argue for addition in language models, THEORY-979 — On one prompt, a transformer whose MLPs are replaced by a cross-layer transcoder, with its attention patterns and normalization denominators frozen and each MLP's reconstruction error added back as a constant, reproduces the model's activations and outputs exactly and is linear between features, so each active feature's pre-activation is the sum of its attribution-graph edges and constants; a perturbation that clamps MLP outputs up to the target feature's encoder layer then confirms a direct edge by construction, and tests only knock-on effects and the output, THEORY-994 — In Ameisen et al.'s evaluations, the local replacement model of their 18-layer language model, built from a 10-million-feature cross-layer transcoder, responded to small perturbations much as the model did one layer after the intervention and increasingly differently over later layers, more in the size of the effect than its direction and slightly more for larger dictionaries, and over 20 prompts the graph's influence of one feature on another ranked the effect of ablating the first on the second with a Spearman correlation of 0.72

Graph View

Ameisen, E., Lindsey, J., Pearce, A., Gurnee, W., Turner, N. L., Chen, B., Citro, C., et al. (2025). Circuit tracing: Revealing computational graphs in language models. Transformer Circuits Thread, 27 March 2025. https://transformer-circuits.pub/2025/attribution-graphs/methods.html

Read on 2026-10-26, main text in full (NOTE-1185). Three accounts were drawn from it: THEORY-979, THEORY-994 and THEORY-971.

Why this is here

Registered and read at the owner’s request of 2026-10-26, after “how’s the record on neural circuit theory?” found that the record held almost none of the core circuits literature. It is the method behind most circuit claims about large language models since 2025, and the companion of LIT-1541, whose case studies use it. It extends Dunefsky et al. (LIT-1554). For the record’s circuits line it supplies what the line lacked: a stated, testable sense in which a circuit is read off a large model, and the authors’ own measurement of where that reading departs from the model. It bears on CLAIM-454, on THEORY-022 (its addition features against the grokking network’s Fourier circuit), and on QUESTION-275.

Backlinks