THEORY-1007: On four subject–verb agreement tasks in Pythia-70M and Gemma-2-2B, circuits of sparse-autoencoder features and autoencoder error nodes, selected by linear estimates of indirect effect, carried half or more of the behaviour on held-out pairs with about 100 and 500 nodes, where circuits of neurons needed about 1,500 and more than 30,000; with the error nodes removed, the same feature circuits carried little of it, under about a quarter in Pythia and almost none in Gemma

Source

Marks, Rager, Michaud, Belinkov, Bau and Mueller (2025), Sparse feature circuits, ICLR 2025, arXiv:2403.19647, LIT-1532, read in full in NOTE-1187: §2 (formulation), §3.1–3.2 with Figure 3 (read from a rendering of p. 5), and Appendices A, B and H.

What was actually shown

The decomposition. At each site with an autoencoder, x = Σ f_i(x)v_i + b + ε(x). Feature activations f_i and the error ε are nodes; a node’s indirect effect on m = log P(correct verb) − log P(wrong verb) for a contrastive pair (subject number flipped) is estimated by attribution patching or integrated gradients, averaged over the training pairs, and thresholded. Neuron circuits are found the same way with basis directions and no error terms.

Faithfulness. On held-out pairs, nodes outside the circuit are mean-ablated; the first third of each circuit is left unablated and unscored. Averaged over the four tasks, feature circuits pass one half by about 100 nodes (Pythia) and 500 (Gemma). Neuron circuits reach about 0.5 at 1,500 nodes in Pythia; in Gemma they are near 0.3 at 30,000, the edge of the plot, and the paper puts the half point at 50,000.

Without the errors. The curves for the feature circuits with every error node removed stay under about 0.25 in Pythia and near zero in Gemma. With only the attention and MLP error nodes removed they stay much closer to the full feature circuits. “Removing residual stream SAE error nodes severely disrupts the model.”

Completeness. Ablating a few feature nodes removes the behaviour even with all error nodes left in; hundreds or thousands of neurons are needed for the same.

What could have come out otherwise. The features could have carried the behaviour by themselves, with the error terms negligible. In these models and tasks they did not.

What this does not say

• Not that features beat neurons as units. A feature circuit’s error nodes are dense and high-dimensional, and the authors say they “cannot be fairly compared to neurons”.
• Not what the errors contain. Features the dictionary missed, or structure no sparse dictionary of this width captures.
• Not about the first layers. The first third of each circuit is not scored, because held-out pairs use different tokens.
• Not general. Two small models, one family of tasks with templated contrasts.

Connections

• CLAIM-454. Features recovered without the task do not carry it on their own; what does is features plus a remainder, arranged across positions and layers.
• THEORY-968. Two results that point the same way from different tests: SAE features localize supplied variables worse than neurons, and SAE features without their error terms carry little of a behaviour.
• THEORY-504. The residual-stream errors matter most here, as residual-stream autoencoders cost Gemma 2 the most loss there.
• THEORY-987. The same family of linear estimates selects the nodes.