nucleation

LIT-1533: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

1 min read

Properties5
StatusActive — read 2026-10-26 (NOTE-1178) in full, from the article's own page on the Transformer Circuits Thread; worth reading as the demonstration that sparse-autoencoder features of a production model are abstract (multilingual, multimodal) and can steer it, for the finding that a concept gets its own feature only once its training frequency passes about one over the number of live features (THEORY-992), and for the observation that the most active features are mostly not the ones that matter to an answer (THEORY-987). Read it knowing that its interpretability and steering evidence is shown on features the authors selected, that the model, its size and some plot units are withheld, that the reconstructions explain only at least 65% of the variance, and that its wider thesis that these features are the model's working concepts is open (THEORY-965).
Version2
Tagsrepresentation-learning, anthology-candidate
Cited as “paper” byNOTE-1178 — Templeton, Conerly, Marcus, Lindsey et al. — Scaling monosemanticity
Cited as “source” byTHEORY-1016 — The features a sparse autoencoder recovers from a language model are a resolution set by the dictionary's width and sparsity penalty, not a unique set of the model's units: as the dictionary widens, features split into families of finer features, and splits also merge, so that the number of features is not well-posed, THEORY-965 — Sparse-autoencoder features of a large language model's residual stream are units the model computes with: they track abstract concepts across languages, modalities and concrete or abstract instances, and intervening on one changes the model's behaviour in the way its label predicts, better than directions found from a few labelled examples, THEORY-987 — In Claude 3 Sonnet's middle-layer residual stream, on two prompts for which every active sparse-autoencoder feature was ablated, how strongly a feature was active predicted its ablation effect on the answer poorly (correlation about 0.12) while its gradient attribution predicted it well (about 0.81), and on the multi-step prompt only three of the ten most active features were among the ten with the largest ablation effect, THEORY-992 — In sparse autoencoders of about 1M, 4M and 34M features on Claude 3 Sonnet's middle-layer residual stream, whether a concept among chemical elements, cities, animals and foods had a dedicated feature depended on how often it occurs in the training data: the frequency at which a concept became more than 50% likely to have one was slightly below one over the number of live features, and the three dictionaries' curves roughly coincided when frequency was rescaled by that number

Graph View

Templeton, A., Conerly, T., Marcus, J., Lindsey, J., et al. (2024). Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet. Transformer Circuits Thread, 21 May 2024. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html

Read in full on 2026-10-26 (NOTE-1178). Three accounts were drawn from it: THEORY-987, THEORY-992 and THEORY-965.

Why this is here

Registered on 2026-10-26 at the owner’s request, when the record was found to hold almost none of the core circuits literature. It is the scaling of Bricken et al.’s method (LIT-1545) to a production model, and the work most often cited for the view that sparse-autoencoder features are a large model’s concepts. The record held the contrary evidence first: in MIB (LIT-1550, THEORY-968) such features localize known causal variables worse than neurons. It bears on CLAIM-454, on QUESTION-275, and on the record’s Gemma Scope reading (LIT-611, THEORY-504) about residual-stream autoencoders.

Backlinks