Why it is a question
Three of the circuits works read on 2026-10-26 build circuits over the features of a learned dictionary: sparse autoencoders in Marks et al.’s sparse feature circuits (LIT-1532), per-layer transcoders in Dunefsky et al. (LIT-1554), and cross-layer transcoders in Ameisen et al.’s attribution graphs (LIT-1552). Each validates its circuit by interventions on those features. None asks whether another dictionary of the same fidelity would have given the same circuit.
The record has two reasons to doubt that it would.
- The units are a resolution. In Towards Monosemanticity and Scaling Monosemanticity, features split and merge as the dictionary widens, so “the number of features is not well-posed” (THEORY-1016), and whether a concept gets a dedicated feature depends on its training frequency relative to the dictionary’s size (THEORY-992).
- The authors concede the choice. Ameisen et al. say features split with the size of the cross-layer transcoder and that “there’s no guarantee that the CLT features are the most parsimonious way to split up the computation” (NOTE-1185).
Against that, Towards Monosemanticity’s features agree across two seeds better than neurons do (median activation correlation 0.72 against 0.46, NOTE-1191), but that compares features by their activations, not the circuits built over them.
It is THEORY-802’s thesis moved from alignments to dictionaries. THEORY-802 says interchange interventions can leave the mediating direction fixed only relative to a decomposition into units; here the decomposition is the dictionary, and the question is whether the circuit, not only the units, is relative to it.
What would answer it
- Invariance. For a stated behaviour and model, dictionaries trained with different seeds, or at different widths matched for fidelity (reconstruction loss, or the replacement model’s agreement with the model under perturbation, THEORY-994), give circuits that map onto each other: the same pathways, with features matched by their effects, up to splitting and merging stated in advance.
- Relativity. The same comparison gives circuits that differ in their pathways, not only in how finely the features are cut, each validated by its own interventions. Then the circuit is a fact about the model-and-dictionary pair.
A natural first case is subject–verb agreement across a relative clause, where sparse feature circuits find several converging pathways (THEORY-1014): whether those pathways survive a change of dictionary is the question in one behaviour.
What would not answer it
- Agreement of features across seeds by activation alone, which is what the record already holds.
- A single dictionary’s circuit validated more thoroughly. The question is about the comparison.
- A difference in circuits between dictionaries of unequal fidelity, which the fidelity gap could explain.
Note of 2026-10-27: a case from the hidden-objectives audit
Marks, Treutlein et al. (LIT-1584, NOTE-1193), read on 2026-10-27, report an observation of the kind this question asks about, though not a test of it. In their model organism, a 1M-feature SAE trained on the pretraining corpus alone gave better reward-model-bias features than one trained with chat data: by their account (footnote 18), even a small amount of chat-formatted data made such features merge into undifferentiated “Assistant” features. Here a dictionary’s training data, not its width or seed, decided whether the units an audit relied on came out separate. It is one observation, reported in a footnote, with no circuit compared across the two dictionaries.