nucleation

THEORY-802: In trained networks as in constructed ones, interchange interventions on a fixed set of variables can be satisfied equally by alignments related by a change of basis, so which internal direction mediates a high-level variable is fixed only relative to a chosen decomposition into units, and complex models will generally have several distinct, fully correct abstractions

4 min read

Properties6
StatusProposed
Version2
Tagscausality, philosophy-of-science, representation-learning
SourceLIT-961 — A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Arguments, LIT-968 — Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching
ExtendsTHEORY-243 — In the three-unit identity network x ↦ x(1, 0, 1) ↦ (0, 2, 1)·h, aligning the input with the third unit and aligning it with the direction (h₁ + h₂)/√2, two alignments related by an orthogonal change of basis, pass every interchange intervention on the input equally, so interchange interventions on that variable alone do not fix which internal direction mediates it
AnswersQUESTION-234 — Do interchange interventions leave the mediating direction underdetermined in trained networks: do two basis-related alignments in a trained model tie on every interchange drawn from the data, and for which networks are alignments that agree on all such interventions unique up to a stated symmetry?

Graph View

Source

Wu et al. (2024), LIT-961, Appendix A, read in NOTE-809: after the rotated identity network, “there are still likely to be multiple distinct fully correct abstractions” for complex models, offered as a conjecture. Makelov, Lange and Nanda (2023), LIT-968, read in NOTE-782, §8 (“The importance of correct model units”): if activation spaces may be re-parametrized across component boundaries, calling a subspace “illusory” is much more tenuous, and they hold that components should be privileged because they do qualitatively different work.

What is asserted, and by whom

The thesis is that the tie THEORY-243 proves in a constructed network is the general case: interchange interventions on a fixed set of variables leave the mediating direction underdetermined up to a change of basis, so the direction is fixed only once a decomposition into units is chosen, and trained models of any complexity will have several correct alignments. Wu et al. assert it as a conjecture. Makelov et al. accept the premise for the toy and dispute only the conclusion drawn about units.

What the record holds toward it

  • THEORY-243 (Active): the exact tie in the network x ↦ x(1, 0, 1) ↦ (0, 2, 1)·h, re-derived and checked on 1000 input pairs. It is an existence result, which is all this thesis has.
  • Wu et al. note that intervening on further variables separates the two alignments in the toy. If that holds generally, the thesis survives only for interventions on a fixed, small set of variables.

What would confirm or reject it

A trained-network tie of the kind the promote_when describes confirms it. A uniqueness bound up to a stated symmetry, for a class of networks that includes the trained models at issue, rejects it there. QUESTION-234 asks the same thing as a question; this entry is the answer Wu et al. propose.

Note of 2026-10-27: the circuits works, and dictionaries

The core circuits works, read on 2026-10-26 at the owner’s request, bear on this thesis without meeting its promote_when.

  • A trained-network case, of refinement rather than of a change of basis. In BERT on MoNLI, Geiger et al.’s DAS (LIT-1551) finds the entailment variable at one site and finds it decomposing there into the two input words’ identities (THEORY-983). Two high-level models pass at one site, but they are related by refinement, not by a change of basis crossing component boundaries, so this is not the tie the promote_when asks for.
  • The verdict changes with the basis. DAS’s boolean toy fails in the neuron basis and passes once rotated back (NOTE-1192), and its learned rotations reach IIA 1.00 where the closest axis-aligned alignments reach 0.73 and 0.51 (THEORY-983). That supports “fixed only relative to a chosen decomposition into units”; it is the converse of THEORY-243, where a change of basis leaves the score unchanged.
  • Several sufficient circuits, at the level of components. A greedy pruner of the ACDC kind keeps one input of a redundant OR gate, chosen by the order it visits edges (THEORY-966): the thesis’s shape for subgraphs rather than directions, in a constructed case.
  • Instruments that cannot see a tie. Causal scrubbing scores a hypothesis only through the partition of the data it induces (THEORY-978), and sparse feature circuits score nodes by their effect on one metric under one family of contrasts (NOTE-1187). Basis-related alignments that induce the same partitions, or the same effects, tie under both by construction.
  • Where the freedom comes from in transformers. Elhage et al.’s framework (LIT-1534, NOTE-1189) states it: the residual stream has no privileged basis, and rotating it together with every matrix that reads from or writes to it leaves the model unchanged. Its path expansion (THEORY-988) is built from products that the rotation leaves alone, so the non-uniqueness does not touch it.
  • Dictionaries. Sparse-autoencoder features are a resolution set by dictionary width (THEORY-1016), and Ameisen et al. (LIT-1552) concede that “there’s no guarantee that the CLT features are the most parsimonious way to split up the computation” (NOTE-1185). Whether two dictionaries of equal fidelity give the same validated circuit is this thesis asked of dictionaries; it is filed as QUESTION-274.

Status, title and promote_when are unchanged.

Backlinks