nucleation

LIT-986: Transformers learn factored representations

2 min read

Properties5
StatusActive — read 2026-10-14 (NOTE-758) in full; worth reading for QUESTION-060 as a controlled case with known relevant variables. When a generator factors, small transformers learn per-factor beliefs in roughly orthogonal subspaces of linear, not exponential, dimension. Under noise that makes factoring lossy they reach the factored form first and leave it later. Read it knowing that the processes are five small synthetic ones, that the objective is never varied, and that the paper leaves open whether the preference is architectural or a feature of training.
Version2
Tagsrepresentation-learning, anthology-candidate
Cited as “paper” byNOTE-758 — Shai et al. — Transformers learn factored representations
Cited as “source” byTHEORY-256 — When a sequence generator's latent dynamics factor as a tensor product, its predictive state embeds losslessly and linearly in orthogonal per-factor subspaces of total dimension Σ(d_n − 1); 4-layer GPT-2-style transformers trained by next-token prediction on five Mess3 and Bloch Walk factors settle near that dimension, in per-factor subspaces that are linearly independent and approximately orthogonal in their leading components, and under the token noise tested reach it first even when noise makes it lossy, THEORY-808 — Transformers trained by next-token prediction on sequence processes whose latent dynamics factor as a tensor product learn to represent the belief state in approximately orthogonal per-factor subspaces of about the factored dimension rather than the joint one, and reach that factored representation first even when noise makes it lossy

Graph View

Adam Shai, Loren Amdahl-Culleton, Casper L. Christensen et al. (2026), arXiv:2602.02385.

Key takeaways

From the reading, NOTE-758.

  • The theory is elementary and exact. Tensor-product operators keep a product belief a product (Prop. 2.2). Product beliefs embed linearly and invertibly in a direct sum of per-factor subspaces (Thm 2.3). Off the product manifold the embedding loses the total correlation among factors (App. A.5). Dependency chains between factors are covered.
  • What trained transformers do. With five factors they use about 10–11 effective dimensions, not 242, even at d_model = 480. Each factor’s belief is linearly decodable. The factor subspaces become nearly orthogonal in their first two components. The structure is visible already in the token embedding.
  • The noise result. With tokens randomized at rate ε, every run collapses to about 10 dimensions within hundreds of steps and then expands. For ε ≤ 0.05 it barely departs from the noiseless case.
  • Its limits. The paper calls the effect an inductive bias of the architecture, but its appendix leaves open whether training dynamics are the source. RNNs and LSTMs factor less sharply. All data are synthetic.

Standing in the record

Filed on 2026-10-12 for QUESTION-060, and read on 2026-10-14 in NOTE-758. THEORY-256 is filed from it as a partial answer: a family with known relevant variables in which the learned variables match them when factoring is lossless. When factoring is lossy, the cheap representation is the one training reaches first. It varies the data, not the objective or the architecture, so it does not apportion the two that QUESTION-060 names. The seeded reading that cheapness “fixes the learned variables” was stronger than the paper: under heavy noise the objective pulls the representation away from the factored form. The anthology-candidate flag stays. An anthology topic on interpretability could hold it, though no instruction for practice was found.

Backlinks