Source
Liu, Liu, Michaud, Gore and Tegmark (2025), LIT-951, read in NOTE-794: §§2.2–2.6 and the paper’s framing of the Domino effect in skill learning.
What is asserted
That when skills share parameters, an optimizer that acts on each coordinate’s sign, as sign descent does and Adam partly does, gives the dominant skill nearly all the update directions until its gradient vanishes, and only then lets the next one move. Under plain gradient descent orthogonal skills do not interact, and each proceeds at a rate set by its frequency. So the order of skill onsets in real training is in part an effect of the optimizer.
What the Active account establishes toward it
THEORY-253 holds the Geometry model’s simulations, reproduced by the record: skills progress together under gradient descent and in sequence under sign descent, with the ratio of onset times approaching 2, and on a rotated quadratic only when the eigendirections are misaligned with the coordinates. That is a toy model and simulation, with no derivation and no trained network.
Why “one source” and not “the cause”
The title THEORY-253 carried before it was narrowed said sequential onset “comes from the optimizer”. The paper’s own §5.1 shows dependent skills in a two-layer network starting after their prerequisites, beyond what frequency predicts (NOTE-794, C5). So the paper does not assert the optimizer as the only cause, and this entry does not.
What would confirm or reject it
The promote_when above, QUESTION-193’s test.
Provenance
Filed on 2026-10-20 by the narrowing-loss sweep (ADR-047 v4, batch 068), from the wider title THEORY-253 carried before it was narrowed on 2026-10-19.
Note of 2026-10-26: dependency measured, and a third source
Two of the works T24 of the exchange “Experimental Infrastructure Review” drew on (QUESTION-275), read on 2026-10-26, bear on this account’s “one source”.
- Dependency, measured (THEORY-973). In small attention-only transformers, in-context learning arrives abruptly in the window where induction heads form, and the induction head needs two heads composed. Letting a single layer attend by the previous token, which removes the need for the composition, brings the jump to one-layer models and moves it earlier in two-layer ones. That is the other source this title names, measured in trained networks by an architectural intervention.
- Redundancy, a third source (THEORY-1009). In Musat, Pimentel, Zucchet and Hofmann’s idealized two-layer model, once an in-weights circuit has memorized the common pairs, the induction circuit’s gradient is scaled by the share of the data the memorized map does not solve. The two circuits occupy orthogonal coordinates and still interact under plain gradient descent, because they solve overlapping parts of the data.
The second does not meet the refutation clause. Its circuits are not independent skills with separate loss terms, which is what “orthogonal skills” means in the Geometry model, and the delay in the paper’s one run owes more to a small-initialization plateau than to the factor, which is 0.9 there. Neither work varies or names the optimizer, so neither bears on the optimizer clause or on QUESTION-193’s test, which stays as stated. “Why one source” now has two others beside it. The account is unchanged.