Source
Musat, Pimentel, Zucchet and Hofmann (2026), arXiv:2607.11875, LIT-1530, read in full in NOTE-1177: §4.3, Fig. 4, Appendix E.1 (setup) and E.2 (Fig. 7, twelve seeds).
What is asserted, and by whom
The measurement. A three-layer, one-head-per-layer attention-only transformer of width 256, trained with vanilla SGD on in-context bigrams with every token rare (fr = 1), so only an induction circuit solves the task. Three induction circuits are possible: on layers (1, 2) with units (α, β, γ), on (1, 3) with (α, β′, γ′), and on (2, 3) with (α′, β″, γ′). After every step the weights are projected onto the seven coordinates α, α′, β, β′, β″, γ, γ′. Each seed draws the initial strengths of β and β″ uniformly from [0, 0.6], zeroes every other coordinate, and is coloured by the circuit whose ablation-reduced model has the lowest loss after training. The scan “aggregates several thousand seeds”.
The pattern (the paper’s words). “The simplest initialization-based predictor … would be that each circuit wins when its own units start large.” It holds for the (2, 3) circuit. “However, the (α, β′, γ′)-circuit … seems to benefit from a larger β, which is not part of this circuit. Moreover, the (α, β, γ)-circuit … seems to require not just a large β … but also a large β″ (not part of the circuit).” The boundaries “are very sharp between the blue and green solutions and between blue and red, but not between green and red”.
The reading, the authors’. “This is because the red and green circuits are cooperative: both share α, while β and β′ are in different layers. Meanwhile, blue is competing with both red and green, trying to use the same layers in different ways. This might also explain why red requires a large β″: it stimulates the blue circuit, which in turn inhibits the green circuit.”
The general thesis filed here is that reading: interaction between circuits in training is set by the coordinates and layers they share.
What the record holds toward it
- THEORY-981 (Active): in this class the circuits are sets of coordinates of an invariant subspace, so “sharing a unit” is well defined, which it is not for circuits found by head-level attribution.
- THEORY-1009 (Active): two circuits that solve overlapping parts of the data interact through the gradient they share even when their coordinates are orthogonal. That is the competitive half in another form.
- Nothing in the record derives the boundaries, and the scan was run in projected training only.
What would confirm or reject it
The promote_when above.
Bearing
This is the interaction QUESTION-275’s construction proposes to model, coupling a feature allocation to circuit interaction: here the allocation (which units each circuit uses) is exact and the interaction follows it. If the thesis holds, the question’s coupled description has a known comparator in this class, the reduced dynamics on the shared coordinates.