Source
Wang et al. (2022), LIT-1540, read in full in NOTE-1167: §3.4 and Appendix F.
What is asserted
That redundancy in the trained network, heads that stay quiet in normal operation and write the answer once the main heads are removed, is a product of a training condition: dropout, which randomly disables activations during training and so rewards computations that survive missing parts.
What the record holds toward it
- The compensation, measured (THEORY-980): knocking out name movers 9.9, 9.6 and 10.0 costs 5% of the logit difference, the negative name movers become less negative (10.7 turns positive), and eight backup heads become the main contributors.
- Nothing on the cause. No model trained without dropout is examined. The authors’ footnote adds that perfect redundancy is unlikely under SGD, and more likely when one analyses a sub-distribution of the training data.
Why it stays Proposed
It is a one-sentence hypothesis with an unexamined alternative, which is the record’s observation, not the paper’s. Mean ablation of heads that later heads read from changes those later heads’ inputs, and attention heads can change what they attend to when their inputs change, which the backup heads’ altered behaviour shows. So compensation could come from how attention responds to the ablation, with or without dropout.
Connections
- CLAIM-367 and CASE-113. If it holds, it is a case of a property of a trained network (redundancy that hides components from single knockouts) whose source is a condition of the training regime, the optimizer–model–data system of CLAIM-367, not the network alone.
- THEORY-980 holds the compensation this would explain.