Source
- Papyan, Han & Donoho (2020), LIT-618, read in NOTE-476: §2J (the four collapses), Figures 2–8, Table 1, Theorems 2, 4 and 5, §7B.
- Papyan (2020, JMLR), LIT-613, read in NOTE-474: Figures 7–10 (class means of features and errors across layers).
What was actually shown
The measurements. The terminal phase of training (TPT) begins at the first epoch with 99.9% training accuracy (99.6% on ImageNet), and training continues while the loss is pushed toward zero. The networks were VGG, ResNet and DenseNet, on MNIST, Fashion-MNIST, SVHN, CIFAR-10, CIFAR-100, STL10 and ImageNet. Four quantities fall through TPT (LIT-618, Figures 2–7):
- NC1: within-class variability relative to between-class, Tr(Σ_W Σ_B†)/C, on a log scale, continuing well into TPT;
- NC2: the spread of class-mean norms and of pairwise cosines, and the average distance of the cosines from −1/(C − 1), so that the centred means approach a simplex ETF;
- NC3: the distance between the normalised classifier and the normalised centred means;
- NC4: the disagreement between the network’s decision and nearest class mean on test data.
Every measure moves in the stated direction in all 21 cells, but not monotonically in all of them, and not to zero everywhere (see the note of 2026-10-20). Each cell is one trained model per learning rate, with the best final test error selected (NOTE-476, Limitations).
The theorems, which assume the end state. If features have collapsed (NC1) onto a simplex ETF (NC2), the MSE-optimal Webb–Lowe classifier and the max-margin limit of gradient descent on cross-entropy are both proportional to the centred means. The decision is then nearest class mean, which gives NC3 and NC4 (Theorems 2 and 4). For a Gaussian channel h = μ_γ + z with ‖μ_c‖ ≤ 1 and a linear decoder, the simplex ETF with the classifier equal to the means is the unique configuration that maximises the error exponent as noise vanishes (Theorem 5).
Before the last layer. Papyan (2020) did not cite this paper. He finds that feature class means separate from the bulk and become closer to orthogonal with depth, and that error class means do the same toward the output (LIT-613, Figures 7–10). That is the layer-by-layer approach to what neural collapse describes at the last layer. It is consistent with this account and does not test it.
What this does not say
- It does not say why. The theorems take the collapsed state as given. Why gradient descent on cross-entropy reaches it is the question the authors leave next.
- It does not say the terminal phase improves generalisation. The median test-accuracy gain from first zero error to the last epoch is 0.35 points, three of 21 cells lose accuracy, and there are no seeds or intervals (Table 1). The robustness gain (Figure 8) is consistent in direction, measured with one attack on 100 images.
- It does not say the Hessian’s class outliers are a symptom of collapse. The paper argues it in §7B. Under NC1 the last-layer feature matrix tends to rank C − 1, so the class means dominate and the outliers leave the bulk. It measures nothing to support the argument. The record keeps the measured part, that the outliers sit in the class-mean span and separate as between-class variation outgrows within-class variation, in THEORY-078.
- It covers balanced classes, no augmentation and the last layer. Whether collapse holds with augmentation, label noise, imbalance, or C larger than the feature dimension is an open question the paper names. For imbalance the record now holds an answer in free-features models (THEORY-286, from LIT-994): NC1 survives, but the equiangular NC2 holds for balanced or binary classes only. An imbalanced test of NC2 should be read against that geometry.
Connections
- THEORY-078. The class-mean outliers of the Fisher are the spectral face of the class means this account describes. Collapse would be the limit in which within-class terms vanish, but the record does not hold a measurement joining the two.
- THEORY-039 compares later phases of training reported elsewhere. The terminal phase is one more, defined by a different quantity, and is not part of that comparison.
- Information theory. Theorem 5 is a codebook result for a Gaussian
channel in the large-deviations regime, which is why this account
carries
information-theory.
Note of 2026-10-13: a symmetry account of collapse
Ziyin (2024), LIT-1014, read in NOTE-716, attributes neural collapse to symmetries of the loss (THEORY-213). Symmetric subspaces are absorbing for gradient training, and favoured when weight decay is strong. This account’s protocol uses weight decay. The paper offers a mapping, not a derivation: it describes neural collapse only as the penultimate representation becoming low-rank, and does not address NC1–NC4 as this account defines them. Whether the symmetry account predicts the simplex equiangular geometry, or only low rank, is open. It bears on QUESTION-080 too: an absorbing collapsed state would stay fixed while other parameters keep moving.
Note of 2026-10-20: what the plots show
Promotion audit, batch 052. NOTE-476 had read Figures 2–7 from their captions; they are now read from the plots (NOTE-476, dated note). Collapse is close to complete on MNIST, Fashion-MNIST, SVHN and CIFAR-10: NC1’s ratio Tr(Σ_W Σ_B†)/C falls to about 10⁻⁴–10⁻² and nearest class mean agrees with the network on almost every test example. It is partial on CIFAR-100 and STL10 and clearly incomplete on ImageNet, where the ratio ends near or above 1 for all three networks and nearest class mean disagrees on about a tenth of test examples. Several cells rise and fall after the terminal phase begins, and some of the largest drops coincide with the learning-rate cuts. The title is narrowed to these runs and to what they show. Whether collapse holds outside this protocol, and why training reaches it, is QUESTION-202.
Note of 2026-10-20: the wider claim filed
The promotion audit narrowed this entry to what was shown. Under ADR-047 v4 the claim the narrowing dropped, neural collapse as a general law of training past zero error, beyond Papyan, Han and Donoho’s protocol, is now filed as THEORY-788, Proposed, which extends this entry and takes it as evidence. This entry’s title and status are unchanged.