nucleation

LIT-952: Neural Networks Learn Statistics of Increasing Complexity

2 min read

Properties5
StatusActive — read 2026-10-14 (NOTE-777) in full; worth reading for its probes of what a network is using at a given checkpoint, which are maximum-entropy samples and optimal-transport grafts of class statistics, and for the result they give. Vision networks act on class means and covariances early in training and lose that later, with first-order sensitivity peaking before second-order. Pythia fits unigram text before bigram text. Read it knowing that the ordering is empirical and covers orders one and two, that the Taylor argument for it is a heuristic, and that its n-gram/moment theorem is an identity of one-hot indicators. Held as THEORY-279.
Version2
Tagslearning-theory, representation-learning, anthology-candidate
Cited as “paper” byNOTE-777 — Belrose et al. — Neural Networks Learn Statistics of Increasing Complexity
Cited as “source” byTHEORY-279 — Early in training, image classifiers and language models act on the low-order statistics of their inputs, first-order before second-order, and later lose that sensitivity as they come to use higher-order structure

Graph View

Nora Belrose, Quintin Pope, Lucia Quirke, Alex Mallen, Brennan Dury and Xiaoli Fern (2024), Proceedings of the 41st International Conference on Machine Learning, PMLR 235; arXiv:2402.04362.

Key takeaways

From the reading, NOTE-777.

  • Grafted statistics fool early networks. ConvNeXt V2, Swin V2 and RegNet-Y are trained on CIFAR-10 and four other small image sets.
    • Between roughly 2^4 and 2^12 steps they classify images whose pixel marginals were quantile-normalised to another class as that class.
    • ConvNeXt exceeds 40% against the target class on images given another class’s mean and covariance, near 2^10 steps; chance is 10%.
  • First order peaks before second, then both fall.
    • Accuracy on maximum-entropy samples that match only the class marginals peaks earlier and lower than on samples that match means and covariances.
    • Later, both fall, often below chance. MNIST and Fashion-MNIST barely fall. On SVHN, first-order samples never beat chance.
    • Model scale changes little.
  • The same order in language. Pythia, 14M to 12B, reaches its lowest loss on unigram-sampled text before bigram-sampled text. The loss then rises and falls again. The second fall is attributed, on correlational evidence, to in-context learning.
  • Theory is light.
    • The n-gram/moment equivalence (Thms. 2.1–2.2) holds because a product of one-hot indicators is a joint probability.
    • The ordering is motivated by a Taylor expansion of the loss in central moments, with an informal argument that its terms become relevant in order.
    • Third-order samples are generated, but no ordered result beyond second order is stated.

Standing in the record

Filed on 2026-10-12 from the owner’s papers-feed for QUESTION-039, and read on 2026-10-14 in NOTE-777. Its finding is held as THEORY-279, a partial and empirical answer to QUESTION-039: the order is the statistical order of the input, and acquiring later structure can retire earlier behaviour. Beside the record’s spectral accounts, THEORY-229 and THEORY-086, which order acquisition by eigenvalues of a second-moment operator, it marks their reach. Those can describe at most the early, second-order phase found here. Carries the anthology-candidate flag. The anthology was checked when it was filed, and it does not hold the work.

Backlinks