nucleation

LIT-370: New Evidence of the Two-Phase Learning Dynamics of Neural Networks

2 min read

Properties5
StatusProposed — read in full 2026-09-30 (NOTE-318); a compact, readable demonstration that an early, short chaotic phase gives way to a confined, stable one in both basin and kernel terms, but it rests on two architectures on one dataset, one run each with no error bars. The chaos half largely re-measures Frankle et al.'s (2020) spawning / linear-mode-connectivity result with a parameter perturbation in place of fresh SGD noise, and the claimed role of the cone in generalisation is asserted. What would settle it: seeds and error bars, the batch size, a stated perturbation norm, other domains, and a quantitative test of confinement (e.g. an angular radius that stops growing) rather than a 3-D picture.
Version2
Tagslearning-theory, anthology-candidate
Cited as “paper” byNOTE-318 — New Evidence of the Two-Phase Learning Dynamics of Neural Networks
Cited as “source” byTHEORY-039 — The later phases reported in neural-network training differ in kind: after the transition a network may simplify its weights, acquire item-specific information, lose generalisation, confine its tangent kernel or saturate its units, and none of these is a measured compression of I(X;T)

Graph View

Zhou et al. (2025), arXiv preprint (v1 only; marked “Preprint. Under review.”); extends the ICLR 2025 Workshop DeLTa paper “On the Cone Effect in the Learning Dynamics” (not read) — arXiv:2505.13900

Standing in the record

Filed on 2026-09-30 at the owner’s request, as evidence on phases of training: whether they corroborate the information-bottleneck account of a fitting then a compression phase (LIT-324), and what phase structure they show. published: is the first appearance (ADR-002).

It was filed Deferred, unread. NOTE-318 is the close reading of 2026-09-30, and it placed the work: Proposed — a compact, readable demonstration that an early, short chaotic phase gives way to a confined, stable one in both basin and kernel terms, but it rests on two architectures on one dataset, one run each with no error bars. The chaos half largely re-measures Frankle et al.’s (2020) spawning / linear-mode-connectivity result with a parameter perturbation in place of fresh SGD noise, and the claimed role of the cone in generalisation is asserted. What would settle it: seeds and error bars, the batch size, a stated perturbation norm, other domains, and a quantitative test of confinement (e.g. an angular radius that stops growing) rather than a 3-D picture.

Backlinks