Source
- Bietti & Bach (2021), LIT-608, read in NOTE-475: Eq. 8 (degree blocks and their multiplicities) and Corollary 3 (k^(−d) decay at every depth).
- Cao et al. (2019), LIT-624, read in NOTE-483: Lemma 4.1 (sampled eigenfunctions are nearly orthonormal) and Theorem 4.2 (guarantees for whole eigenspaces, with an eigengap condition).
- Basri et al. (2019), LIT-612, read in NOTE-481: Theorem 4 and Figure 4 (the bias-free zero block, seen in a finite sample).
The argument
Step 1, linear algebra. Let J be the n × p matrix whose rows are the gradients of the network output at the n training inputs. For any matrix, JᵀJ and JJᵀ have the same non-zero eigenvalues with the same multiplicities. Under squared loss, the likelihood is Gaussian with noise precision β. The Gauss–Newton matrix is then βJᵀJ, and it is the Fisher information. JJᵀ is the empirical NTK Gram matrix, K_ij = ∇f(x_i)·∇f(x_j). So the non-zero Fisher spectrum is the empirical NTK spectrum times β. This step needs no source, and it is now CLAIM-377, with its proof. NOTE-475 records it as a fact of linear algebra and not a claim of LIT-608.
Step 2, from the sources. In the kernel regime the empirical NTK stays close to its infinite-width limit. On uniform data on S^(d−1), that limit’s integral operator has one eigenvalue µ_k per harmonic degree, with multiplicity N(d, k) (LIT-608, Eq. 8). Its non-zero µ_k decay as k^(−d) at every depth (Corollary 3). The sampled eigenfunctions are nearly orthonormal, ‖V_{r_k}ᵀV_{r_k} − I‖_max ≤ CM²√(log(r_k/δ)/n) (LIT-624, Lemma 4.1). So the leading eigenvalues of the Gram matrix are close to nµ_k, in groups of N(d, k).
Together. The lazy-regime Fisher of such a network should show plateaus of N(d, k) near-equal eigenvalues at heights near βnµ_k, ordered by height, with a null space of dimension at least p − n.
Step 3, thresholds. Within a degree block the population eigenvalues are exactly equal. In a sample they are split by an amount that shrinks with n. A threshold that falls inside a block keeps an arbitrary, sample-dependent subspace of an eigenspace that has no preferred basis. THEORY-019 says that the basis inside a multiplet is invisible to the spectrum. A threshold that falls in a gap keeps the span of every harmonic up to some degree, which is a stable object. Cao et al.’s guarantee has this shape. It is stated only for the first r_k eigenfunctions, a set closed under multiplicity, and its sample requirement grows as (λ_{r_k} − λ_{r_k+1})^(−2) (Theorem 4.2). Basri et al. add a warning (LIT-612, Figure 4). The bias-free two-layer kernel’s odd-degree block has eigenvalue exactly zero, yet in a finite sample it appears as the smallest non-zero eigenvalues. A threshold would remove those directions for the right reason without being able to tell them from directions the data merely under-determine. That the stability of a kept subspace is governed by the gap is a Davis–Kahan-type statement. The record holds no source for it, so this step is the filer’s.
What this does not say
- It does not cover cross-entropy. There the Gauss–Newton matrix is JᵀΛJ, with Λ the softmax Hessian, and the identity holds for Λ^{1/2}J instead. Papyan’s class-structured Fisher spectra (THEORY-078) are that case, outside the lazy regime. They show a different block structure, indexed by class rather than by degree.
- It does not say the blocks survive feature learning. Whether a trained network outside the lazy regime keeps any trace of the degree blocks is the comparison this account would make possible. It does not answer it.
- The ordering is by height, not degree. The constants in Corollary 3 depend on parity, so two adjacent degrees may swap. Gaps between degrees also shrink as k grows, so for high degrees the blocks blur together at any finite n.
- No source states it. Bietti & Bach say nothing about Fisher matrices, thresholds or evidence. Cao et al. say nothing about the Fisher. Each step is checkable, but the combination is the filer’s.
Connections
- Presupposes THEORY-086. That account gives the NTK’s eigenstructure and its role in training dynamics. This one carries the same blocks into the Fisher and asks what a threshold on them can mean.
- Extends THEORY-019. That account leaves near-degeneracy and the thresholds a test would need open. This one takes the case of a symmetry-forced block split by sampling noise, and says where a threshold is well posed.
- THEORY-080 uses the same spectrum to compute an Occam factor. There, directions inside a block are priced equally, so no threshold is needed.
Note of 2026-10-19: Step 1 filed as the record’s own
Step 1 is CLAIM-377, demonstrated and Active: the model’s Fisher under the Gaussian likelihood is βJᵀJ, and it shares its non-zero spectrum and multiplicities with the empirical NTK Gram matrix JJᵀ, at every width, in every regime and on any data. Other entries, such as CLAIM-164, can cite that claim for the identity. What keeps this THEORY Proposed is Steps 2 and 3: the sampled Gram spectrum’s approximation of the degree blocks, the Davis–Kahan-type step, and the numerical check its promote_when names.