Source
Karkada, Korchinski, Nava, Wyart and Bahri (2026), LIT-860, Propositions 1, 3 and 4, Corollary 2 and Appendix E, as read in NOTE-664. Proposition 1 and Corollary 2 are re-derived here.
What was actually shown
Setting. M_ij = (P_ij − P_iP_j)/(½(P_ij + P_iP_j)), split as M = M⁺ − M⁻ into its positive and negative parts, so |M*| = M⁺ + M⁻. An embedding W has W Wᵀ = |M*| with d ≥ rank M*. A set S of words sits on a periodic lattice Λ = (ℤ/L)^D with coordinates x_i. On S, the blocks of M⁺ and M⁻ each depend only on x_i − x_j (Assumption B.1). Then the Gram matrix of the vectors w_i, i ∈ S, is G_ij = c(x_i − x_j) for an even c. That is the S-block of |M*|, not |M_SS|, which is why B.1 rather than translation invariance of M is needed.
Proposition 1, re-derived.
- G is circulant. For each wavevector k of the dual lattice, e_k(i) = exp(ik·x_i) satisfies (G e_k)(i) = e^{ik·x_i} Σ_{y∈Λ} c(y) e^{−ik·y} = m̃(k) e_k(i).
- c is even, so m̃(k) is real and m̃(k) = m̃(−k). G is a Gram matrix, so m̃(k) ≥ 0. For k ≢ −k, cos(k·x) and sin(k·x) span the eigenspace of the pair {k, −k}. That pair is the degeneracy. At a Nyquist wavevector only cos survives.
- The unit eigenvectors are √(2/|S|) cos(k·x_i) and √(2/|S|) sin(k·x_i). A Nyquist mode’s is |S|^(−1/2) cos(k·x_i).
- Centring by H = I − 11ᵀ/|S| commutes with G, because the constant vector is the k = 0 eigenvector. It removes that mode alone.
- From the SVD of the centred embedding matrix, the coordinate of word i on the principal direction for eigenvalue λ_k with unit eigenvector u_k is √λ_k u_k(i). So the PCA coordinates are √(2 m̃(k)/|S|) cos(k·x_i) and √(2 m̃(k)/|S|) sin(k·x_i), in pairs, plus √(m̃(k)/|S|) cos(k·x_i) for Nyquist modes. This holds up to rotation within each degenerate eigenspace and the order of the directions.
Corollary 2, re-derived. For D = 1, take L sites with spacing 2/L and the kernel e^(−|Δx|/σ), periodised. With q = e^(−2/(σL)) and θ = 2k/L, m̃ = Σ_{m∈ℤ} q^|m| e^{−iθm} = (1 − q²)/(1 − 2q cos θ + q²), so a = √((2/L) m̃). As L → ∞, a² → 2σ/(1 + σ²k²), with k_n = πn: a closed loop with integer frequencies.
Proposition 3, as read. On an interval with open boundary, in the continuum limit, the exponential kernel is the Green’s function of (1 − σ²∂²)/(2σ) with Robin boundary conditions. Its eigenfunctions are sin kx with tan k = −σk, and, after centring, cos kx − sin k / k with tan k = k/(1 + σ(1 + σ)k²). The eigenvalue is 2σ/(1 + σ²k²) in both sectors. The curve is open, with non-integer frequencies (NOTE-664, C2, strong).
Proposition 4, as read. On a periodic lattice whose eigenvalues do not increase with |k|, the best rank-r linear probe for the latent coordinate has relative squared error O(r^(−1/D)) (NOTE-664, C3, strong for the periodic lattice).
Each could have come out otherwise. A kernel that is not translation invariant gives eigenvectors that are not Fourier modes. And B.1 is needed: invariance of M* alone does not make the S-block of |M*| invariant.
What this does not say
- That any corpus has this symmetry, or that a trained embedding or a language model reads it off. Every hypothesis above is about the statistics and an idealised factorisation. Whether Wikipedia’s month and year statistics satisfy B.1, and whether trained networks inherit the geometry, is THEORY-183, which stays Proposed.
- That word2vec is such an embedding. That is THEORY-182, shown only for a tied quartic proxy.
- Anything at moderate dimension. d ≥ rank M* is assumed. Compression is treated in the source only empirically (§4).
- Absolute amplitudes on real data. M* is indefinite, so the kernel fitted to M* is not the one factorised. Only relative amplitudes and frequencies are predicted.