Source
Eckart & Young (1936), “The approximation of one matrix by another of lower rank”, Psychometrika 1(3):211–218, LIT-330, read in full in NOTE-301.
What was actually shown
The problem. For a real n×N matrix a of rank R, find the matrix β of rank r < R nearest to a in the least-squares distance, the square root of Σ_ij (a_ij − β_ij)².
The tools, cited (Thms I–II, pp. 213–214). Every real matrix has a “canonic resolution” a = u′λU with u, U orthogonal and λ diagonal and non-negative, now the SVD. Two matrices are diagonalised by one orthogonal pair iff aβ′ and β′a are symmetric. Both are taken from the literature, not proved.
The argument (pp. 215–216). The distance is invariant under orthogonal transformations on either side (eq. 9). Writing β = u′μU and varying u and U along skew-symmetric increments, stationarity forces aβ′ and β′a symmetric, so a and β share singular vectors, and the squared distance becomes Σ_i (λ_i − μ_i)² with only r of the μ_i non-zero.
The result (eq. 17). With λ₁ ≥ λ₂ ≥ ⋯, take μ_i = λ_i for i ≤ r and 0 otherwise. “This is also the only solution unless λ_r = λ_{r+1}.” The minimum squared error is Σ_{i=r+1}^R λ_i², so the error is at most l(1 − r/R)^{1/2}, with l the length of a. NOTE-301 checked the bound.
A corollary, asserted (p. 216). The best rank-r approximation of a gives the best of aa′ (as ββ′) and of a′a (as β′β). The proof is “readily supplied” and not written out.
What could have come out otherwise. The best approximation could have needed singular vectors other than a’s. Stationarity rules that out.
What this does not say
- That the factors are unique. Uniqueness is of β, and only under a gap λ_r > λ_{r+1}. u and U are not unique: the rows of U beyond the rank are arbitrary, and equal singular values allow rotations. ββ′ determines β only up to an orthogonal transformation on the right.
- That the canonic components are the real underlying factors. The paper’s own last section argues that reading them as abilities and test demands, with λ² as variances, is “not a tenable interpretation”, because tests and individuals enter symmetrically (pp. 217–218).
- The spectral norm or other unitarily invariant norms. That is Mirsky (1960), not in the record.
- A complete optimality proof. The argument is first-order and assumes a minimiser exists; existence follows by compactness, which the paper does not say.
- Priority. Standard histories credit E. Schmidt (1907) with the result for integral operators. The reading did not check it.
- Anything about models trained separately. The theorem fixes the truncation of one given matrix; it says nothing about whether two models’ representations agree. A “Universal Convergence” argument that leans on it for uniqueness of factors goes beyond it (NOTE-301).
Connections
- THEORY-004: a representation is fixed by its kernel up to an orthogonal transformation. The ββ′ corollary is its matrix form: the best Gram-matrix approximation fixes β only up to that freedom.
- THEORY-002: convergence of kernels fixes representations only up to a symmetry group. Eckart–Young gives the low-rank truncation of a kernel, not a unique factor.