Runtian Zhai (2025), arXiv:2504.19792, PhD dissertation.
Key takeaways
From the reading, NOTE-770.
- The object is a singular value decomposition, not an eigenspace. The operator (T_P⁺g)(x) = E[g(A)|x] maps functions of the context to functions of the input. Its adjoint is given by Bayes’ rule, and the self-adjoint operators T_P⁺T_P⁺* and T_P⁺*T_P⁺ share the squared singular values (Lemma 1.4). The density ratio expands as Σ s_i μ_i(x)ν_i(a) (Lemma 1.5). Both compositions are assumed Hilbert–Schmidt.
- The optimality theorem. If 1 − ε ≤ s₁, an encoder minimizes the worst-case approximation error of a linear probe over tasks of compatibility at least 1 − ε if and only if it spans the top-d left singular functions. The error is then (s₁² − (1 − ε)²)/(s₁² − s²_{d+1}) (Theorem 3.4).
- Compatible tasks are nearly invariant. They vary little across two context draws for the same input (Theorem 3.2).
- Objectives that reach it. Balanced MSE, the spectral contrastive loss, covariance-constrained non-contrastive learning and a two-encoder objective (SVME) reach the span at their population optimum (Theorems 2.4, 2.9, 2.11). Cross-entropy and InfoNCE are outside the theory.
- Weakly supported. Alignment of trained networks with the span is shown on one tabular dataset. The diminishing-returns argument about scaling rests on it. The moderate-association metric correlates with error at a median of 0.587 across 28 datasets and fails on some.
Standing in the record
Filed on 2026-10-12 for QUESTION-052, and read on 2026-10-14 in NOTE-770. It supplies the singular-function result the question asked for, for a pairing between two spaces. THEORY-239 is filed from it as a partial answer. The directed case on a single space, a non-reversible kernel with A = X, is not worked in the thesis and stays open. It bears on THEORY-009: here self-adjointness comes from composing the operator with its adjoint, with no Riesz. It is a nearer miss for that account’s promote_when than LIT-228, but it does not analyse a BYOL-type objective. It also bears on QUESTION-060, where it treats architecture as an unworked part of the context. It is also held in the Anthology of the SOTA, as ANTH-LIT-459. Holding it in both follows ADR-013: the anthology reads it as a theory of what pretraining objectives learn, and this record reads it for QUESTION-052. The training recipes belong there.