Source
Johnson, El Hanchi & Maddison (2023), LIT-258, App. B.2–B.3, read in NOTE-751 (its claim C1, “strong”, “exact”).
What was actually shown
Let A be finite, p⁺ the positive-pair distribution on A × A with marginal p, and K⁺ = p⁺/(p⊗p).
- Spectral contrastive loss. Over all kernels K̂ on A × A, the loss −2 E_{p⁺}[K̂] + E_{p⊗p}[K̂²] equals Σ_{a,b} (p⁺(a,b)/√(p(a)p(b)) − √(p(a)p(b)) K̂(a,b))² − C, with C not depending on K̂ (B.3). Each term is minimized separately, at K̂ = K⁺.
- NT-Logistic. With one negative per positive drawn from p⊗p and equal priors on the two classes, the logistic loss is a binary cross-entropy between p⁺ and p⊗p. Pointwise its minimizer is the log-odds log(p⁺/(p⊗p)) = log K⁺ (B.2), so the exponentiated score is K⁺.
Both derivations are pointwise and exact. They could have failed: a loss whose optimum depended on how the negatives are drawn, in a way that did not cancel, would have another target. Here the negatives are drawn from the product of the marginals, and the target is the ratio.
What this does not say
- That a network reaches K⁺. The minimum is over unconstrained functions. A finite-capacity encoder, or a low-rank inner-product parametrization, reaches at best a constrained optimum, and for the spectral loss that is the rank-k one, which is HaoChen et al.’s (LIT-249).
- Anything about InfoNCE/NT-Xent. Its minimizer is taken from van den Oord et al., not derived, and is fixed only up to a factor constant on communicating classes (THEORY-001).
- Anything for continuous views. The derivation is on a finite space; in the continuous case the ratio needs P⁺ ≪ P⊗P, and the spectral loss needs K⁺ ∈ L²(p⊗p), which is finite χ² (CLAIM-370).
- That the two losses give the same d-dimensional features. The spectral loss fits K⁺ and the logistic loss fits log K⁺, and truncation to rank d does not commute with the logarithm.