Source
Johnson, El Hanchi & Maddison (2023), LIT-258, Table 1 and App. B.1–B.3. Supporting: HaoChen et al. (2021), LIT-249, Lemma 3.2 (the factorization only; the step to K⁺ is Johnson et al. App. B.3); Poole et al. (2019), LIT-246, §2.3 (the InfoNCE optimum in form, cited there, not proved); Tosh, Krishnamurthy & Hsu (2021), LIT-251, §2.3. Levy & Goldberg’s shifted-PMI factorization of word2vec (ANTH-LIT-612, Eq. 7) is the log-domain case with k negatives, and CPC (ANTH-LIT-589, Eq. 5) states the critic’s proportionality to the ratio.
What was actually shown
On a finite augmentation space, the logistic loss with one negative per positive and the spectral contrastive loss are minimized over all functions of the two views by exactly the ratio K⁺ = p⁺/(p⊗p), and these minimizers are derived (LIT-258, App. B.2–B.3). HaoChen et al. show that the spectral loss is a matrix-factorization loss (LIT-249, Lemma 3.2) but never name K⁺; their own minimizer is the rank-k one. InfoNCE/NT-Xent is handled differently. Johnson et al. take the form of its minimizer, f*(c,a) = (p(a|c)/p(a))·b(c), from van den Oord et al., with the negatives drawn i.i.d. from the marginal rather than from other positive pairs as in practice. They then restrict to symmetric functions, which forces b to be constant on each communicating class of the positive-pair chain, so the minimizer is K⁺ times such a factor (LIT-258, Table 1, App. B.1). They note that this optimum “is not unique” for the cross-entropy loss. Poole et al. state the optimal InfoNCE critic as log p(y|x) + c(y), the log ratio plus a function of y, for general densities, on Ma & Collins’s authority rather than by derivation (LIT-246, §2.3). In the log domain, skip-gram with k negatives is minimized by PMI − log k (ANTH-LIT-612). K⁺ is the Radon–Nikodym derivative of the positive-pair distribution with respect to the product of its marginals, so each of these objectives estimates the same change of measure; LIT-251 §2.4 names it as such: “g⋆ provides the change-of-measure from the marginal distribution of Z to the conditional distribution of Z given X = x”.
The derivation could have come out otherwise: a loss whose optimum depended on the negative-sampling scheme or the batch size in a way that did not cancel would have a different target. It does not, for these three, in the unconstrained population limit.
What this does not say
- That a finite-capacity encoder reaches K⁺. Every result here is over unconstrained functions, and for InfoNCE over symmetric functions only.
- Anything about InfoNCE with in-batch negatives drawn from other positive pairs. Johnson et al.’s NT-Xent minimizer assumes negatives i.i.d. from the marginal.
- That low-rank truncations under the different losses coincide. InfoNCE and skip-gram fit the log of the ratio, while the spectral loss fits the ratio itself; exp and log do not commute with rank-d truncation, so the d-dimensional representations can differ. Tosh et al. make the same contrast for their two losses: the logistic loss is minimized by log g*, the direct loss by g* (LIT-251, §5).
- Anything about continuous augmentation spaces without an integrability condition (see the operator claim that extends this one).
- That the features are good. That is a separate question (the eigenbasis claim, and Tschannen et al. on why the MI story is not the explanation).
Connections
- Tan et al. (LIT-227, THEORY-305) show a different statement: InfoNCE equals a Laplacian form on the positive-pair graph plus log normalizers, at every embedding. They name no optimum and no density ratio.
Note of 2026-10-19: the confirmed half filed apart
The spectral and NT-Logistic minimizers, which NOTE-751 confirmed, are now THEORY-719, Active. What keeps this THEORY Proposed is only the third loss: the InfoNCE/NT-Xent minimizer, taken from van den Oord et al. and derived over symmetric functions only. The promote_when says what would settle it.