nucleation

LIT-227: Contrastive Learning Is Spectral Clustering On Similarity Graph

2 min read

Properties5
StatusActive — read 2026-10-14 (NOTE-846) in full; worth reading as an exact identity: SimCLR's InfoNCE loss, over all n objects with a Gaussian kernel on the unit sphere and a symmetric augmentation matrix π, equals tr(Zᵀ L(π) Z) + log R(Z), a Laplacian quadratic form plus the log of InfoNCE's normalizers. Read it knowing that its "spectral clustering" admits any regularizer (Def. 2.7), so no minimizer is characterized and no eigenvector or density-ratio statement follows. The CLIP extension is a two-sentence proof, the batch-size reading is a remark, and the kernel gains are 1 to 4 points on three small datasets.
Version2
Tagsrepresentation-learning, probabilistic-modeling, anthology-candidate
Cited as “paper” byNOTE-209 — Tan et al. 2023 — contrastive learning is spectral clustering, NOTE-846 — Tan et al. — contrastive learning is spectral clustering on a similarity graph
Cited as “source” byTHEORY-305 — SimCLR's InfoNCE loss, over all n objects with a Gaussian kernel on the unit sphere, equals a graph-Laplacian quadratic form on the augmentation matrix plus the log of InfoNCE's normalizers, which is spectral clustering only in a sense that admits any regularizer and does not make the minimizer the Laplacian's eigenvectors

Graph View

Zhiquan Tan, Yifan Zhang, Jingqin Yang, Yang Yuan (2023), International Conference on Learning Representations (ICLR 2024) — ARXIV-2303.15103

Key takeaways

From the reading, NOTE-846.

  • An exact identity. InfoNCE over all n objects equals the cross-entropy between two Markov random fields on directed subgraphs with one outgoing edge per node, one built from π and one from the kernel Gram matrix of the embeddings (Thm 3.1, Step 1).
  • Its spectral form. With a Gaussian kernel on the unit sphere and symmetric π, that objective is tr(Zᵀ L(π) Z) + log R(Z), where log R(Z) is the sum of InfoNCE’s log normalizers (Thm 3.1, Step 2; the 1/τ on the trace term is dropped in the statement).
  • A wide definition. “Spectral clustering” is any min_Z tr(Zᵀ L(W) Z) + E(Z) with any regularizer E (Def. 2.7). No minimizer is characterized, so the theorem does not say that InfoNCE’s embedding is, or spans, the Laplacian’s eigenvectors.
  • CLIP. The same identity on a directed bipartite pair graph, called “generalized spectral clustering”, with a two-sentence proof and a sampling scheme CLIP does not use (Thm 4.2).
  • Kernels. The maximum-entropy argument (Thm 5.1) gives the softmax form for any similarity. Kernel-InfoNCE mixtures gain 1 to 4 points on CIFAR-10/100 and TinyImageNet; single Laplacian and γ = 0.5 kernels lose to SimCLR on CIFAR-100 at 400 epochs.

Standing in the record

Filed on 2026-09-26 at the owner’s request, from a list they grouped under the heading Representation learning as a spectral approximation. Read in full on 2026-10-14 in NOTE-846, which supersedes the seeded skim NOTE-209.

The reading filed THEORY-305: the identity holds at the strength shown, and it is spectral clustering only in a sense that admits any regularizer. THEORY-001 cites this paper in support of a shared density-ratio optimum, which the paper never names; the reading reports that citation as unsupported. The reconciliation with Balestriero and LeCun (LIT-228) that the seed asked for is not in either paper.

Tagged anthology-candidate (ADR-005): Kernel-InfoNCE is an instruction for machine-learning practice, and that part belongs in the anthology. It is kept here by the owner’s decision of 2026-09-26 that new work stays in nucleation until a transfer is judged appropriate (ADR-010).

No DOI found; the published venue is ICLR 2024.

Backlinks