nucleation

LIT-253: When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective

3 min read

Properties5
StatusActive — read 2026-10-14 (NOTE-738) in full; worth reading as the exact statement that softmax models are identified only up to the paired map f = Af′, g₀ = A⁻ᵀg′₀, and as the proof that this identification is not stable under KL divergence but is stable under a log-likelihood variance distance (d_LLV ≤ ε ⇒ d_f,g ≤ 2Mε). Read it knowing that the KL counterexample needs the unembedding norm to grow without bound, that how far apart its embeddings are is shown numerically rather than proved, that everything concerns the final embedding and unembedding only, and that the experiments use 2- to 5-dimensional representations.
Version2
Tagsrepresentation-learning, learning-theory, probabilistic-modeling, anthology-candidate
Cited as “paper” byNOTE-221 — Closeness in Distribution vs Representational Similarity, NOTE-738 — Nielsen et al. — When Does Closeness in Distribution Imply Representational Similarity?
Cited as “source” byTHEORY-002 — Convergence of representations, in the Platonic hypothesis's sense, is convergence of kernels, which fixes representations only up to the symmetry group of what is observed; whether closeness in distribution implies closeness of representation depends on the distance used, THEORY-018 — Softmax training identifies a language model's final-layer representation only up to an invertible linear map, so no inner product on it is intrinsic, and Park, Choe and Veitch's causal inner product is fixed by a stipulation, THEORY-723 — For the final embedding and unembedding of a softmax model, the output distribution fixes the representations only up to an invertible linear map, and whether closeness in distribution bounds representational dissimilarity depends on the distance: not for KL in the saturation regime, but yes for the log-likelihood variance distance

Graph View

Beatrix M. G. Nielsen, Emanuele Marconato, Andrea Dittadi, Luigi Gresele (2025), NeurIPS 2025 (39th Conference on Neural Information Processing Systems), per the arXiv v2 PDF footer; first appeared as arXiv preprint — ARXIV-2506.03784

Key takeaways

From the reading, NOTE-738.

  • Exact identification (Thm 2.2, a prior result restated). For p(y|x) ∝ exp(f(x)ᵀg(y)) under a diversity condition, equal distributions give f(x) = Af′(x) and g₀(y) = A⁻ᵀg′₀(y), with A invertible. The embedding is fixed up to GL(M), and the unembedding up to the paired A⁻ᵀ and a translation.
  • Not stable under KL (Thm 3.1, Cor. 3.2). Hold the embeddings fixed and let the unembedding norm ρ grow. Both softmaxes saturate on the same top label, so the KL divergence between the models goes to zero, and both models’ loss on deterministic labels goes to zero.
    • The construction permutes the other labels, so the embeddings are not related by any continuous map, let alone a linear one.
    • How far apart they are is shown numerically: mean canonical correlation ≈ 0.001 while d_KL falls from 0.89 to 0.0001 (Table 1).
  • Stable under another distance (Thm 4.7). The log-likelihood variance distance compares standardized log-probability differences with equal weight, not weighted by probability. d_LLV ≤ ε implies d_f,g ≤ 2Mε. The bound is non-vacuous only for ε < 1/(2M) and needs all labels but one.
  • The representational measure. d_f,g is built on the log-odds coordinates Lᵀf(x), which are identified. It is constant on each equivalence class and vanishes up to rotation and per-coordinate scaling. Its core, m_SVD, equals the normalized Bures similarity of LIT-250 on those standardized coordinates. That identity is the reader’s, not the paper’s.
  • Experiments are small. They use 2- to 5-dimensional representations.
    • One CIFAR-10 pair shows permuted label clusters at similar test loss. Its KL is about 1.1 nats, so it is not small.
    • On a synthetic 2-D task, wider networks give closer distributions and more similar representations.
  • Scope. Final embedding and unembedding only. The paper says hidden layers need identifiability results that do not exist yet.

Standing in the record

Filed on 2026-09-26 while pursuing, at the owner’s request, the connection GNS, unitary equivalence and the convergence of representations (see the curation entry of that date). Read in full on 2026-10-14 in NOTE-738.

It is held as the exact statement, for language-model-style softmax outputs, that the distribution determines the representation only up to a symmetry group, here GL(M) with paired A and A⁻ᵀ, not the orthogonal group. It is also held as the proof that how stable this determination is depends on the divergence. THEORY-002 cites it for the symmetry group and for the KL negative result. NOTE-738 finds that THEORY-002 leaves out the positive result (Thm 4.7), and that the paper’s scope, the output layer only, is narrower than THEORY-002’s use of it for the Platonic hypothesis. THEORY-018 cites it as starting from the same symmetry as Park, Choe and Veitch (LIT-304), which is accurate. It bears on QUESTION-027: Cor. 3.2 is a case where the same decisions are reached with representations that are not linearly related.

Tagged anthology-candidate (ADR-005): the seed judged it chiefly about machine-learning practice. On reading, it is representation-learning theory and carries no instruction for practice. It is kept here by the owner’s decision of 2026-09-26 that new work stays in nucleation until a transfer is judged appropriate (ADR-010).

Backlinks