nucleation

LIT-250: Duality of Bures and Shape Distances with Implications for Comparing Neural Representations

3 min read

Properties6
StatusActive — read 2026-10-14 (NOTE-753) in full; worth reading as the clean finite-dimensional statement that, for centred linear kernels, the Procrustes distance equals the Bures distance and normalized Bures similarity is the cosine of the Riemannian shape distance, so the kernel is a complete invariant of a representation up to translation and orthogonal maps, and as the proof that CKA and NBS are related only within a rank-dependent envelope. Read it knowing that it covers linear kernels only, assumes the orthogonal group rather than arguing for it, and treats the infinite limits by a law-of-large-numbers sketch and a citation; it does not reach an RKHS or GNS statement.
Version3
Tagsrepresentation-learning, neuroscience, mathematics, anthology-candidate
SupportsCLAIM-290 — A positive-definite kernel's minimal realizations are unique up to a unitary, and two finite representation matrices with the same centred Gram matrix differ by a translation and an orthogonal map
Cited as “paper” byNOTE-219 — Duality of Bures and Shape Distances, NOTE-753 — Harvey, Larsen & Williams — Duality of Bures and Shape Distances
Cited as “source” byTHEORY-002 — Convergence of representations, in the Platonic hypothesis's sense, is convergence of kernels, which fixes representations only up to the symmetry group of what is observed; whether closeness in distribution implies closeness of representation depends on the distance used, THEORY-004 — A representation is determined by its centred kernel up to a translation and an orthogonal transformation, and the rotation-aligned distance between two representations equals the Bures distance between their kernels, THEORY-233 — CKA and normalized Bures similarity are the Euclidean and the Bures comparison of normalized kernels, and each bounds the other only within a factor set by the kernels' ranks, so agreement in one does not carry to the other with a dimension-free constant, THEORY-301 — The Procrustes distance between regularization-normalized representations and their average linear-decoding distance bound each other only through the participation ratio of the kernel difference: a small Procrustes distance forces a small decoding distance, but not conversely, THEORY-723 — For the final embedding and unembedding of a softmax model, the output distribution fixes the representations only up to an invertible linear map, and whether closeness in distribution bounds representational dissimilarity depends on the distance: not for KL in the saturation regime, but yes for the log-likelihood variance distance

Graph View

Sarah E. Harvey, Brett W. Larsen, Alex H. Williams (2023), Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models (UniReps 2023), PMLR 243:11–26 (2024); first appeared as arXiv preprint — ARXIV-2311.11436

Key takeaways

From the reading, NOTE-753.

  • The duality (Thm 1). For centred linear kernels K_X = CXXᵀC, B(K_X, K_Y) = P(X, Y) and NBS(K_X, K_Y) = cos θ*(X, Y).
    • B is the Bures distance and P the Procrustes distance min_Q ‖CX − CYQ‖_F.
    • Both rest on Lemma 2, F(K_X, K_Y) = Tr[(K_X^½K_YK_X^½)^½] = ‖XᵀCY‖_*, proved by an eigenvalue argument (App. A.2).
  • Unequal widths. Written as ‖Σ_XY‖_* over traces, shape distances are defined for N_x ≠ N_y, and they stay metrics because B is one.
  • Uhlmann (eq. 25). The fidelity is the largest overlap |Tr XᵀY| over all representations X, Y consistent with the two kernels. Such representations differ by orthogonal maps, a fact the paper states without proof.
  • NBS is a fidelity of trace-normalized kernels. NBS(K_X, K_Y) = F(K_X/Tr K_X, K_Y/Tr K_Y). App. A.3 treats K/Tr K as a unit-trace PSD matrix, a density matrix in all but name.
  • CKA is a different geometry. CKA is the Euclidean cosine of the kernels, and NBS the Bures one. They are tied only by CKA/√(r_X r_Y) ≤ NBS² ≤ min(r_X, r_Y)·CKA (eq. 24) and by eqs. 27–28 for the square-root kernels. On random matrices they differ two- to three-fold.
  • Limits (§4). As stimuli M → ∞ or neurons N → ∞ the normalized distances converge, by a law-of-large-numbers sketch. The joint limit, a Bures distance between covariance operators, is asserted with citations. The paper never mentions RKHS or GNS.

Standing in the record

Filed on 2026-09-26 while pursuing, at the owner’s request, the connection GNS, unitary equivalence and the convergence of representations (see the curation entry of that date). Read in full on 2026-10-14 in NOTE-753.

It is held as the finite-dimensional, linear statement that the kernel is the invariant and a representation one realization of it. Distance between kernels in the Bures geometry is distance between representations after optimal orthogonal alignment. It uses quantum-information language (fidelity, Uhlmann’s theorem) to say so. THEORY-004 rests on it, and NOTE-753 finds THEORY-004 sound with three corrections of detail:

  • the NBS–shape-cosine equality is Theorem 1, not Lemma 2;
  • a centred kernel fixes the representation up to translation as well as an orthogonal map;
  • the “equal kernels iff zero Procrustes distance” statement is made in §2.2 for RDMs.

THEORY-002 cites it for the kernel-side metric, which is accurate for linear kernels. Its CKA results bear on THEORY-002’s promote_when, which says a CKA result does not count by itself, since the two bound each other only within a rank-dependent factor (§5, eq. 24). That envelope is filed as THEORY-233. The GNS reading the strand wants comes from LIT-259; this paper supplies none of it.

Tagged anthology-candidate (ADR-005): the seed judged it chiefly about machine-learning practice. On reading, it is mathematics of similarity measures and carries no instruction for practice. It is kept here by the owner’s decision of 2026-09-26 that new work stays in nucleation until a transfer is judged appropriate (ADR-010).

Backlinks