nucleation

LIT-257: What Representational Similarity Measures Imply about Decodable Information

2 min read

Properties5
StatusActive — read 2026-10-14 (NOTE-834) in full; worth reading as the decoding reading of representational similarity: an optimal regularized linear readout outputs K_X z, so linear CKA, mean squared CCA, ENSD and the GULP distance are task-averaged readout statistics, and Procrustes distance and decoding distance bound each other through a participation ratio. Read it knowing that every identification fixes the target covariance to the identity at finite M, and that its population appendix is an outline that does not reach that case.
Version2
Tagsrepresentation-learning, neuroscience, anthology-candidate
Cited as “paper” byNOTE-225 — What Similarity Measures Imply about Decodability, NOTE-834 — Harvey, Lipshutz & Williams — What Representational Similarity Measures Imply about Decodable Information
Cited as “source” byTHEORY-008 — On a finite set of stimuli, what a regularized linear readout can decode from a representation is a function of its normalized kernel, and CKA, CCA and GULP are averages of readout agreement, THEORY-301 — The Procrustes distance between regularization-normalized representations and their average linear-decoding distance bound each other only through the participation ratio of the kernel difference: a small Procrustes distance forces a small decoding distance, but not conversely

Graph View

Sarah E. Harvey, David Lipshutz, Alex H. Williams (2024), Proceedings of the II edition of the Workshop on Unifying Representations in Neural Models (UniReps 2024), as printed on the arXiv PDF; first appeared as arXiv preprint (proceedings volume and pages unverified) — ARXIV-2411.08197

Key takeaways

From the reading, NOTE-834.

  • One matrix. The readout maximizing zᵀXw/M − ½wᵀG(X)w outputs Xw* = K_X z with K_X = XG(X)⁻¹Xᵀ/M (eqs. 2, 7–8). The best and worst task alignments are the extreme eigenvalues of ½(K_X K_Y + K_Y K_X) (Prop. 1). The average is Tr(K_X K_z K_Y) (Prop. 2).
  • The family. With K_z = I (Cor. 1, Table 1, eqs. 17–21, re-derived):
    • G = bI gives linear CKA, a ratio of average similarities;
    • G = C_X gives mean squared CCA, with K_X the projection onto col(X);
    • G = C_X + λI gives the plug-in GULP distance, an average distance;
    • a scaled identity gives ENSD.
  • Invariance. K_X is invariant under orthogonal maps when G = aC_X + bI (eqs. 22–23). Only for CCA is it invariant under every invertible linear map.
  • Procrustes against decoding (Prop. 3, App. A.1). P⁴/R_Δ ≤ E‖Xw* − Yv*‖² ≤ (2(α + β)P² − P⁴)/R_Δ. So a small Procrustes distance forces a small decoding distance, but not conversely. The Procrustes–Bures equivalence is imported from LIT-250.
  • The population limit is a sketch (App. A.3). It needs b > 0, so it excludes CCA. It asserts concentration, and eq. 77 has a stray factor. It treats a trace-class task covariance, and at K_z = I its limit is zero.

Standing in the record

Filed on 2026-09-26 while pursuing, at the owner’s request, the connection GNS, unitary equivalence and the convergence of representations (see the curation entry of that date). Read in full on 2026-10-14 in NOTE-834, which supersedes the seeded skim NOTE-225.

It is the source of THEORY-008. The reading meets the first clause of THEORY-008’s promote_when, confirming Prop. 2 and Cor. 1. It fails the second, the population version in App. A.3, which this paper cannot supply. THEORY-301 was filed from Prop. 3.

Tagged anthology-candidate (ADR-005) at seeding. The reading found no instruction for ML practice. It is kept here by the owner’s decision of 2026-09-26 that new work stays in nucleation until a transfer is judged appropriate (ADR-010).

Backlinks