The claim
Let X and Y be n × p representations of the same n examples, and let s be an index with s(X, Z) = s(XA, Z) for every invertible p × p matrix A. If col(X) = col(Y), then s(X, Z) = s(Y, Z). So s(·, Z) is a function of the column space alone.
Proof
Let W = col(X) = col(Y), of dimension r ≤ p, and let B be an n × r matrix whose columns are a basis of W. Then X = BC and Y = BD for r × p matrices C and D. Each has rank r, because rank(X) = r and rank(BC) ≤ rank(C) ≤ r. Extend the r independent rows of C by p − r further rows E to an invertible p × p matrix C′ = [C; E], and likewise D′ = [D; F]. Put A = C′⁻¹D′, which is invertible. Then C′A = D′, whose first r rows give CA = D. So XA = BCA = BD = Y, and s(Y, Z) = s(XA, Z) = s(X, Z).
Conversely, col(XA) = col(X) for every invertible A, so the orbit of X under GL(p) is exactly the set of n × p matrices with column space W.
Two cases. If r = n, then W = ℝⁿ, which is Kornblith et al.’s Theorem 1 (THEORY-236). If the columns are centred, every column is orthogonal to the all-ones vector 1, so col(X) ⊆ 1^⊥. At rank n − 1 it equals 1^⊥, for every such X. So an invariant index takes one value on all centred representations of rank n − 1, given Z.
What it does not say
- Anything below the maximal rank. There an invariant index can separate representations, but only by their column spaces.
- Which group a comparison should use. That is THEORY-236’s caveat, and it is empirical.