Source
Park, Choe and Veitch (2024), LIT-606 — ARXIV-2311.03658, §3.2 and App D.2. The account is THEORY-090.
The claim
Before measuring the similarity of two concept directions, projecting one out of another, or orthogonalizing a set, change the inner product:
⟨u, v⟩_C = uᵀ Cov(γ)⁻¹ v
where Cov(γ) is the covariance of the model’s unembedding rows over the vocabulary. Equivalently, map every direction by Cov(γ)^(−1/2) and use ordinary cosine and projection afterwards. The steering direction matching an unembedding direction γ̄ is Cov(γ)⁻¹γ̄.
Why: training leaves the representation defined only up to an invertible linear map, so a Euclidean cosine reports a coordinate system. Under this product, concepts that can vary independently are orthogonal, and each concept’s probe and steering directions coincide.
Evidence
The instruction and its derivation come from LIT-606. Park, Choe and Veitch show that the softmax is unchanged when unembeddings are multiplied by any invertible A and embeddings by A⁻ᵀ, so training fixes no inner product, and prove (Thm 3.4) that if a word drawn uniformly from the vocabulary has uncorrelated values on separable concepts, Cov(γ)⁻¹ is a causal inner product: one under which separable concepts are orthogonal and a concept’s probe and steering directions coincide. Their empirical support is the following.
- LLaMA-2 7B, 27 concepts (Fig 3, Fig 8). Whitened, separable concepts are near-orthogonal with interpretable blocks. Euclidean is “somewhat” orthogonal as well. The visible gains are frequent⇒infrequent, whose spurious Euclidean overlaps disappear, and language pairs sharing French, whose real overlap appears.
- Gemma-2B (Fig 9). Euclidean does not capture semantics and whitening does. The authors attribute this to tied embeddings.
- All of it is heatmaps read by eye, with no statistic.
Why Proposed, and why contested
- One source, qualitative, output space only. Intermediate layers, where directions are usually taken, are untested in the source.
- LIT-526 tested the obvious use and found nothing: for cross-lingual concept transport, whitening adds nothing over spectral regularization across 17 models and four language pairs. It also finds contextual concept directions concentrated in the low-variance tail of this very covariance, where whitening amplifies them most (THEORY-063). Whether that helps or hurts is not known.
- The covariance is one choice among the causal inner products the theorem allows.
Conditions
- Applies to directions in the unembedding or final-layer space. Anywhere else it is an extrapolation.
- Cov(γ) over the whole vocabulary, not over word frequencies in text: the assumption behind it is about words drawn uniformly from the vocabulary.
- Cheap: one d×d covariance and its inverse, once per model.
Known implementations
github.com/KihoPark/linear_rep_geometry(the authors’ code)