Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
Concept geometry — how concepts are laid out in a model’s representation space — linear directions and the regions, lattices and spectra built from them, superposition and the dictionaries that pull it apart, reading concepts out with probes and writing them in with steering vectors, and whether different models arrive at the same geometry.
6 of 129 THEORY documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| THEORY-034 v2 | Concepts sit in embedding space as intersections of half-spaces, so inclusion, intersection and union are geometric operations | Xiong (2026), LIT-460 — a linear attribute direction plus a separating threshold is a half-space, a concept is the intersection of its attributes’ half-spaces, and the resulting regions form a complete lattice. Conceptual refinement is geometric meet, generalisation is join. Evidence is WordNet, which is a hand-built hierarchy and so the friendliest test there is. | Proposed |
| THEORY-036 | Representations converge across architectures, objectives and modalities, and the endpoint is a model of what generated the data | Huh et al. (2024), LIT-458 — vision and language models measure distance between datapoints increasingly alike as they scale, across architectures and objectives. The conjecture is that they are converging on a representation of the joint distribution that generated the observations. The convergence is measured; the endpoint is proved only for a world of bijective observations. | Proposed |
| THEORY-063 | Contextual concept directions sit in the low-variance tail of the unembedding spectrum while vocabulary contrasts sit at the top | Acharya, Rimal and Dhakal (2026), LIT-526 — concept directions read out of the residual stream anti-concentrate in the low-eigenvalue directions of the unembedding second moment, in 17 of 17 models and by three independent extraction methods. Static unembedding-row contrasts do the opposite. So the same model carries semantic content at both ends of one spectrum, depending on whether the representation is contextual. | Proposed |
| THEORY-089 v3 | Word vectors trained on co-occurrence behave linearly because they encode log co-occurrence statistics: differences carry log ratios, which gives analogies, and sums carry products of context distributions, which gives composition | Pennington et al. (2014), LIT-602, and Mikolov et al. (2013), LIT-603 — what distinguishes words is the ratio of how often they co-occur with probe words (ice and steam with solid versus gas). A model whose dot products fit log co-occurrence turns ratios into vector differences, so relations become offsets. The same log-linearity makes a sum of vectors act like a product of context distributions, an AND of their contexts. Derived from the objectives and illustrated. The mechanism is not measured. | Proposed |
| THEORY-090 | A concept’s shared word-pair direction, its linear probe and its steering vector are one object, related by an inner product under which causally separable concepts are orthogonal | Park, Choe and Veitch (2024), LIT-606 — the output-space direction that counterfactual word pairs share is provably a logit-linear probe, and the input-space direction is provably a steering vector that leaves separable concepts alone. Training fixes neither space’s inner product, so cosine similarity is arbitrary. The inner product that makes separable concepts orthogonal maps one direction onto the other, and one member of that family is the inverse unembedding covariance. Proved under stated assumptions and shown on LLaMA-2 7B. | Proposed |
| THEORY-094 | Word vectors approximate PMI in low dimensions, and relations appear as clean directions despite a noisy fit, because text behaves as if generated by a slowly drifting discourse over isotropic word vectors | Arora et al. (2016), LIT-613 — if each word is emitted with probability ∝ exp(⟨c, v_w⟩) as a discourse vector c drifts, and word vectors are isotropic, the partition function is nearly constant and PMI(w,w’) ≈ ⟨v_w, v_w’⟩/d holds in low dimensions. That carries THEORY-093 from its unconstrained optimum to practical sizes. Isotropy also makes solving for a relation’s offset a well-conditioned regression that averages away per-word error, which is the step THEORY-089 skipped. The assumptions are checked only loosely. | Proposed |