Key takeaways
- The gap it is filling is compositional, not existential. The Linear Representation Hypothesis establishes that a concept is a direction. What it does not give is inclusion, intersection and union — the set-theoretic relations that make conceptual knowledge a hierarchy rather than a bag of features.
- The construction is one step. A linear attribute direction plus a separating threshold is a half-space. A concept defined by several attributes is the intersection of their half-spaces. Order those regions by inclusion and you have a lattice, with a well-defined greatest lower bound (meet) and least upper bound (join) for any pair.
- Meet is intersection; join is the conic hull of the two concepts’ attribute directions — the smallest region covering both. Both are operations on embeddings, not on symbols.
- A canonical form exists when the attribute directions are linearly independent.
- Soft versions are defined, with fuzzy projection profiles, because real embeddings do not sit cleanly on one side of a threshold.
- Evidence is WordNet sub-hierarchies: LLM embeddings are reported to encode concept lattices and their logical structure, supporting generalisation (join) and refinement (meet).
Standing in the anthology
It formalises a bridge the record now holds both sides of. When this was
filed the record had nothing on concept geometry, nothing on the Linear
Representation Hypothesis this paper unifies with Formal Concept Analysis,
and nothing on steering. This entry said so, from under
representation-and-encoding, a topic about encoding the input. That gap is
why concept-geometry exists (ADR-060), and this paper now takes it as
its primary topic.
THEORY-034 records the account. No practice is filed from it, and that is deliberate: the paper establishes a structure and demonstrates it on a curated ontology, and nothing here says what to do differently when training or serving a model. The obvious candidate — steer by constructing meets and joins of attribute directions — is not evaluated as an intervention, only as a geometry.
The antecedent is now filed. This paper’s premise is Park, Choe and
Veitch’s formalization of the Linear Representation Hypothesis,
LIT-606, which this paper extends. Its account is THEORY-090, and
THEORY-034 extends that. The antecedent’s evidence is 27 hand-picked concepts
on two models, and its one independent test (LIT-526) found its inner product
no help for cross-lingual transport. This paper inherits both limits: the
foundation is proved under assumptions and has been tested only on concepts
somebody chose, and so has the lattice.
Other concept-geometry work filed since: LIT-526 on where concept directions sit in the unembedding spectrum, the word-vector papers where the offsets were first seen (LIT-604, LIT-603, LIT-602), and sparse-autoencoder and probing work (LIT-571, LIT-568).
What it does not establish
WordNet is a hand-built ontology with exactly the clean hierarchical structure the hypothesis predicts, which makes it the friendliest possible test and not a severe one. Whether the lattice structure holds for concepts nobody curated — the ones a model learns and no taxonomy names — is the question, and it is not asked here.
The canonical form requires linearly independent attribute directions, which is a condition on the concepts chosen rather than a property of the embedding space.