concept-geometry

Oct 01, 2026

LIT-759: From monoliths to modules: Decomposing transducers for efficient world modelling

Sep 25, 2026

LIT-696: Rarely categorical, highly separable representations along the cortical hierarchy

concept-geometry

Sep 25, 2026

LIT-698: Topos and Stacks of Deep Neural Networks

Sep 23, 2026

LIT-568: Are Sparse Autoencoders Useful? A Case Study in Sparse Probing

Sep 23, 2026

LIT-571: Scaling and evaluating sparse autoencoders

Sep 23, 2026

LIT-602: GloVe: Global Vectors for Word Representation

Sep 23, 2026

LIT-603: Distributed Representations of Words and Phrases and their Compositionality

Sep 23, 2026

LIT-604: Efficient Estimation of Word Representations in Vector Space

Sep 23, 2026

LIT-606: The Linear Representation Hypothesis and the Geometry of Large Language Models

Sep 23, 2026

LIT-613: A Latent Variable Model Approach to PMI-based Word Embeddings

Sep 23, 2026

SOTA-342: Report a sparse autoencoder's fidelity as downstream loss in compute-equivalent terms, not as fraction of loss recovered against zero ablation

Sep 23, 2026

SOTA-344: Train sparse autoencoders with a TopK activation instead of an L1 penalty, and prevent dead latents with transposed-decoder initialization and an auxiliary loss

Sep 23, 2026

SOTA-345: Probe language-model activations with logistic regression on the raw activations, and credit sparse-autoencoder probes only against that baseline under validation-based selection

Sep 23, 2026

SOTA-378: Compare, project and orthogonalize a language model's concept directions after whitening by the unembedding covariance, not by raw cosine

Sep 23, 2026

THEORY-089: Word vectors trained on co-occurrence behave linearly because they encode log co-occurrence statistics: differences carry log ratios, which gives analogies, and sums carry products of context distributions, which gives composition

Sep 23, 2026

THEORY-090: A concept's shared word-pair direction, its linear probe and its steering vector are one object, related by an inner product under which causally separable concepts are orthogonal

Sep 23, 2026

THEORY-094: Word vectors approximate PMI in low dimensions, and relations appear as clean directions despite a noisy fit, because text behaves as if generated by a slowly drifting discourse over isotropic word vectors

Sep 22, 2026

LIT-526: Concepts Whisper: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations

Sep 22, 2026

LIT-544: Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition

Sep 22, 2026

THEORY-063: Contextual concept directions sit in the low-variance tail of the unembedding spectrum while vocabulary contrasts sit at the top

Sep 20, 2026

LIT-458: The Platonic Representation Hypothesis

Sep 20, 2026

LIT-460: The Lattice Representation Hypothesis of Large Language Models

Sep 20, 2026

THEORY-034: Concepts sit in embedding space as intersections of half-spaces, so inclusion, intersection and union are geometric operations

Sep 20, 2026

anthology-of-the-sota

Explorer

concept-geometry

Graph View

LIT-759: From monoliths to modules: Decomposing transducers for efficient world modelling

LIT-696: Rarely categorical, highly separable representations along the cortical hierarchy

LIT-698: Topos and Stacks of Deep Neural Networks

LIT-568: Are Sparse Autoencoders Useful? A Case Study in Sparse Probing

LIT-571: Scaling and evaluating sparse autoencoders

LIT-602: GloVe: Global Vectors for Word Representation

LIT-603: Distributed Representations of Words and Phrases and their Compositionality

LIT-604: Efficient Estimation of Word Representations in Vector Space

LIT-606: The Linear Representation Hypothesis and the Geometry of Large Language Models

LIT-613: A Latent Variable Model Approach to PMI-based Word Embeddings

SOTA-342: Report a sparse autoencoder's fidelity as downstream loss in compute-equivalent terms, not as fraction of loss recovered against zero ablation

SOTA-344: Train sparse autoencoders with a TopK activation instead of an L1 penalty, and prevent dead latents with transposed-decoder initialization and an auxiliary loss

SOTA-345: Probe language-model activations with logistic regression on the raw activations, and credit sparse-autoencoder probes only against that baseline under validation-based selection

SOTA-378: Compare, project and orthogonalize a language model's concept directions after whitening by the unembedding covariance, not by raw cosine

THEORY-089: Word vectors trained on co-occurrence behave linearly because they encode log co-occurrence statistics: differences carry log ratios, which gives analogies, and sums carry products of context distributions, which gives composition

THEORY-090: A concept's shared word-pair direction, its linear probe and its steering vector are one object, related by an inner product under which causally separable concepts are orthogonal

THEORY-094: Word vectors approximate PMI in low dimensions, and relations appear as clean directions despite a noisy fit, because text behaves as if generated by a slowly drifting discourse over isotropic word vectors

LIT-526: Concepts Whisper: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations

LIT-544: Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition

THEORY-063: Contextual concept directions sit in the low-variance tail of the unembedding spectrum while vocabulary contrasts sit at the top

LIT-458: The Platonic Representation Hypothesis

LIT-460: The Lattice Representation Hypothesis of Large Language Models

THEORY-034: Concepts sit in embedding space as intersections of half-spaces, so inclusion, intersection and union are geometric operations

THEORY-036: Representations converge across architectures, objectives and modalities, and the endpoint is a model of what generated the data