capability-thresholds

Oct 19, 2026

THEORY-040: A metric that composes or thresholds per-token error turns a smooth capability curve into a sharp one, with nothing happening in the model

Oct 01, 2026

LIT-769: Structural Decoupling: A Scaffold-Flow Theory of Generalization and Alignment

Sep 25, 2026

LIT-667: Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization

Sep 25, 2026

SOTA-402: To teach a model to reason over facts it holds, spend added data on facts derived from other facts rather than on more atomic facts

Sep 25, 2026

THEORY-103: A transformer stores a fact where it needed it, so two-hop composition generalizes only to facts it already saw as a second hop

Sep 24, 2026

LIT-617: Scaling Language Models: Methods, Analysis & Insights from Training Gopher

Sep 24, 2026

LIT-656: Intriguing Properties of Quantization at Scale

Sep 24, 2026

THEORY-100: Activation outliers are a product of pre-training optimization choices, not an emergent property of scale

Sep 22, 2026

LIT-537: Grokking as the Transition from Lazy to Rich Training Dynamics

Sep 22, 2026

LIT-538: Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

Sep 22, 2026

LIT-539: Explaining grokking through circuit efficiency

Sep 22, 2026

LIT-540: Omnigrok: Grokking Beyond Algorithmic Data

Sep 22, 2026

LIT-543: Loss Landscape Degeneracy and Stagewise Development in Transformers

Sep 22, 2026

LIT-544: Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition

Sep 22, 2026

THEORY-069: Grokking is a regime rather than a property of algorithmic data, and at least three knobs move it

Sep 22, 2026

THEORY-070: Grokking is the transition from lazy to rich training dynamics

Sep 22, 2026

THEORY-071: A memorising and a generalising circuit compete on logits per unit norm, and which one wins flips when the data makes memorising more expensive

Sep 22, 2026

THEORY-072: Training and test loss disagree as functions of weight norm, and shrinking the norm through the gap is what takes so long

Sep 22, 2026

THEORY-073: Training passes through stages marked by changes in loss-landscape degeneracy, and the loss curve does not show them

Sep 22, 2026

THEORY-074: A phase transition of the Bayesian posterior and a transition in the training trajectory are different events, and only the first is well defined

Sep 21, 2026

LIT-470: Emergent Abilities of Large Language Models

Sep 21, 2026

LIT-471: Are Emergent Abilities of Large Language Models a Mirage?

Sep 21, 2026

SOTA-279: Put worked reasoning steps in the few-shot exemplars when the task needs more than one step, and only once the model is large enough

Sep 10, 2026

SOTA-200: Check whether an emergent capability is a metric artefact before believing it

Aug 24, 2026

LIT-085: Progress measures for grokking via mechanistic interpretability

Aug 24, 2026

anthology-of-the-sota

Explorer

capability-thresholds

Graph View

THEORY-040: A metric that composes or thresholds per-token error turns a smooth capability curve into a sharp one, with nothing happening in the model

LIT-769: Structural Decoupling: A Scaffold-Flow Theory of Generalization and Alignment

LIT-667: Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization

SOTA-402: To teach a model to reason over facts it holds, spend added data on facts derived from other facts rather than on more atomic facts

THEORY-103: A transformer stores a fact where it needed it, so two-hop composition generalizes only to facts it already saw as a second hop

LIT-617: Scaling Language Models: Methods, Analysis & Insights from Training Gopher

LIT-656: Intriguing Properties of Quantization at Scale

THEORY-100: Activation outliers are a product of pre-training optimization choices, not an emergent property of scale

LIT-537: Grokking as the Transition from Lazy to Rich Training Dynamics

LIT-538: Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

LIT-539: Explaining grokking through circuit efficiency

LIT-540: Omnigrok: Grokking Beyond Algorithmic Data

LIT-543: Loss Landscape Degeneracy and Stagewise Development in Transformers

LIT-544: Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition

THEORY-069: Grokking is a regime rather than a property of algorithmic data, and at least three knobs move it

THEORY-070: Grokking is the transition from lazy to rich training dynamics

THEORY-071: A memorising and a generalising circuit compete on logits per unit norm, and which one wins flips when the data makes memorising more expensive

THEORY-072: Training and test loss disagree as functions of weight norm, and shrinking the norm through the gap is what takes so long

THEORY-073: Training passes through stages marked by changes in loss-landscape degeneracy, and the loss curve does not show them

THEORY-074: A phase transition of the Bayesian posterior and a transition in the training trajectory are different events, and only the first is well defined

LIT-470: Emergent Abilities of Large Language Models

LIT-471: Are Emergent Abilities of Large Language Models a Mirage?

SOTA-279: Put worked reasoning steps in the few-shot exemplars when the task needs more than one step, and only once the model is large enough

SOTA-200: Check whether an emergent capability is a metric artefact before believing it

LIT-085: Progress measures for grokking via mechanistic interpretability

SOTA-037: LM in-context learning emerges at scale