nucleation

LIT-341: Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

1 min read

Properties5
StatusActive — read in full 2026-09-29 (NOTE-287); worth reading as the paper that named grokking and measured its data and optimisation dependence. It is a testbed paper with ablations and no mechanism, so it supplies the phenomenon row 15 cites, not the resolvability or singular-learning-theory reading.
Version2
Tagslearning-theory, representation-learning
Cited as “paper” byNOTE-287 — Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
Cited as “source” byTHEORY-022 — The reverse-engineered grokking network computes modular addition by multiplying characters of ℤ/p at a few frequencies, the same real two-dimensional irreducibles later found as circles in language models; in both the group comes from the task, not the network, THEORY-181 — In a model that decodes the sum of two learned embeddings, at zero training loss with an injective decoder, a training set fixes the generalising representation of addition when its same-answer pairs leave only translation and scale free; for non-modular addition with p = 10 and one-dimensional embeddings, generalisation sets in near the training fraction, about 0.4, at which that becomes likely, THEORY-782 — In networks that learn addition from summed embeddings, the critical training fraction of grokking is the fraction at which the training set's same-answer pairs come to fix the generalising representation up to translation and scale, at every group size, as Liu, Kitouni, Nolte, Michaud, Tegmark and Williams propose

Graph View

Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, Vedant Misra (2022), arXiv preprint (earlier ICLR 2021 MATH-AI workshop version) — ARXIV-2201.02177

Standing in the record

Registered on 2026-09-29 at the owner’s request, because the owner’s prior-art novelty map (a private working document on concepts as projectors, the linear representation hypothesis and their neighbours) cites it: row 15 must-cite: grokking as a concept-resolvability phenomenon. It is seeded from metadata and published descriptions and has not been read here.

It is also held in the Anthology of the SOTA as ANTH-LIT-538. Holding it here too follows ADR-013: this record holds it for the owner’s prior-art map, the anthology for ML practice.

Identification notes: Held in anthology (ANTH-LIT-538).

It was filed Deferred, unread. NOTE-287 is the close reading of 2026-09-29, and it placed the work: Active — worth reading as the paper that named grokking and measured its data and optimisation dependence. It is a testbed paper with ablations and no mechanism, so it supplies the phenomenon row 15 cites, not the resolvability or singular-learning-theory reading.

Backlinks