nucleation

LIT-345: Progress measures for grokking via mechanistic interpretability

1 min read

Properties5
StatusActive — read in full 2026-09-29 (NOTE-283); worth reading as the fullest mechanistic account of a grokking network. Its learned structure is exactly the character basis of ℤ/113 (five real 2-dimensional irreps), which makes it the nearest existing empirical instance of the map's row-7 "predicate harmonics = characters" prediction, in a setting where the group was known in advance.
Version2
Tagsrepresentation-learning, learning-theory
Cited as “paper” byNOTE-283 — Progress measures for grokking via mechanistic interpretability
Cited as “source” byTHEORY-022 — The reverse-engineered grokking network computes modular addition by multiplying characters of ℤ/p at a few frequencies, the same real two-dimensional irreducibles later found as circles in language models; in both the group comes from the task, not the network, THEORY-039 — The later phases reported in neural-network training differ in kind: after the transition a network may simplify its weights, acquire item-specific information, lose generalisation, confine its tangent kernel or saturate its units, and none of these is a measured compression of I(X;T)

Graph View

Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, Jacob Steinhardt (2023), ICLR 2023 — ARXIV-2301.05217

Standing in the record

Registered on 2026-09-29 at the owner’s request, because the owner’s prior-art novelty map (a private working document on concepts as projectors, the linear representation hypothesis and their neighbours) cites it: row 15: grokking mechanism (also relevant to row 7: learned Fourier/character structure). It is seeded from metadata and published descriptions and has not been read here.

It is also held in the Anthology of the SOTA as ANTH-LIT-085. Holding it here too follows ADR-013: this record holds it for the owner’s prior-art map, the anthology for ML practice.

Identification notes: Held in anthology (ANTH-LIT-085).

It was filed Deferred, unread. NOTE-283 is the close reading of 2026-09-29, and it placed the work: Active — worth reading as the fullest mechanistic account of a grokking network. Its learned structure is exactly the character basis of ℤ/113 (five real 2-dimensional irreps), which makes it the nearest existing empirical instance of the map’s row-7 “predicate harmonics = characters” prediction, in a setting where the group was known in advance.

Backlinks