nucleation

LIT-308: Stochastic Thermodynamics of Learning

1 min read

Properties5
StatusActive — read in full 2026-09-29 (NOTE-296); worth reading as the founding letter of the "stochastic thermodynamics of learning" line, and as the precise statement of what learning costs in that framework: entropy production, of which heat is only one part. Read the main-text claim "for arbitrary learning algorithm … at all times" alongside its supplement. The derivation uses a γ ≫ 1 (fast-label) limit and a weight-independence step that holds for batch but is not argued for online learning.
Version2
Tagslearning-theory, thermodynamics, natural-sciences, information-theory
Cited as “paper” byNOTE-296 — Stochastic Thermodynamics of Learning
Cited as “source” byTHEORY-030 — For a memory evolving reversibly with its environment, Landauer's bound prices only steps that lose logical distinguishability, at least k ln 2 of entropy per bit lost, exported to the environment and not necessarily as heat, so copying onto a blank register, measuring or evaluating a bit has no minimum cost; learning a label has a floor only in total entropy production, at least the information learnt, which in Goldt and Seifert's perceptron can be paid in the weights' own entropy with vanishing heat

Graph View

Sebastian Goldt, Udo Seifert (2017), Physical Review Letters 118, 010601 — DOI-10.1103/PhysRevLett.118.010601

Standing in the record

Registered on 2026-09-29 at the owner’s request, because the owner’s prior-art novelty map (a private working document on concepts as projectors, the linear representation hypothesis and their neighbours) cites it: row 13 and §4 must-cite: thermodynamics of learning. It is seeded from metadata and published descriptions and has not been read here.

Identification notes: arXiv v1 2016-11-28; published (PRL) 2017-01-06. Not held in either record (anthology ANTH-LIT-340 is a different Goldt paper, 2019 teacher-student SGD dynamics).

It was filed Deferred, unread. NOTE-296 is the close reading of 2026-09-29, and it placed the work: Active — worth reading as the founding letter of the “stochastic thermodynamics of learning” line, and as the precise statement of what learning costs in that framework: entropy production, of which heat is only one part. Read the main-text claim “for arbitrary learning algorithm … at all times” alongside its supplement. The derivation uses a γ ≫ 1 (fast-label) limit and a weight-independence step that holds for batch but is not argued for online learning.

Backlinks