Hennick, M., & De Baerdemacker, S. (2025). Almost Bayesian: The fractal dynamics of stochastic gradient descent. arXiv:2503.22478. Published at ICLR 2026 as “Almost Bayesian: Dynamics of SGD Through Singular Learning Theory”.
Read in full in NOTE-1467.
Why this is here
A4 gave it as the strongest recent form of the SGD–Bayes connection, with “accessibility constraints” as what separates the two. That is the term QUESTION-363 asks about. The reading finds less there than the question’s title expects. Within a region the paper’s steady state is a tempered posterior, so the correspondence it gives is between a noise scale and a sample size, as in Mandt et al. (LIT-1898), and not between training time and sample size. Accessibility, the part that would make SGD’s weighting of regions differ from the posterior’s, is not derived (THEORY-1258). Phase transitions are explicitly outside its steady-state theory. Its displacement results bear on how SGD moves late in training (THEORY-1272).