Source
Li et al. (2017), LIT-014 — ARXIV-1712.09913.
The correlation, and what it is not
LIT-014 visualises the loss surface around trained minima by filter-normalised random directions, and the visible regularity is that solutions sitting in wide, flat basins generalise better than solutions in sharp ones. The filter normalisation is what makes the comparison meaningful — without it, rescaling a network’s weights changes the apparent sharpness without changing the function at all, which is how earlier sharpness claims were shown to be artefacts.
So the practice is a diagnostic reading rather than an objective: sharpness is correlated with test error across the architectures and training configurations examined, and the paper does not establish that flattening a minimum causes better generalisation.
Why the distinction matters here
Methods that optimise for flatness directly — sharpness-aware minimisation and its relatives — exist and sometimes help, and they are a different claim with their own evidence, which this record does not currently hold.
A deeper objection than the rescaling one
Filter normalisation answers the objection that sharpness is not reparameterisation-invariant. There is a second objection it does not answer, and the record now holds it: curvature may be the wrong quantity even when measured perfectly.
LIT-541 makes the case. Neural networks are singular statistical
models — many parameters give the same function, so the set of optima is a
variety rather than a point and the loss is not locally quadratic. In
Watanabe’s volume law V(ε) ∝ ε^λ, the quantity that enters the model-selection
criterion (n L_n(w_0) + λ log n) and the Bayes generalisation rate (λ/n) is
the exponent λ — the number of directions that change the function.
Curvature is the prefactor, and the prefactor appears in neither. In the source’s
own words, the RLCT “matters more than the curvature of those directions (as
measured for example by eigenvalues of the Hessian) laying bare some of the
confusion over ‘flat’ minima”.
This does not retire the practice. LIT-014’s correlation is a measurement and
it holds; the advice below — treat a visibly sharp basin as worth suspecting —
costs nothing and is still good. What it does is bound the explanation: the
correlation is evidence that something about the local geometry tracks
generalisation, and the geometry a visualisation displays may not be the part
doing the tracking. SOTA-325 is the practice that measures the exponent
instead, and THEORY-075 is the account behind it.
Reading this practice as “make the minimum flatter and the model generalises” is the failure it invites. What it supports is narrower and still useful: a model whose basin is visibly sharp is worth suspecting, and a change that sharpens the landscape (removing skip connections, per THEORY-011, or raising depth without them) has a known cost that shows up in the surface before it shows up in the metric.