Berta, Holzmüller, Jordan and Bach (2025) — ARXIV-2501.19195, read as NOTE-259.
Standing
This record held nothing on calibration before it. Searching the practices
for calibration returns quantization scales, data-mixing temperature
(SOTA-102, and Superseded) and incidental uses of the word
“confidence”. A corpus with eighty practices touching training and none about
whether a model’s probabilities mean anything was missing a subject, not a
paper.
It contributes two documents, on ADR-031’s split: SOTA-315 takes the instruction, and THEORY-060 takes the claim underneath it — that the two error terms have separate minimizers, which the paper both measures and analyses for high-dimensional logistic regression.
The evidence is unusually broad for a single group. 196 classification datasets, three model families, 30 hyperparameter configurations per split; separately, CIFAR-scale vision with ten runs per dataset and about 300 GPU hours. Most of what this record holds is a single run.
What it takes from LIT-616 is the instrument. Guo et al. found networks overfitting NLL while their error keeps falling, and fixed the resulting miscalibration with temperature scaling: one scalar fitted on held-out NLL, which cannot change the argmax. This paper turns that property into an estimator. Because temperature scaling moves only confidence, the validation loss after fitting it measures refinement error alone, so it fits a temperature every epoch and stops on that. Its bisection fit, on a loss convex in the inverse temperature, is reported as faster and lower in test loss than Guo et al.’s implementation.