Source
Kim et al. (2025), LIT-441 — ARXIV-2509.14786.
The argument
Scaling comparisons are conventionally reported at a compute budget: recipe A
against recipe B at 10^20 FLOPs. That is the right comparison when compute
is what you are short of.
When data is the constraint and compute is not, it answers a question nobody
asked. The interesting quantity is how good this recipe can get — the limit
of its loss as its scaling variable goes to infinity. Fit
L(x) = A/x^α + E and compare E.
This is LIT-441’s evaluation protocol, proposed there rather than measured:
the paper fits L(N) = A/N^α + E to its regularized recipe and
L(K) = A/K^α + E to its ensembles, and reports every comparison as the
asymptote — about 3.43 for parameter scaling, 3.34 for ensembling and 3.13 for
the two composed, against 3.75 for the unregularized baseline.
Why it matters rather than being bookkeeping
Because the two orderings disagree. In the source, the ensembling recipe
loses to parameter scaling at small member counts and wins on asymptote; the
best hyperparameters at K = 1 are not the best hyperparameters for the
limit. A comparison run at a budget would have reached the opposite
conclusion and would have tuned the losing configuration.
That is the whole content of the recommendation: a fixed-budget comparison of two recipes is a statement about that budget, and reporting it as a statement about the recipes is a category error whenever the recipes’ curves have different shapes.
The precondition is doing real work
Monotone. The criterion applies only where loss decreases in the scaling variable without turning up. The source’s own standard recipe is not monotone in parameter count under a data constraint — it overfits — and for it the asymptote is undefined and the criterion says nothing. Getting a recipe into the regime where this question can be asked is itself an intervention (SOTA-255).
What it costs
An asymptote is an extrapolation, and it is the least well-determined parameter of a power-law fit — small changes in the exponent move it a lot. The source’s best number is an asymptote of asymptotes, and its sensitivity analysis covers seed variance in the individual fits rather than error propagated through the double limit.
So the criterion buys the right question at the price of a less certain
answer, and a report using it owes the reader the fit, the range fitted over,
and the sensitivity — not just E.
Conditions, and why this is Proposed
One group, and unlike the rest of that paper this is argued rather than measured: it is the protocol the results are reported under, not a result. The record files it because two other practices drawn from the same source (SOTA-255, SOTA-257) rest their comparisons on it, and a reader weighing those is entitled to see the criterion stated and its cost named rather than inheriting it silently.
Known implementations
- None in the record. Every scaling comparison it holds is reported at a budget.