Source
Kumar et al. (2024), LIT-186 — ARXIV-2411.04330.
Standard scaling laws are silent about numerical precision, which is strange given that precision decides both training cost and serving cost. This fits precision-aware laws for both sides.
The training side is the tidy half: low precision reduces the model’s effective parameter count, which makes the extra loss from training in low precision predictable rather than empirical, and lets the law price parts of a model held at different precisions. A practical suggestion falls out — training larger models in lower precision may be compute-optimal.
The inference side is the counter-intuitive one, and it is the recommendation. Degradation from post-training quantization increases with how much data the model was trained on. Past a point the law can locate, additional pretraining data becomes actively harmful if the model will be quantized afterwards. More training makes a model less robust to being compressed.
Both halves are LIT-186’s fits, and the recommendation originates in its inference result: fitting over 465 pretraining runs, the paper finds post-training quantization damage growing with training data and concludes that, for a model to be quantized, extra data past a locatable point is actively harmful. This practice is that conclusion turned into an instruction about when the token budget is set.
Why that is a recommendation and not a curiosity
The usual instinct — train on more data, it can only help — is wrong for any model that will be served quantized, which is most of them. And the decision it breaks is one normally made by different people at different times: the pretraining budget is set months before anyone picks a serving format.
The practice is therefore about sequencing rather than about a number: if the serving plan is FP4, that fact belongs in the room when the token budget is chosen, and the law is what lets it be priced instead of argued.
Conditions, and why this is Proposed
Both halves are unified into a single functional form, fit on over 465 pretraining runs — which is a lot of evidence for the shape — and validated up to 1.7B parameters and 26B tokens, which is far below everything that ships FP4 today. That gap is the whole reason for the status: the effect is well measured where it was measured, and the regime it matters most in is not that regime.
Read with SOTA-163, which says what each quantization width can bear. That practice answers “what will this format survive”; this one answers “does my pretraining choice make the answer worse”. Two frontier reports in the record quantize — DeepSeek-V4 and Kimi K3 — and neither says whether it priced this in.
Known implementations
- None reported. The two frontier reports that quantize are silent on the coupling.