Source
Dey et al. (2025), LIT-150 — CompleteP.
What CompleteP changes
µP (SOTA-143) transfers hyperparameters across width; across depth, as commonly used, it does not — the optimal base learning rate moves, and deep layers can learn lazily, barely leaving their initialisation. CompleteP, the parametrization with depth exponent α = 1, gives depth-wise transfer and non-lazy learning in every layer. Against µP it saves 11.8% of FLOPs for optimally shaped models and 34.4% for a 179-layer one, and it keeps a wider range of width-to-depth ratios compute-efficient, so shape can follow the hardware.
Why this is Proposed
One group and one paper, and no production report in the record is trained under it.
A result that does not count
One result has already arrived that does not count, and is worth naming so the next reader does not count it either. LIT-153 reports improved learning-rate transfer across widths and depths from an independent group — the same property this practice claims, by an unrelated mechanism (pinning weight-matrix norms rather than a depth exponent). Two routes to one property is evidence the property is reachable, not evidence that this route works. The question it answers — how deep to go at a fixed budget — is the one
LIT-119 settled by ablation (SOTA-125), and the reason to want it is to stop ablating.
The independent derivation, and what it does and does not settle
(Added at v2.) Zheng et al., LIT-462, reach this parameterization from a different direction: a spectral condition on RMS operator norms under joint width-depth scaling, derived with elementary linear algebra rather than the machinery LIT-150 used. CompleteP-style scaling comes out as the case where a residual branch holds two or more transformations. They sweep depth from 4 to 256 on GPT-2-style models under Muon-Kimi-AdamW, Muon-AdamW, Shampoo-AdamW and Sophia, and the optimal base learning rate holds.
Settled: that this is not one group’s result. The derivation is
independent, the optimizers are different, and the depth range is larger than
LIT-150’s. consensus moves to emerging on that.
Not settled: the scale. Every run is 300M tokens. The original
promote_when was written about independence because that was the visible
weakness; the weakness that remains is that no depth sweep in the record,
from either group, runs long enough to say the transferred value is still
right at the end of a real training run. That is now what promote_when
asks for.
Not claimed: that CompleteP itself was replicated. The paper’s own word is “recovers CompleteP-style results”, and it does not run AdamW — the optimizer LIT-150 defined the exponent for — on the grounds that prior work covered it. The record counts this as corroboration of the route and not as the literal test, and says so here rather than quietly treating the field as discharged.