Source
Tay et al. (2021), LIT-052 — ARXIV-2109.10686.
Why this is superseded rather than rejected
The other four practices drawn from LIT-052’s replaced takeaways claim things no paper in the record supports. This one is different: gradient clipping is real, near-universal, and already in the record with a correct citation. SOTA-035 recommends it from Pascanu et al. (LIT-037), where the argument about exploding gradients was actually made.
So the failure here is narrower and worth separating from the other four. The attribution is wrong — the source contains “gradient clip” zero times — and the qualifier is wrong too.
The qualifier is the part that would have misled
“During early training phase” implies clipping is a warmup-period measure to be relaxed once the run settles. It is not. Clipping stays on for the whole run in every frontier recipe in the record, because the spikes it exists to bound are not confined to early training — a bad batch, a stale checkpoint resume, or a data-mixture switch can produce one at any point, and by then the practice as written would have had you turn the guard off.
That is why this needed a status change rather than a body explaining the
mistake. Left Active, it read as a live recommendation to stop clipping.