Li et al. (2025) — ARXIV-2503.01840
Key takeaways
- The observation that motivates it is a negative one, and a good example of a scaling law failing usefully: the community’s default move is to scale up training data, and EAGLE barely improves when you do. The draft model was not data-limited; it was limited by what it was asked to predict.
- The constraint was feature prediction. EAGLE-3 abandons predicting the target model’s features in favour of predicting tokens directly, and replaces reliance on top-layer features with multi-layer feature fusion via a technique the authors call training-time test.
- With that constraint removed the draft model scales with data as expected. Up to 6.5× speedup, about 1.4× better than EAGLE-2, and a 1.38× throughput improvement in SGLang at batch size 64.
- Evaluated on chat and reasoning models across five tasks.
Standing in the anthology
Filed because frontier models in the record turn their multi-token-prediction head into a draft model of this shape (two, not the three this section first counted): LIT-131 says so explicitly — the MTP layer mirrors a backbone block, EAGLE-3’s draft is a single decoder layer of matching structure, so the pretrained head becomes the draft with the target frozen. LIT-135 ships an MTP head in the checkpoint for EAGLE-style speculative decoding. LIT-139 does not show the arrangement: this note used to say it did, and DeepSeek-V4’s full report (NOTE-361) keeps V3’s MTP module “without modification” as a depth-1 training objective and never mentions a draft model or speculative decoding. It is an adopter of the head, not of this use of it.
So the pair LIT-163 and this note explain a design that had been appearing in the record fully formed: train n heads for sample efficiency, then spend them at inference. Neither paper proposes that combination; the model reports discovered it.
The thing it accelerates is now filed. SOTA-227 is speculative decoding itself, from LIT-376 and LIT-375; this paper is a method for pushing the draft toward the near-free regime that practice’s walltime formula rewards. Until #87 there was no document for it, which left this note a refinement with nothing under it — DP-007’s second shape.
Read, and sourced. The reading is NOTE-163 and SOTA-229 is filed from it. Found by the reverse coverage query under #87 — papers supporting no practice — having been flagged by #121 as unsourced and unread and left that way.
It also carries a measurement this record needed and did not know it had: speculative sampling improving throughput by 40% at batch size 64, against the expectation that it is latency-only. SOTA-227’s conditions are corrected accordingly.