DP-010: The most citable sentence in a paper is often its least supported one

Papers are written to be read once. The sentence that survives that reading — the one that ends up in an abstract, a talk, a survey, a summary: field — is selected for memorability, and memorability is not correlated with evidential weight. Often it is anti-correlated, because the claims that are easiest to state are the ones a paper has had to hedge.

This is not an accusation of bad faith. In every case below the authors said what they had done and what they had not, in the same document, usually within a page of each other. The hedge is present and it does not travel.

Four from one week

• nGPT (LIT-472). The abstract reports 4–20× fewer training steps. Footnote 3, same page as Figure 1, reports step time 80% higher at 4k context and 60% at 8k. The wall-clock figure is roughly half the headline, and the headline is the one in the unit that flatters.
• The Coverage Principle (LIT-474). Globally normalized SGD is proved to remove a sequence-length dependence. The next sentence says “somewhat speculatively, we believe that it may be possible” to extend this to Adam, and notes that Adam’s per-coordinate normalization is a meaningful difference. One of those two sentences is a theorem.
• Local volume (LIT-476). Section 3.3 argues that adaptive optimizers generalize worse because they partly undo the architecture’s parameter-to-function map. It is the best idea in the paper and no figure touches it.
• Zero-shot chain of thought (LIT-469). MultiArith 17.7 → 78.7 is the number everyone quotes. The sentence that decides the practice is in the body: it underperforms few-shot CoT and beats eight-shot standard prompting, which makes it a first move rather than a better method.

Four papers, four weeks apart in filing, one shape.

Four more, from a single session’s reading

Filed as a deliberate test rather than as a coincidence: DP-009 says counting is the antidote, so the four above were a claim about a sample of four. These are the next four papers read end to end, chosen off a revisit ranking for unrelated reasons, and every one of them has the shape.

• Mixture-of-Transformers (LIT-483). “Dense performance at 55.8% of the FLOPs” is the headline and it is FLOP-matched, not memory-matched: a sparse architecture at identical FLOPs holds more total parameters than the dense baseline it matches. The paper’s framing is standard for sparse models and correct; the sentence still travels without the denominator that decides a deployment.
• SimpleStories (LIT-485). The abstract reports “improved sample efficiency compared to the TinyStories dataset” from a comparison varying dataset, tokenizer and architecture at once. The one fully controlled arm in the paper is about the tokenizer — data and architecture held, +26.7 coherence and +27.5 quality — and it is a footnote to the paper’s own story. The Limitations section names the missing arm precisely.
• ParScale (LIT-486). It is called a parallel scaling law. The theoretical result makes the parameter multiplier depend on the diversity between streams, which is never measured; log P was chosen after looking at the curve because the gains from 1→2, 2→4 and 4→8 were about equal. The theory permits anything from logarithmic to a power law, and the evidence stops at P = 8.
• AI on Change.org (LIT-487). The inverse case, and the most striking. The abstract reports the null — outputs changed, outcomes did not. The sentence that makes the null mean something is in an appendix and one discussion line: the text features, and rated quality and persuasiveness, became weaker or negative predictors of outcomes afterwards. Here the underreported sentence is the well-evidenced one; what it lacks is space, not support.

That last case matters for what this principle is. The failure is not that authors overclaim. It is that compression is not selective for evidence, in either direction: it promotes the memorable unsupported sentence and it demotes the unmemorable load-bearing one. LIT-487’s inversion was not preregistered and is reported once, and everything that makes that paper general rather than a fact about petitions rests on it.

Eight papers now, across two batches counted separately, with the second batch chosen for reasons having nothing to do with this principle. That is the difference between a pattern and four anecdotes, and it is why this is a principle rather than an observation.

Why it matters to a record rather than to a reader

A reader who quotes the wrong sentence is wrong once. A record that files it is wrong for everyone who comes after, and the error is laundered: the summary: field is read far more often than the document, and the document far more often than the paper.

The failure is also invisible to the usual checks. Every citation resolves. The paper says the thing. The lint is clean. Nothing in the record can tell that the cited sentence is the one the authors were least sure of, because that information lives in the hedge, and the hedge is what compression removes first.

What to do

Find the hedge before writing the summary. It is usually there: a footnote, a “somewhat speculatively”, a limitations paragraph, a sentence beginning “we leave to future work”. If a striking claim has no hedge anywhere near it, that is worth a second look rather than a relief.

State the unit, and prefer the one the reader decides on. Tokens versus wall clock, relative versus absolute, against which baseline. SOTA-282 says to quote the wall-clock figure for nGPT, and that instruction is the whole practical content of this principle for that document.

Put the unmeasured claim in the record as unmeasured, not out of it. Leaving it out is safe and lossy; the next reader hits the same sentence and has to do the work again. Naming it, and saying no experiment touches it, is what stops it entering as a finding later.

And say which practice it does not move. SOTA-001 recommends Adam, and two of the first four papers above contain a sentence that looks like it bears on that recommendation. Neither does. Writing that down costs a line and is the only thing that prevents the inference being drawn silently in six months.

Look for the underreported sentence too, not only the overreported one. LIT-487 is the case that makes this a separate instruction: its load-bearing finding is real, checked and given one line in an appendix. A reading that only guards against overclaiming will file the abstract’s null and lose the thing that made the null worth filing. The question to ask is not “is the headline supported” but “which sentence is doing the work”, and those have different answers about as often as they have the same one.

Its relation to DP-005

DP-005 says adoption is not evidence — that a practice being widespread says nothing about whether it was ever tested. This is the same failure one level down: a sentence being widespread says nothing about whether it was ever supported. Both are about the gap between how well-travelled a claim is and how well-founded, and both have the same remedy, which is to go and look.