DP documents

1. A recommendation without a citation is an opinion — make the citation structural

The premise of an anthology of current practice is that every entry has a reason, and the reason is somebody’s published work. That premise survives exactly as long as the link from claim to evidence is mandatory. Make it optional and it decays quietly: nothing breaks, the list still renders, and over time it becomes a list of assertions with a provenance field that is usually blank.

The mechanism is not “remember to cite”. It is that a document which does not name its source fails the build. In this record that is requires = ["source"] on the SOTA scheme, and it is a small enough thing that it is worth being precise about why it works: the check fires at the moment of writing, while the person filing the practice still knows where they read it. A citation reconstructed six months later costs a session; one written at filing time costs a line.

Applied here: SOTA requires source:, and LIT requires a source of its own — an arxiv: or doi: a remote can resolve, or failing both a url:; at least one, enforced (ADR-009). While the requirement was arxiv: alone it immediately found two papers in the corpus that had reached it with no identifier at all; it was widened when the first report worth filing had never been posted there. The arXiv and DOI remotes turn those ids into resolvable references rather than strings, so “cite the paper” and “the citation resolves” are the same check rather than two — which is also why a bare url: is the last resort and not the general case.

The corollary is the expensive part. Requiring the field is easy; keeping it meaningful means the evidence has to be a document in the record, not a URL — because a URL cannot be found to be retired, cannot be counted, and cannot tell you that eleven practices depend on a paper you just decided was wrong.

v2 · shaped by ADR-001, ADR-002, ADR-009 · origin: 119 recommendations carried a source field and not one carried the supporting_evidence the schema defined for it — the field existed, was empty in every record, and nothing noticed for two years

2. A field every record shares is not a field — it is a comment with a schema

When every record agrees, the field has told you nothing. The dangerous part is that it does not look like nothing: it looks like a tracked attribute. It has a type, it appears in the documentation, downstream code branches on it. A reviewer reading the schema concludes the project distinguishes three kinds of recommendation. A reviewer reading the data would have found it distinguishes none.

The failure that produces this is almost never “nobody set the field”. It is that the code which would set it cannot run. Here, both branches that could assign a non-default status read a list that no qualifying record had, so they were unreachable on the actual corpus — and unreachable code fails silently by construction, because the way it announces itself is by not happening.

So the check has to be on the data, not on the code. “Do the records disagree?” is answerable by looking at them and unanswerable by reading the branch that was supposed to make them disagree.

Applied here: luria lint reports inert-status when every document in a scheme shares one status, above a floor where uniformity stops being explainable by youth. The SOTA scheme now holds 119 Active and one Deferred — a thin margin, and an honest one, because the Deferred record is the one hedge the corpus actually contained.

The corollary: a uniform field is not automatically a bug, and treating it as one produces a check people switch off. Sometimes every record really is in force. The finding is “this field is currently carrying no information”, addressed to a person, and a scheme that declares a vocabulary of one is a project saying so deliberately.

v1 · shaped by ADR-001, ADR-002 · origin: All 119 recommendations sat at one status because the two code paths that could set any other read a field no qualifying paper had. The schema defined three states, the data expressed one, and the build was green throughout

3. Keep what you stopped believing — the contrast is most of the content

“Do this” is advice. “Do this, and here is the thing everybody did until 2021 and why it stopped being right” is knowledge. The second is what a reader comes for, and it only exists if the abandoned position is still written down.

There is a sharper reason than pedagogy. A document that silently drops what it used to say cannot be checked. Every version of it looks equally authoritative, because the evidence of it having been wrong is removed at the same time as the wrongness. This is the specific way a “living document” goes bad: not by being wrong, but by becoming unable to be shown to have been.

Retirement therefore has to be an edit to a status, never a deletion — and the reason has to render where a reader meets the entry, not sit in a field that only the file’s author ever opens.

Applied here: Rejected and Superseded are statuses on a document that keeps its body; retired entries appear in the generated index with their reason in the status column; and a live practice citing a retired paper is a lint finding rather than something to be noticed. The migration surfaced a chain nobody could have read from the source data — a paper superseded by one that was itself later rejected — which is exactly the shape of thing that only exists if you kept both.

The corollary people skip: keeping the entry obliges you to keep it findable. A retired document that stays in the record but drops out of the indexes has been deleted in every sense that matters to a reader; it just still occupies a file.

v1 · shaped by ADR-004 · origin: Nine papers had been retired with reasons good enough to be worth reading — “too domain-specific”, “needs per-case tuning, no consistent benefit” — and every one of those reasons was invisible in generated output

4. A pass finds only the defects its query is shaped like — a clean run is evidence about the query, not about the record

An audit begins by choosing a candidate list, and the choice is usually made in a sentence, early, while thinking about something else. That sentence decides everything the pass can find. Run it to completion, fix every finding, and what you have earned is the absence of one shape of defect — which feels identical, from the inside, to the absence of defects.

The gap is not carelessness and cannot be closed by care. A query is a filter and a filter has a complement; running the filter harder does not sample the complement. The only thing that finds the complement is a different query, chosen deliberately, usually by asking what the first one could not have returned.

Three ways it goes wrong, all of them observed here in one week:

Direction. A relation has two ends, and a query over it picks one. A pass whose candidate list is “codes cited in the body that the evidence field does not name” can only add evidence and can never remove any, because a code wrongly present produces no body-only citation to flag. Every batch of that pass was correct. The pass could not have found the three practices citing adoption as support, and asking the opposite question — which entries in this field are model reports? — returned them immediately.

Recency. A pass keyed to what changed cannot see a document whose surroundings changed. A re-derivation of consensus values, whose candidate list was “practices whose support lists moved”, missed a practice whose note said “nobody has contradicted it either” while its own body argued with two papers that do. Its support list had not moved. Everything around it had.

Evidence of absence. A refusal leaves a sentence you can search for. An omission leaves nothing. Grepping the record for “not filed as a practice” is a complete method for one half of a problem and returns literally nothing of the other half — the techniques so settled that nobody ever argued about them, and so were never written down at all.

Applied here: every pass in this record’s history is recorded in the journal with the query it ran, which is the only form in which a future reader can tell what it did not cover. The check that shipped with the evidence-field decision has this property and says so in its own text — it verifies one containment and is blind to the reverse, and that is written down beside it rather than discovered later by someone trusting a green run.

The corollary, and it is cheap: after a pass over a relation, run it once in the other direction. It takes minutes, it is mechanical, and here it found three defects in a field that had just been declared clean. The expensive version of this lesson is the one where a clean report is quoted as a guarantee six months later by someone who was not there when the query was chosen.

v1 · shaped by ADR-015, ADR-017

5. Adoption is not evidence — who does a thing and whether it works are different questions

Two facts about a recommendation feel like the same fact and are not. One: somebody tested it and reports what happened. Two: somebody shipped it. The second is easier to collect, arrives in greater volume, and is what a practitioner most wants to hear — which is precisely why it ends up doing a job it cannot do.

A report that ships a design without measuring it is evidence that the design was chosen. It is not evidence that the choice was right, and it is frequently not even independent: the group shipping it read the same paper you did, or wrote it. Stack four such reports in the column meant for evidence and the recommendation looks four times better supported while nothing has been tested twice.

The confusion is not sloppiness. It is that both facts genuinely bear on the recommendation, so the instinct to record both is correct. What is wrong is recording them in one place, because they answer different questions and a reader cannot recover which is which once they are merged.

Applied here: source: holds the work that produced evidence about the claim — the ablation, the controlled comparison, the paper that isolated a change — while adoption goes to consensus: and its note, where counting is the right operation. One practice had written the distinction in its own prose, calling four model reports “four generations of adoption rather than one result”, and then listed all four as support anyway. It now names the one work that ran the comparison.

The corollary that makes this pay: once separated, both columns start answering questions they could not before. A recommendation with one source and eight adopters is a different object from one with eight sources and no adopters, and telling a reader which they are looking at is most of what a collection like this is for.

v1 · shaped by ADR-010, ADR-015, ADR-017

6. Filing is not endorsement — the bar for having a document is lower than the bar for believing it

A collection of recommendations has two thresholds and it is easy to run them together. One decides whether a claim gets a document. The other decides whether the project is willing to say the claim is right. They are not the same threshold, and using the second for the first is a mistake that looks like rigour.

The failure is specific: a paper arrives with a real recommendation and weak-but-honest evidence, and the reflex is not yet. Nothing bad appears to happen. But three things are lost at once, and none of them is the endorsement that was correctly withheld.

A refusal has nowhere to put what would reverse it. The condition — “promote this when a second group reports X” — is the most perishable fact in the whole encounter, and with no document to carry it, it goes in prose, in a note about a different subject, where nothing will point at it when the evidence arrives.

A judgement about the field needs something to be a judgement about. How far the field has converged is a property of a recommendation. Withhold the recommendation and the second lab to publish has nowhere to be recorded; the counter that would eventually earn the endorsement was removed by the refusal.

And the unused status decays into no status at all. If only claims that clear the top bar are filed, the provisional statuses fill with nothing but the cases somebody filed optimistically and forgot — which is exactly the distribution DP-002 is about, reached from the other direction.

Applied here: status: carries the belief, consensus: carries the field’s, and promote_when: carries the condition — so the entry question is only whether the work contains an instruction. Twelve practices were filed in one day that the evidence had supported for months, three of them refused in writing on the grounds that nobody at frontier scale had adopted them yet. That is a correct reason not to mark something Active and not a reason for it to be absent.

The corollary people skip: this cuts the other way too. A document filed at a provisional status is a promise to revisit, and a provisional status with no condition attached is the same refusal wearing a document’s clothes.

v1 · shaped by ADR-014, ADR-015

7. What everyone agrees on has no author, so nothing prompts anyone to write it down

A collection assembled from published work inherits the publication record’s shape, including its blind spot. Papers are written about what is new. So the filing habit runs on arrivals: something appears, it has a name and a claim, and the claim becomes an entry.

Agreement has no arrival. Nobody publishes the thing four papers assume, because assuming it is not a contribution — and a step everybody takes generates no argument, so there is nothing to notice. The result is a collection that is systematically strongest on what is disputed and weakest on what is settled, which is close to the opposite of what a reader wants.

It shows up in two shapes, and the second is the one that looks absurd afterwards.

The trunk with no document. Four papers propose four ways to constrain the same mechanism. Each disagreement gets an entry, because each is somebody’s contribution. The thing all four take for granted — the reason any of them is worth doing — gets none. The disputed part is documented and the agreed part is not.

The refinement filed before the thing it refines. Somebody measures a knob on a technique, and the measurement is recent and local and feels like news, so it is filed. The technique it is a knob on was established years earlier and simply assumed, so it is not. A reader then finds a footnote with no text above it.

Applied here: the record carried a practice about whether to mask the loss on part of a training sample for four months before it carried the practice of constructing that sample at all; and it carried two rival constraints on a widened residual stream while the widening itself — the thing both rivals exist to make usable — had no entry until somebody went looking. Neither gap was a judgement. Nothing had ever prompted the filing.

The corollary, which is the only reliable defence: you cannot find these by reading what is here. A refusal leaves a sentence you can search for; an agreement leaves silence. The queries that work run backwards from the evidence — which sources support no recommendation, which notes explain a decision made in some other document — and they have to be run deliberately, because nothing will ever remind you to.

v1 · shaped by ADR-011, ADR-017

DP-008: A vocabulary is a scope decision — a subject with no category is one you declined to hold

A closed list of categories reads like a filing convenience. It is not. Every entry has to take exactly one, so the list is the answer to “what is this project about” — written down in a config file, usually by someone solving a smaller problem, and thereafter binding on everything.

That is fine, and it is better than the alternative of an unbounded list that decides nothing. The hazard is that the decision becomes invisible. Nobody re-reads a vocabulary; they read the entries that use it. So a subject the list cannot express does not produce an error or an argument — it produces nothing, and the absence is indistinguishable from nobody having got round to it.

The practical test is worth stating, because it converts a vague sense of scope into a checkable question: if a claim arrived tomorrow with excellent evidence and no category to take, would you add a category or decline the claim? An honest answer to that is the scope, and it is usually more specific than anyone would have said out loud.

Applied here, and this project is its own worked example twice over.

The two schemes carry different vocabularies. Read one way, that arrangement said: the reading list holds vision, generative and theory work, the recommendations hold seven topics none of which is about vision, so a vision paper can be a note and cannot be a recommendation — not on judgement, not on evidence, but because there is no category for it to take. The recommendations are scoped to language-model training by construction, which nobody had written down.

That reading was load-bearing for about a year and it was wrong, in both directions at once. It was too narrow: two Active serving practices were already sitting outside it and nobody noticed, which is this principle’s own warning about vocabularies working in the dark. And it was too broad: it declined work whose transferable content — a preconditioning scheme, an annotation bootstrap — is exactly the kind of claim the seven topics exist to hold, so the scope was making the record ignorant of its own subject.

The repair is the corollary below, taken seriously. ADR-026 — superseding ADR-020, whose decision it carries forward — scopes the recommendations by the kind of claim, so that where a technique was discovered stops deciding whether it can be recommended. Read that decision, not this paragraph, for what the scope now is.

The corollary: because it is a decision, it can be revisited — but only deliberately, by changing the vocabulary, and never by one entry quietly taking a tag that does not fit. A category added to admit a single document is how a scope stops being one.

The mechanism this principle warns about is now gone for this pair

ADR-026 went further than the repair above and gave the practice registry the same thirteen topics the reading list has. So the arrangement that made this a worked example — two vocabularies, one narrower, the gap between them silently encoding a scope — no longer exists here. No LIT tag can read as a scope boundary, because every one of them is a word a practice can carry.

That decision adds two categories admitting no documents at all, which is worth holding against the corollary directly. It passes, and the reason is the distinction the corollary is drawing. What it forbids is widening a scope by the back door, one entry at a time, to avoid making the decision — a category bent to fit a document somebody wanted in. ADR-026 makes the decision in the open and admits nothing: everything now admissible was already admissible under ADR-020, by kind of claim. The vocabulary caught up with a scope that had already changed, rather than a scope changing because a vocabulary did.

The principle survives its own application intact, and gains a sharper test from it: ask whether the category was added because a decision was made, or so that one would not have to be.

v2 · shaped by ADR-002, ADR-003

9. A claim with no category is a finding about the vocabulary, not about the claim

DP-008 establishes that a controlled vocabulary is a scope decision: the list of categories answers “what is this project about”, and a subject the list cannot express produces nothing rather than an argument. This is the operational half of that, and it points the other way.

When a document arrives with a real claim and no category to take, there are two available readings and they are not symmetric:

1. The claim is out of scope. The vocabulary is right and the document is declined.
2. The vocabulary is short a category. The document is in scope and the list has not caught up.

The first is the cheaper reading, it requires no work, and it is self- confirming: declining the document leaves the vocabulary looking complete, because the evidence that would have shown otherwise has just been turned away. Reading (2) costs a decision and some retagging, and it is the one that can be wrong in a way anybody notices.

So the default has to be (2). A claim the categories cannot express is evidence about the categories until someone shows otherwise, and showing otherwise means an argument about the kind of claim — not a shrug at the absence of a slot.

The failure this prevents is specific and it has a shape: the project reads its own filing convenience back as a statement about the world. It is especially quiet when the vocabulary is nearly right, because then only a few documents fall outside and each looks individually marginal. Counting is the antidote — if several documents are being declined for the same missing category, the category is missing.

Two tests worth applying before accepting reading (1):

• Would you decline this claim if it arrived with excellent evidence and a familiar author? If not, the objection was never about scope.
• Are the claims already in the record on this subject filed coherently? A subject scattered across four ill-fitting categories is a category that exists in the corpus and not in the list.

The corollary from DP-008 still binds: the vocabulary changes deliberately, by decision, and never by one entry quietly taking a tag that does not fit. This principle says which way to lean when writing that decision, not that the decision can be skipped.

v1

DP-010: The most citable sentence in a paper is often its least supported one

Papers are written to be read once. The sentence that survives that reading — the one that ends up in an abstract, a talk, a survey, a summary: field — is selected for memorability, and memorability is not correlated with evidential weight. Often it is anti-correlated, because the claims that are easiest to state are the ones a paper has had to hedge.

This is not an accusation of bad faith. In every case below the authors said what they had done and what they had not, in the same document, usually within a page of each other. The hedge is present and it does not travel.

Four from one week

• nGPT (LIT-472). The abstract reports 4–20× fewer training steps. Footnote 3, same page as Figure 1, reports step time 80% higher at 4k context and 60% at 8k. The wall-clock figure is roughly half the headline, and the headline is the one in the unit that flatters.
• The Coverage Principle (LIT-474). Globally normalized SGD is proved to remove a sequence-length dependence. The next sentence says “somewhat speculatively, we believe that it may be possible” to extend this to Adam, and notes that Adam’s per-coordinate normalization is a meaningful difference. One of those two sentences is a theorem.
• Local volume (LIT-476). Section 3.3 argues that adaptive optimizers generalize worse because they partly undo the architecture’s parameter-to-function map. It is the best idea in the paper and no figure touches it.
• Zero-shot chain of thought (LIT-469). MultiArith 17.7 → 78.7 is the number everyone quotes. The sentence that decides the practice is in the body: it underperforms few-shot CoT and beats eight-shot standard prompting, which makes it a first move rather than a better method.

Four papers, four weeks apart in filing, one shape.

Four more, from a single session’s reading

Filed as a deliberate test rather than as a coincidence: DP-009 says counting is the antidote, so the four above were a claim about a sample of four. These are the next four papers read end to end, chosen off a revisit ranking for unrelated reasons, and every one of them has the shape.

• Mixture-of-Transformers (LIT-483). “Dense performance at 55.8% of the FLOPs” is the headline and it is FLOP-matched, not memory-matched: a sparse architecture at identical FLOPs holds more total parameters than the dense baseline it matches. The paper’s framing is standard for sparse models and correct; the sentence still travels without the denominator that decides a deployment.
• SimpleStories (LIT-485). The abstract reports “improved sample efficiency compared to the TinyStories dataset” from a comparison varying dataset, tokenizer and architecture at once. The one fully controlled arm in the paper is about the tokenizer — data and architecture held, +26.7 coherence and +27.5 quality — and it is a footnote to the paper’s own story. The Limitations section names the missing arm precisely.
• ParScale (LIT-486). It is called a parallel scaling law. The theoretical result makes the parameter multiplier depend on the diversity between streams, which is never measured; log P was chosen after looking at the curve because the gains from 1→2, 2→4 and 4→8 were about equal. The theory permits anything from logarithmic to a power law, and the evidence stops at P = 8.
• AI on Change.org (LIT-487). The inverse case, and the most striking. The abstract reports the null — outputs changed, outcomes did not. The sentence that makes the null mean something is in an appendix and one discussion line: the text features, and rated quality and persuasiveness, became weaker or negative predictors of outcomes afterwards. Here the underreported sentence is the well-evidenced one; what it lacks is space, not support.

That last case matters for what this principle is. The failure is not that authors overclaim. It is that compression is not selective for evidence, in either direction: it promotes the memorable unsupported sentence and it demotes the unmemorable load-bearing one. LIT-487’s inversion was not preregistered and is reported once, and everything that makes that paper general rather than a fact about petitions rests on it.

Eight papers now, across two batches counted separately, with the second batch chosen for reasons having nothing to do with this principle. That is the difference between a pattern and four anecdotes, and it is why this is a principle rather than an observation.

Why it matters to a record rather than to a reader

A reader who quotes the wrong sentence is wrong once. A record that files it is wrong for everyone who comes after, and the error is laundered: the summary: field is read far more often than the document, and the document far more often than the paper.

The failure is also invisible to the usual checks. Every citation resolves. The paper says the thing. The lint is clean. Nothing in the record can tell that the cited sentence is the one the authors were least sure of, because that information lives in the hedge, and the hedge is what compression removes first.

What to do

Find the hedge before writing the summary. It is usually there: a footnote, a “somewhat speculatively”, a limitations paragraph, a sentence beginning “we leave to future work”. If a striking claim has no hedge anywhere near it, that is worth a second look rather than a relief.

State the unit, and prefer the one the reader decides on. Tokens versus wall clock, relative versus absolute, against which baseline. SOTA-282 says to quote the wall-clock figure for nGPT, and that instruction is the whole practical content of this principle for that document.

Put the unmeasured claim in the record as unmeasured, not out of it. Leaving it out is safe and lossy; the next reader hits the same sentence and has to do the work again. Naming it, and saying no experiment touches it, is what stops it entering as a finding later.

And say which practice it does not move. SOTA-001 recommends Adam, and two of the first four papers above contain a sentence that looks like it bears on that recommendation. Neither does. Writing that down costs a line and is the only thing that prevents the inference being drawn silently in six months.

Look for the underreported sentence too, not only the overreported one. LIT-487 is the case that makes this a separate instruction: its load-bearing finding is real, checked and given one line in an appendix. A reading that only guards against overclaiming will file the abstract’s null and lose the thing that made the null worth filing. The question to ask is not “is the headline supported” but “which sentence is doing the work”, and those have different answers about as often as they have the same one.

Its relation to DP-005

DP-005 says adoption is not evidence — that a practice being widespread says nothing about whether it was ever tested. This is the same failure one level down: a sentence being widespread says nothing about whether it was ever supported. Both are about the gap between how well-travelled a claim is and how well-founded, and both have the same remedy, which is to go and look.

v2