Context
Two documents from the current registry, side by side:
- SOTA-044 — “Pin memory for CPU-GPU transfers”
- SOTA-121 — “Use Muon with decoupled weight decay and AdamW-matched update RMS in place of AdamW”
The first is a PyTorch flag. The second is an optimizer change with scaling-law evidence, a production run at 1T parameters behind it, and three follow-on practices. In the schema they are the same kind of thing, carry the same statuses, and count equally in every total the record reports.
The distribution says how far this goes: 103 of 144 practices have no body
at all — heading, ## Source, nothing else. And they are not scattered.
They arrive in fours: SOTA-042–SOTA-045 on data loading, SOTA-054–SOTA-057 on
checkpointing, SOTA-081–SOTA-084 on compilers. Those quartets are visibly
what they are: one synthetic note’s four bullets, promoted one-to-one into
four recommendations.
So a large fraction of the registry’s apparent weight is a generation artefact, and nothing in the schema makes that visible. A reader counting “139 practices” is counting bullets and design decisions in one number.
This also made the citation failure harder to see. Four sibling practices sharing one source looks like thoroughness. It is equally the signature of a bullet list, and the schema could not tell the two apart.
Decision (proposed)
Add kind: to SOTA, from a small closed vocabulary:
design-decision— a choice about what to build, with published evidence and consequences that propagate. Muon over AdamW; a 3:1 hybrid interleave; staged context extension.recipe-knob— a setting within an approach already chosen. One epoch of DPO; unmasked FIM loss; a decay stage of 10% of tokens.operational-default— a well-known engineering default that the anthology records for completeness. Pin memory; four workers per GPU; use tar archives.
And one lint rule, which is the point of the field rather than a side effect:
a design-decision with no body is a finding. If a practice claims to be
a choice with consequences, it owes the reader the consequences — the
conditions under which it holds, what it costs, what it replaced. An
operational-default owes nothing beyond its source, which is why the
vocabulary needs a floor as well as a ceiling.
Alternatives considered
Infer altitude from body length. Rejected: it measures the wrong thing and would be gamed instantly by a paragraph of padding. Length is evidence about a document; altitude is a claim about the world.
Split into two schemes. A PRACTICE scheme and a DEFAULT scheme would
be structurally cleaner and is rejected for the reason ADR-002 gives about
SOTA/LIT: two schemes are worth it when the documents have genuinely
different statuses and lifecycles, and these do not. A pinned-memory default
is retired the same way a design decision is. One scheme with a field is the
smaller change that carries the same information.
Do nothing; the titles already say it. They do to a human reading carefully, and not to anything that counts. The registry’s own summary statistics are the first casualty, and those are what a newcomer sees.
Why this is Proposed
Assigning kind: to 144 existing practices is a curation pass, not a
migration — it is exactly the judgement a schema change cannot make for you,
and doing it badly would bake in a wrong taxonomy that everything downstream
then quotes. The three-value vocabulary above is also the part most likely to
be wrong on contact: it may want a fourth value for measurement practices
(“profile data loading separately”), which are neither a design choice nor a
default but an instruction about how to know.
The right order is to assign kind: to the practices added since the
migration — where the filer’s intent is recent and recoverable — see whether
the vocabulary survives that, and only then work backwards through the
inherited 119.