Context
data/research.yaml and data/registry.yaml held 118 papers and 119
recommendations, and between them they modelled four ideas: a family of
referable documents, a status vocabulary, a tag taxonomy, and citations
pointing at work outside the project.
Three of the four were inert, and an audit before this migration could show it rather than suspect it:
- Every recommendation was
standard.MLRStatusdefinedexperimentalanddeprecated, and the builder had branches that set them — but both branches readpaper.sota, and no experimental paper (3 of them) and no superseded paper (2) had asotalist. The branches could not execute on this data. - The attic pointed at nothing checkable.
attic.superseded_byandattic.see_alsoheld bare arXiv ids, three of which —2110.08058,2002.05709,1911.05722— name papers that are not in the corpus. Nothing resolved them, so nothing noticed. - The taxonomy was a specification nobody applied. See ADR-003.
The fourth, citations, worked in the sense that an id was recorded, and not in any sense a reader could use: no link, no check that the target existed.
That is the specific failure mode: a schema rich enough to state the rules,
with no mechanism that could ever object. The Recommendation dataclass has
a superseded_by field. It is None 119 times out of 119.
Decision
Adopt luria and make the record the
ground truth. Four schemes: SOTA for practices, LIT for papers (ADR-002),
ADR and DP for the project reasoning about itself. arXiv becomes a
configured remote, so ARXIV-1412.6980 written anywhere in the record is a
resolvable citation rather than a string in a field.
The load-bearing part is not the file format. It is that every rule the old
schema stated in prose now has something that fails when it is broken —
inert-status for a status column carrying no information, unresolved-codes
for a citation that resolves to nothing, retired-citations for a live
practice sourced from an atticked paper, tag_groups for one-primary-topic.
Alternatives considered
- Write validators against the existing YAML. The honest comparison, and
it is most of the same work: a resolver for arXiv ids, a checker for
superseded_bytargets, a taxonomy validator, a status audit. What it does not get is the generated views, the citation graph across schemes, or a format where a paper’s note can be prose. The YAML’s real limit is thatkey_takeawaysandattic.reasonare editorial writing crammed into list items, and no amount of validation fixes that. - Keep the registry and add a docs site. Treats the symptom. The reason
REGISTRY.mdis thin is that its source is thin. - Status quo. Costs nothing today and keeps costing what it has been costing: five consecutive commits named “Update ML training registry”, a status field that has never once distinguished two records, and an attic whose cross-references have been broken for an unknown length of time.
Consequences
238 documents where there were two files, which is a real cost: the corpus is
no longer greppable in one pass, and a bulk edit is now a script rather than
a search-and-replace. The migration script stays in the tree
(src/scripts/migration/to_record.py) so that the transform is auditable
rather than archaeological.
data/registry.yaml keeps working, unchanged, for as long as the frontend
needs it. Inverting it into a generated projection is ADR-006, deliberately not part of this change.
The lint is warn-first: adopting this does not turn the backlog it finds into
a red build. [luria.lint] fail_on is empty, and promoting classes into it
happens once their findings are worked through.