Context

data/research.yaml and data/registry.yaml held 118 papers and 119 recommendations, and between them they modelled four ideas: a family of referable documents, a status vocabulary, a tag taxonomy, and citations pointing at work outside the project.

Three of the four were inert, and an audit before this migration could show it rather than suspect it:

  • Every recommendation was standard. MLRStatus defined experimental and deprecated, and the builder had branches that set them — but both branches read paper.sota, and no experimental paper (3 of them) and no superseded paper (2) had a sota list. The branches could not execute on this data.
  • The attic pointed at nothing checkable. attic.superseded_by and attic.see_also held bare arXiv ids, three of which — 2110.08058, 2002.05709, 1911.05722 — name papers that are not in the corpus. Nothing resolved them, so nothing noticed.
  • The taxonomy was a specification nobody applied. See ADR-003.

The fourth, citations, worked in the sense that an id was recorded, and not in any sense a reader could use: no link, no check that the target existed.

That is the specific failure mode: a schema rich enough to state the rules, with no mechanism that could ever object. The Recommendation dataclass has a superseded_by field. It is None 119 times out of 119.

Decision

Adopt luria and make the record the ground truth. Four schemes: SOTA for practices, LIT for papers (ADR-002), ADR and DP for the project reasoning about itself. arXiv becomes a configured remote, so ARXIV-1412.6980 written anywhere in the record is a resolvable citation rather than a string in a field.

The load-bearing part is not the file format. It is that every rule the old schema stated in prose now has something that fails when it is broken — inert-status for a status column carrying no information, unresolved-codes for a citation that resolves to nothing, retired-citations for a live practice sourced from an atticked paper, tag_groups for one-primary-topic.

Alternatives considered

  • Write validators against the existing YAML. The honest comparison, and it is most of the same work: a resolver for arXiv ids, a checker for superseded_by targets, a taxonomy validator, a status audit. What it does not get is the generated views, the citation graph across schemes, or a format where a paper’s note can be prose. The YAML’s real limit is that key_takeaways and attic.reason are editorial writing crammed into list items, and no amount of validation fixes that.
  • Keep the registry and add a docs site. Treats the symptom. The reason REGISTRY.md is thin is that its source is thin.
  • Status quo. Costs nothing today and keeps costing what it has been costing: five consecutive commits named “Update ML training registry”, a status field that has never once distinguished two records, and an attic whose cross-references have been broken for an unknown length of time.

Consequences

238 documents where there were two files, which is a real cost: the corpus is no longer greppable in one pass, and a bulk edit is now a script rather than a search-and-replace. The migration script stays in the tree (src/scripts/migration/to_record.py) so that the transform is auditable rather than archaeological.

data/registry.yaml keeps working, unchanged, for as long as the frontend

needs it. Inverting it into a generated projection is ADR-006, deliberately not part of this change.

The lint is warn-first: adopting this does not turn the backlog it finds into a red build. [luria.lint] fail_on is empty, and promoting classes into it happens once their findings are worked through.