ADR-098: One config file, one format, vocabularies declared once

Context

Three formats for one system:

luria.toml            TOML   the config
tags.yaml             YAML   per scheme, beside the records
statuses.yaml         YAML   per scheme, beside the records
<vocabulary>.yaml     YAML   per scheme, declared by name ([ADR-076](ADR-076.md))
remotes.lock.json     JSON   generated, committed

Every document’s frontmatter is already YAML. The config was TOML for no reason either format could state, which is one more set of quoting rules to know and one more parser to reason about when a regex in a uid does not mean what it looks like.

The vocabulary split was worse than untidy. A scheme could only name a file beside its own records, so a vocabulary two schemes share had to be two files. Measured across the two records running on luria at the time of writing:

• Four byte-identical copies of one statuses.yaml — ADR and DP in this repository, and the same pair again in anthology-of-the-sota.
• That record’s ADR-026 decided its practice registry and reading list share one topic vocabulary, and says so in its CLAUDE.md. On disk it was two files with the same thirteen keys and ten of thirteen blurbs different. Two edits to one vocabulary, months apart, that nothing could reconcile because nothing knew they were the same vocabulary.

This project already knew. primary_tags’ own docstring records the identical failure from the other direction: “seven terms across four places, and the blurbs for the same tag already disagreed between two of them (ADR-060)”. ADR-060 fixed the copies within a scheme. The copies between schemes were unreachable, because the config had no way to say “these are the same words”.

Decision

One luria.yaml. TOML is gone rather than deprecated — see the alternatives.

A central vocabularies: table, referenced by name:

vocabularies:
  statuses:
    Active: {label: Current, blurb: in force}

schemes:
  ADR:
    fields: {status: {vocabulary: statuses}}
  DP:
    fields: {status: {vocabulary: statuses}}   # said once

One place names it. status is a field, so fields.status.vocabulary is where its vocabulary is named and there is no second schemes.X.statuses: beside it. This change shipped with both, and they were not merely redundant: statuses.declared read the scheme key while statuses.undeclared read the field, so a scheme could render a status legend and be reported as having no status check at the same time. What stays privileged is active: — WHICH word means in force, the role the whole citation apparatus rests on — and that names a word, not a vocabulary. tags: stays a scheme-level key for the opposite reason: it is the one axis the code still assumes, with no fields. spelling to collide with.

A vocabulary’s per-value pages render at <view>/<name>/, so its name is part of a published path. Luria’s own shared vocabulary is called statuses rather than something more descriptive for exactly that reason: renaming it would move docs/decisions/statuses/ and leave the old directory behind as an orphan no generator claims.

Scheme.tags_yaml and Scheme.statuses_yaml — paths — become Scheme.tags and Scheme.statuses, which return values. Vocabulary.file becomes Vocabulary.values_by_name. Nothing downstream opens a vocabulary file because there is no longer one to open.

omegaconf goes underneath the validation, not in place of it. OmegaConf.merge replaces the hand-rolled _merge over DEFAULTS, and the schema types the result. The 44 raise ValueErrors stay, and should: they are cross-field semantic rules — a converse must be mutual and point back at the declaring scheme, a derivation’s from must name a real reference, cite = "view" needs render = "document" — and a structured config validates the shape of a value, not a relationship between two.

label has one fallback. It had three — tag.title() in the tag pages, "" in the status legend, the raw value in the vocabulary pages — so a scheme declaring no label rendered a title-cased tag heading and an empty legend cell. vocabularies.label_of is the one answer.

Reference gains label and blurb (#254), the two keys a vocabulary value already carried. A relation’s meaning lived in a TOML comment, which nothing could render, quote in a finding, or scaffold from — which is how a record ended up with fourteen practices citing adoption as evidence while every mechanical check stayed green.

Two YAML libraries, and each has one job. omegaconf composes a config: OmegaConf.merge folds a project’s file onto the defaults and the schema types the result. ruamel edits one: luria init adds a scheme, luria upgrade wires up a field, luria migrate renames one, and all three rewrite a file somebody wrote and has to keep reading.

Neither library does the other’s job, and the failure in each direction is silent. OmegaConf.merge drops every comment — right for building a config out of defaults, wrong for editing one on disk, where the comments are most of what the file is. Editing the text instead keeps the comments and gets the structure wrong: TOML tables concatenate, so a [schemes.RFC] appended to a file belonged to schemes wherever it landed, while YAML nests by indentation, so the same block appended joins whichever top-level key happens to be last — and RFC: occurs under schemes: and under every remotes.<R>.schemes:, so a sweep cannot tell one from the other either. This change was written the second way first, and all four sites had the bug.

luria/yaml_edit.py is the one place that knows how a config is edited: ruamel in round-trip mode, four operations, comments intact and the structure addressed by path. It is also the only emitter — the TOML converter writes through it too, so a converted config starts in the shape every later edit produces and a one-key change never reads as a rewrite.

tags is a field too, and the axis is named rather than assumed. This was the last thing the code knew by name, and it held out for two stated reasons:

tags stays: it is OPEN, and a vocabulary is closed by construction (ADR-054 deferred even a closed flag), and its tag_groups constrain a subset of values, which a vocabulary cannot express.

Both are true, and both are things a declaration can say once it is allowed to. Vocabulary gains closed (default true; tags sets it false — the values a project has an opinion about supply order, label and blurb, and reaching for a new one stays an edit to a document). tag_groups moves from schemes.X.tag_groups to fields.<field>.groups, which is where it always belonged: a group constrains a subset of ONE field’s values, and the scheme-level table could only ever have meant tags. The check read meta["tags"] whatever the group was about, so a group on any other field saw nothing and passed every document — a rule that looked enforced.

What is left is one key:

schemes:
  SCENE:
    axis: worlds        # which field heads this scheme's index
    fields:
      worlds: {vocabulary: worlds, many: true, closed: false}

axis is not tags renamed. It says which of this scheme’s own fields gets the categories block and a page per value — a rendering choice about a scheme, and one a world-bible answers differently from an anthology. A scheme naming none has no taxonomy and renders none, which the old code could not express: every scheme had tags, and every document was told to carry them.

BUILT_IN_AXES is gone, BUILT_IN_CONDITION_FIELDS is ("status",) — status alone, and only because it has a default vocabulary no scheme has to declare — and the lint’s “no tags: in frontmatter” branch is required: true on the axis field, checked with every other obligation.

Two smaller things went with it. many: true now types a field on its own: it says the field holds a list, which is what makes it nameable in a chain or a derivation, and that is exactly what being built in used to say about tags. And axis values are no longer lower-cased on the way in — a tags convention the code applied to every value, which disagreed with the vocabulary check beside it that has always compared the value as written.

One renderer for every field a view groups by. With tags a declared field, the two renderers wrote into the same directory — so they could not both stay. vocabularies.py renders all of them; tag_order, render_categories and render_tag_page are gone.

They had drifted, which is the argument for collapsing rather than choosing. Three ways:

• The label fallback. Already one answer since vocabularies.label_of, but the tag page reached it and the value page reached it differently.
• The blurb. The tag page sentence-cased it with a hand-rolled raw[:1].upper() — str.capitalize() having silently destroyed a blurb running to more than one sentence — while the value page printed **Label** — blurb. One template now: **Label** — blurb.
• A value the vocabulary does not declare. The axis showed it, every other field dropped it. Neither was reasoning about it; closed is, and a field naming no vocabulary is open for the plainer reason that there is no closed set to fall outside of.

What is left of the difference is a shape the scheme chooses, not one the code knows: the axis lists the documents under each value because it is the browsing surface, and every other field is a row of chips because the value’s own page already holds the table.

Scheme.grouped_fields is the one answer to which fields those are — three places needed it (which directories the generator owns, which are exempt from the docs index, which paths count as generated) and a fourth disagreeing with them is how a page becomes an orphan.

This changes published pages, once. No path moves. A value page’s heading goes from # ADRs tagged \record`to# ADRs with `tags` `record`, its blurb gains the label, and a declared value nobody uses now gets a row reading (0)` and a page of its own, where the axis used to omit it and every other field already showed it.

A vocabulary-backed field’s pages render under the FIELD’s name, not the vocabulary’s. That was the last thing to go wrong here and the least obvious: pages rendered at <view>/<vocabulary>/, so sharing a vocabulary between two schemes — the entire point of this decision — moved published pages and left the old directory behind as an orphan. What that broke said nothing about paths: the bare-reference check skips generated views and recognises them by generated-output path, so an orphan is not one, and pages generated for a year started being scanned as hand-written prose.

A vocabulary’s name is a config detail; a published path is not. Keyed on the field, no config change can ever move a page. This moves existing pages once — <view>/statuses/ becomes <view>/status/ — which is the right place to spend it: at a format boundary a record crosses deliberately, rather than in a later release where it would surprise someone.

Alternatives considered

• Keep vocabularies beside the records. The locality argument: a scheme’s vocabulary is the one piece of config a person editing that directory actually reads. It was mine, it was speculation about a user, and the user reported the opposite — the files were hard to locate. The drift evidence above settles the rest.

• Central by default, with a per-scheme path override. The compromise, and close to what tags_file already was. Rejected because the override is the thing that permits the drift: two schemes can point at one file today and simply do not. A mechanism nobody reaches for is not a mechanism.

• Deprecate TOML over a release or two, dual-reading both. The careful path, and normally right. Rejected on instruction — and the cost is real and bounded: two known records, both converted in this change.

• Let structured configs replace the hand-written validation. Tempting, since it looks like the point of adopting them. It would trade luria’s errors — “a converse names the field holding the same relation read backwards, and it has to exist to be written into (declared: …)” — for a schema complaint about a key of the wrong type, at exactly the moment a user is confused. The plumbing was worth replacing; the checks are the part that earns its keep.

• Key the view path on the vocabulary’s name, and have the converter keep the old stem. It would move nothing in the common case. It fails exactly where this decision is aimed: a record whose five schemes each have a statuses.yaml with different words cannot keep all five on the name statuses, so four move anyway — and the rule “your pages move unless your vocabularies happen not to collide” is not one anybody can hold.

• Edit configs with omegaconf, since it is already a dependency. One library instead of two, and it composes correctly. Rejected because it drops comments, and a command that silently deletes a person’s notes from their own config has taken more than it was asked for. The two libraries are not redundant; they answer different questions.

• Edit configs by rewriting the lines, to keep the comments. What this change did first, in four places. It keeps the comments and puts the block in the wrong mapping, which is worse: a lost comment is visible in the diff, and a scheme that quietly became a journal is not.

• Leave tags built in and generalize the rest. Where this change stopped the first time, on the two reasons quoted above. Rejected once they turned out to be missing features rather than facts: closed was deferred, not impossible, and a group naming its own field is strictly more expressive than one that cannot. The tell was the group check reading meta["tags"] — a rule that could only be written for one field was silently wrong for any other.

• Derive the axis instead of naming it (the one open, many-valued field). It works until a scheme has two, and then it picks one without saying so. A rendering decision that a reader cannot find in the config is the failure this whole decision is about.

• JSON for the lockfile stays. It is generated, not authored, and machine round-tripping is the only thing it is for. Unifying it would be consistency for its own sake.

Consequences

luria lint runs end to end on this repository under the new config, and ADR and DP now share one status vocabulary declared once — the duplication that motivated this is gone from the record making the decision.

This is a breaking change with no migration path in-tree. Every existing record has a luria.toml and per-scheme vocabulary files. A converter is owed, and the hazard it has to own is escaping: uid = "(\\d{4})[.:](\\d{4,5})" does not survive TOML → YAML by copying bytes, and that is the kind of thing that converts silently and breaks at runtime. Round-tripping every uid and title_re in a real record is the test, not a fixture.

primary_for remains the right mechanism and gets better: it was a way for one file to say which schemes a tag is primary for, and it now says that inside the vocabulary every one of those schemes actually shares.