The objection
CASE-040 is the design A218 “names as the first experiment to run”. It crosses topic (spending, relationships) with act (teasing, reprimand), and CLAIM-129 is defeated if “On held-out, correlation-breaking items of a design that crosses topic with act (CASE-040), a topic-only factorization of lexical co-occurrence recovers the act distinction as well as factorizations fitted to interpretive-judgement or uptake data do.”
The act is assigned. CASE-040 “makes the act a manipulated factor rather than an outcome”, and “every cell is an example label, not a stimulus”. The items do not yet exist, and whoever writes them decides which are teasing and which are reprimand. A factorization of judges’ responses that recovers the act then recovers the writer’s intent as the judges read it. That is close to a manipulation check: it shows the items were written as intended, not that the act is a structure the judgements reveal and the words hide.
The baseline is built to fail. The critical items are held out and “break the familiar correlation”: “test whether the model correctly distinguishes a joking use of admonitory vocabulary from a serious reprimand.” A joking item written with admonitory words has, by design, the vocabulary of a reprimand. A topic-only factorization of lexical co-occurrence sees only the vocabulary, so on these items it must assign the act the words suggest, which is the wrong one. CLAIM-129’s defeat condition can then scarcely be met, and a pass shows that the items were built as intended.
The baseline is weak. It is “topic-only NMF”. CLAIM-139’s own defeat condition names “topic-factor, embedding or surface similarity”, and A218’s Chapter 15, as CLAIM-078 quotes it, asks for prediction “beyond standard lexical, embedding, and factorization baselines”. A supervised classifier of the text, or a language model, reads wording in context rather than co-occurrence counts, and is the rival a critic would field on “a joking use of admonitory vocabulary”.
What it reaches. CASE-040 illustrates CLAIM-129 and CLAIM-135, and CLAIM-135 is a premise of CLAIM-139’s epistemic clause. So the only planned evidence for that clause is a design whose outcome is fixed by its construction.
CASE-040’s “What it cannot show” names two limits, that it has not been run and that its interventions manipulate a described speaker (CLAIM-127, CLAIM-130). It does not name these.
What it does not say
- It does not say the design is useless. As a first pilot it shows whether judges and models can be made to separate topic from act at all.
- It does not say CLAIM-129 is false. A topic is plausibly not an act. It says this design cannot be the evidence for it.
What would answer it
- Acts fixed independently of the item writers: naturally occurring items whose act is labelled by raters who did not write them, or by observed uptake.
- Critical items whose vocabulary is not chosen to mislead the baseline, or a report of how the baseline does on items where the lexical cue is neutral.
- A baseline at least as strong as a supervised text classifier or a language model, alongside topic-only NMF.
Note of 2026-10-26: the same lesson for latent-structure recovery
The lesson in general form recurred three times in the Northstar turns of the exchange “Experimental Infrastructure Review”, about latent-structure models of an organization’s members (QUESTION-276). T21: “the generator’s vocabulary would risk making the intended topic structure artificially easy to recover.” T22: “If we start generating employees from the IBP and then successfully recover the IBP structure, we’ve mostly demonstrated that inference works under matched assumptions.” T23 gave the remedy this claim’s “What would answer it” gives for items: fit the latent-feature model to a mechanistic simulator that does not assume it (“Option A”, so that “feature discovery can be tested against a world that did not itself assume IBP-distributed employees”), and keep planted-structure recovery as a check on inference only (CASE-165). Recovering a structure from data generated under its own prior tests the inference, not the structure.
The hLDA paper the exchange cited, read the same day, has the published form of that check (THEORY-1010): on ten corpora simulated from three-level hLDA itself, the sampler’s best state recovered the generating tree exactly in eight, and in one of the others a wrong tree had the higher posterior. It shows the inference works under matched assumptions, and the paper offers it as no more. The claim is unchanged.
Note of 2026-10-28: moved to ARG-011
Under ADR-061, “a win on this test would not support the thesis” is the argument from the CASE that describes the design to the thesis. This claim is now ARG-011, from CASE-040 to CLAIM-129, form evidence, Rejected, and the failing critical question is “would the evidence be expected if the hypothesis were false?”. Its note of 2026-10-26 is carried into that argument’s body.