The claim
Circularity, A45 §9: “I would explicitly avoid a circular experimental design in which we: 1. Generate translations with an LLM. 2. Ask the same LLM whether they preserve pragmatic meaning. 3. Declare its preferred translations more faithful. That would primarily characterize the model’s own preferences.” The remedy: “different models for generation and evaluation, independently constructed framing conditions, calibrated response measurements, and held-out human assessments”. And: “The existence of a systematic framing effect in one LLM establishes a property of that model under those prompts. Generalization to human pragmatics requires additional evidence.”
Normative versus descriptive, A45 §8: the paper’s two goals are “a normative theory of translation fidelity and a descriptive theory of how interpreters construct communicative significance. LLMs can help test the second. Human communicative practice remains essential for justifying the first.” A48 §8.5: “model response distributions describe interpreters, not automatically normative translation quality”.
Where it went
A48 kept both, but neither survived A52. U19 asked to “generally withhold hedging” (CLAIM-095), and these went in the same sweep, apparently by inadvertence: A49 itself said the rule did not mean “concealing limitations”. The manuscript keeps parts of the remedy: independent interpreters, split annotators and held-out human judgements (§10), “heterogeneous-agent chains” (Case II). It states neither the circularity nor the normative/descriptive split; its §11 says the choice of observables is “normative and empirically contestable”, which is the nearest it comes.
The circularity limb returned at U26, when the owner told the assistant it was itself a text-generation action and should delegate. A68: “generating multiple responses within this conversation doesn’t give us independently isolated inference calls … That introduces potential experimenter bias and cross-condition contamination.” C6 Appendix F then made it a rule: “Do not use assistant-authored examples as independent LLM samples or the same generative model’s preferences as an unblinded ground truth” (CLAIM-028). The manuscript keeps the remedy, not the rule.
Note of 2026-10-28: split under ADR-061
The circularity limb, that generating and judging with one model “would primarily characterize the model’s own preferences”, says what a design does not show, so under ADR-061 it is the verdict of ARG-008, recorded in that argument’s version 2 with the remedy of separate generator and evaluator. This claim keeps two points about the subject matter: a framing effect in one model is a property of that model under those prompts, and model response distributions describe interpreters, so they cannot settle what a translation ought to preserve. Every claim that rests on it uses one of those. The quoted text above is unchanged.