The objection
CLAIM-058 holds that context-conditioned language models are a setting in which the theory’s effects “can be tested before human studies”. It is defeated only if “Framing and order effects found in models fail to replicate, even in direction, in matched human studies”. So it survives any human result that has the same sign as the model’s.
The bar is low. Agreement in direction is what a model trained on human writing would show whatever its mechanism. Such a model has read a great deal of text in which people describe, enact and react to teasing, reproach and shifts of footing. ARG-008 already grants that “An LLM’s context-conditioning behavior need not correspond to human inference over communicative situations.” If that is so, matching direction is weak evidence that the model tests anything about the theory’s claims on human interpretation.
The founding stimuli are famous. Villon, Henley’s rendering of Villon and Hofstadter’s translations of Marot are widely published and discussed. They are likely in the training data, together with commentary on them. A model’s reading of their footing may then be recall of what has been written about them, not interpretation. No entry in claims, cases or questions raises training-data contamination. The word appears only as cross-condition contamination within one conversation (CLAIM-085, CASE-032).
One model is not enough. Lu et al. (LIT-833), as CLAIM-029 reads them, find that order alone moves accuracy from near chance to near the state of the art, and that good orders do not transfer across model sizes. That is order sensitivity, not a reversal of a pragmatic effect, but it shows that what a context does to one model need not be what it does to another. A directional agreement found in one model may not hold in the next.
What is already documented: CLAIM-085 grants that a framing effect in one model “establishes a property of that model under those prompts”, and ARG-008 does not extend “to evidence about human pragmatics”. This objection adds that CLAIM-058’s own defeat condition is too weak a test of validity, and the contamination point.
What it does not say
- It does not say language models are useless as a test bed. Control of frames and orders is real (ARG-008).
- It does not say a directional match is no evidence, only that it is weak.
What would answer it
- A defeat condition on magnitude or on the pattern across conditions, not only on sign.
- Stimuli held out from training: new texts, or texts written after the models’ training cut-off, alongside the famous ones.
- Agreement required across several model families and sizes.
Note of 2026-10-28: split under ADR-061
“Matching direction is weak evidence that the model tests anything about the theory’s claims on human interpretation” is a verdict on the inference from language models to a test bed for the theory, which is ARG-008. Under ADR-061’s one ARG per inference it goes into that argument’s version 2, as an answer to “what does not transfer?”. The point about the defeat condition was answered by CLAIM-058 v2. This claim keeps what it adds about the subject matter: the founding stimuli and commentary on them are likely in training data, and context effects need not carry from one model to the next. It undercuts ARG-008 with that. The text above is unchanged.