Found on 2026-10-10 by a search the owner asked for: documented tests of contextuality proper on elicited pragmatic or meaning judgements, for CLAIM-201, whose concession rests on NOTE-235’s “none” and on one paper’s data sets.
The case
The conditions a study must meet to bear on the fixed-scenario form of CLAIM-105 are four: (a) elicited judgements, not corpus counts; (b) no signalling within tolerance, or contextuality after a Contextuality-by-Default (CbD) correction; (c) enough observations per context; (d) pragmatic or meaning content, not perception. Four bodies of data were checked against them.
The held null: Dzhafarov, Zhang and Kujala (2016), LIT-264. Read in NOTE-235. Under CbD, none of the re-analysed data sets is contextual: 73 question-order pairs, mostly Pew and Gallup split ballots of several hundred respondents per order; Bruza et al.’s 23 primed word combinations, with ΔC from −2.882 to −0.418; conjoint choices; psychophysics. It meets (a), (b) and (c). For (d) its content is attitudinal and lexical, not pragmatic in the sense of a speaker, an act and its uptake.
A Bell-type null on meaning judgements: Wang’s thesis (2024). D. Wang, A Quantum-Inspired Analysis of Human Disambiguation Processes, UCL thesis, arXiv:2408.07402. 400 noun–verb phrases were rated by 25 MTurk workers per batch on a 0–7 plausibility scale for each meaning combination. The rank-4 result, p. 105: “none of the empirical models obtained in the corpus or human judgment dataset were contextual (in either framework)”. Among the rank-2 systems, ten are listed as CbD-contextual (Fig. 3.4b, from 0.232 for file/admit down to 0.001). No significance test or bootstrap is given for any of them, and the “probabilities” are averaged normalised ratings, not frequencies of outcomes (§3.1.2), so CbD’s sampling reading does not plainly apply. Lo, Sadrzadeh and Mansfield (2024, arXiv:2402.04505, §1) report “4.5% of the phrases (13 out of 290)” as contextual, attributing human judgements to Wang et al., the papers the record holds as corpus studies (LIT-842, LIT-851). The two counts are not reconciled here.
The strongest positive, off content: Cervantes and Dzhafarov (2018), “Snow Queen”. Decision 5(3): 193–204, arXiv:1711.00418. About 450 MTurk respondents per context, four contexts, each choosing a character from Andersen’s The Snow Queen and a characteristic, under an instruction that the pair match the story. The marginals differ by context (P(Kind) .841 against .374 in the correct-response system), yet the CbD criterion is violated. p. 6: “the left-hand side of Inequality 2 equals 0.452 … the 99.99% bootstrap confidence interval … is [0.226, 0.668]”. p. 9, of the working hypothesis that behavioural context-dependence is direct influence only: “This hypothesis is now falsified.” Basieva, Cervantes, Dzhafarov and Khrennikov (2019, J. Exp. Psychol.: General 148(11): 1925–1937, arXiv:1807.05684) found five of six further instructed-choice systems contextual. Yearsley and Halliwell (2019, arXiv:1905.12570) reply that “it is … possible to construct a non-contextual model which explains the experimental data via direct influences”, and they “cast doubt on the idea that any experimental data in psychology could provide conclusive evidence for contextuality”. No reply to the comment was found (one search).
The right content, underpowered: Lo, Sadrzadeh and Mansfield (2023). Their generalised Winograd schema, EPTCS 384: 187–202, doi:10.4204/EPTCS.384.11, arXiv:2308.16498. A story with two ambiguous pronouns, each resolved under one of two words: “A and B belong to the same [cannibalistic / herbivorous]1 species of animal. … After a while, one of them2 is no longer [hungry / alive]2.” Coreference from world knowledge is meaning content of the pragmatic kind. Of 410 MTurk responses 348 were valid, about 87 per context. §4.4: “The model violates the Bell-CHSH inequality by 0.192 with a standard deviation of 0.176”, and “87% of them exhibited a positive violation”. No signalling holds by construction: “all symmetric models are non-signalling … By considering only symmetric models, we thereby avoid the complications of dealing with non-signalling models”, and responses that broke the symmetry were discarded. The authors also concede the story “contains several incorrect uses of English”, noticed after data collection.
What it can show
That CLAIM-201’s concession stands on the documented evidence, read wider than the one paper it cites. No published study of elicited pragmatic or meaning judgements finds contextuality proper at an adequate sample size. The one that meets all four conditions finds none (LIT-264). The one Bell-type test on meaning judgements finds none at rank 4 (Wang). The one test on pragmatic content finds a violation that cannot be told from zero (Lo et al.). So the fixed-scenario form of CLAIM-105, which needs a contextual model on one side, still has no case but a constructed one (CASE-037).
It also corrects the scope of the concession. “None” in behavioural data (NOTE-235) is out of date: since 2018, instructed choice tasks have been reported CbD-contextual with high reliability, though the verdict is disputed in print. The defensible sentence is “none in pragmatic or meaning data adequately tested”. The Snow Queen data are also a documented case for CLAIM-037: CbD finds contextuality in heavily signalling data, where a test that presupposes no signalling cannot be applied at all.
What it cannot show
- Condition (d) is met only partly, and (c) only where (d) fails. The adequately powered studies are attitudinal, lexical or instructed matching; the pragmatic one has about 87 responses per context. So the case documents an absence of adequate tests as much as a null. It does not show that pragmatic judgements are noncontextual. CLAIM-201 itself says it does not say contextual pragmatic data cannot exist.
- The strongest reply. A defender of CLAIM-105’s fixed-scenario form would say the null is uninformative where it matters. The one pragmatic test points the right way (87% of resamples positive), was run once, in faulty English, and its forced symmetry threw away the signalling that LIT-842 and LIT-851 find to be the norm in language. Wang’s ten rank-2 positives, untested, point the same way. On this reply the concession is premature: the right study has not been done. The reply is fair as to evidence, and the case does not refute it. It answers only that the claim’s concession was conditional on “the evidence the record holds”, and the wider evidence does not change it.
- The positives do not transfer. Snow Queen’s correlations are made by the instruction to match the story, not by what an utterance does; it gives CLAIM-105 no pragmatic contextual model. Whether its CbD verdict survives richer direct-influence models is disputed (Yearsley and Halliwell), and this case does not settle it.
- What would close the gap. A powered rerun of Lo et al.’s design, without forced symmetry and analysed under CbD, with pragmatic items (whose uptake, which act, which relation). As a rough figure, computed here from their reported standard deviation and not by them: a violation of 0.19 at three standard deviations needs about 0.064, so about seven to eight times their responses, some 650 valid per context. Such items could sit in the discriminating study (CASE-055) as its contextuality arm, run on source and rendering alike. A positive result there would reopen the fixed-scenario form; a powered null would make the concession evidence of absence.
Sources
Held in the record:
- Dzhafarov, Zhang and Kujala (2016), LIT-264, read in NOTE-235.
- Wang, Sadrzadeh, Abramsky and Cervantes (2021), LIT-842, read in NOTE-645; and its journal follow-up, LIT-851. Corpus data only.
Not held, cited by identifier:
- Cervantes, V. H. and Dzhafarov, E. N. (2018), “Snow Queen is Evil and Beautiful: Experimental Evidence for Probabilistic Contextuality in Human Choices”, Decision 5(3): 193–204, doi:10.1037/dec0000095, arXiv:1711.00418. Read directly in the arXiv v8 PDF.
- Basieva, I., Cervantes, V. H., Dzhafarov, E. N. and Khrennikov, A. (2019), J. Exp. Psychol.: General 148(11): 1925–1937, doi:10.1037/xge0000585, arXiv:1807.05684. Read directly (arXiv PDF).
- Yearsley, J. M. and Halliwell, J. J. (2019), “Contextuality in Human Decision Making in the Presence of Direct Influences: A Comment on Basieva et al. (2019)”, arXiv:1905.12570. Read directly; no journal version found.
- Lo, K. I., Sadrzadeh, M. and Mansfield, S. (2023), EPTCS 384: 187–202, doi:10.4204/EPTCS.384.11, arXiv:2308.16498. Read directly.
- Lo, K. I., Sadrzadeh, M. and Mansfield, S. (2024), “Developments in Sheaf-Theoretic Models of Natural Language Ambiguities”, EPTCS 408, arXiv:2402.04505. Read directly for the reanalysis (contextual fraction 0.096, same data) and the “13 out of 290” count.
- Wang, D. (2024), A Quantum-Inspired Analysis of Human Disambiguation Processes, UCL thesis, arXiv:2408.07402. Read directly, §§3.1–3.2 and p. 105.
The DOIs of Cervantes and Dzhafarov, Basieva et al. and Lo et al. 2023 were checked against Crossref by the search; the others were not.
Note of 2026-10-12: the null and the notion it is under
QUESTION-067, filed on main on 2026-10-12, separates contextuality over measurement contexts from Spekkens’s generalized notion, which also constrains preparations. It says preparation contextuality “can show even with a single binary judgement. A sheaf test cannot see it”. The tests this case reads are of the CbD and Bell type, and its finding that no contextual pragmatic data are documented holds under those notions. It says nothing about preparation contextuality, which none of the tests looks for. CLAIM-201’s concession and CLAIM-219’s “in no behavioural data set” (now held as CLAIM-325) are scoped the same way.