nucleation

CLAIM-013: Existing text-to-image benchmarks evaluate object presence, attributes and whether spatial or interactive relations are depicted, not the stance, social alignment or uptake an image conveys, and need extending to pragmatically consequential relations and social uptake

2 min read

Properties10
StatusProposed
Version2
Rolethesis
Warrantinterpretive
Tagscompositionality, philosophy-of-language
Linepragmatic-transport
Workswhat-survives-translation
GroundsLIT-782 — GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment, LIT-783 — T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation, THEORY-522 — Open text-to-image diffusion models of 2022–2023 render a single named object almost always but fail most on relative position and next on attribute binding, by human rating and by detector checks alike; scaling the image model or continuing pretraining does not fix position, and interactions are rated near ceiling only by rubrics that ask whether an action is depicted
Objected byCLAIM-097 — T2I-CompBench is not purely object-centered: its non-spatial relation category already probes interactions, but scores them with CLIPScore and near human ceiling
Illustrated byCASE-004 — Two people, an empty wallet: shared complicity, normative authority, and an underspecified relation

Graph View

The claim

Case III holds people, objects and actions fixed and varies social alignment and communicative purpose, then compares text→image→caption chains with text-only chains. GenEval and T2I-CompBench are named as the object-centered baselines the proposal extends.

What it does not say

Both readings (NOTE-593 for GenEval, NOTE-592 for T2I-CompBench) support the premise behind the proposal: embedding similarity (CLIPScore) tracks human judgements of counting, position and binding worse than detector-based or per-pair checks.

Since the journal version

T2I-CompBench++ (LIT-798) adds numeracy and 3D spatial relations, and nothing social or pragmatic. The gap this claim names is still open.

Fu et al. (LIT-938, read in NOTE-813; THEORY-250) qualify it as CLAIM-097 does. Theirs is a relational benchmark scored by human raters, including agentic relations (helping, hindering, chasing), so not every existing benchmark is object-centered. It does not reach the pragmatic consequence or social uptake this claim asks for.

Note of 2026-10-29: narrowed

The manuscript’s §10 calls GenEval and T2I-CompBench object-centred. That holds of GenEval and of T2I-CompBench’s attribute and spatial categories. It does not hold of its non-spatial category, whose rubric asks only whether “actions, events and relationships” are portrayed (THEORY-522) and whose human scores sit at 0.95–0.99, nor of Fu et al.’s agentic relations. The gap the claim names is narrower, and it is still open.

Note of 2026-10-29: what holds it back

The claim generalises over “existing benchmarks” from the four the record has read (GenEval, T2I-CompBench and its journal version, Fu et al.), and no survey in the record meets or rules out its defeat condition. Scoped to the benchmarks the record has read, it would be a candidate for Active.

Backlinks