Contribution
It applies Shanahan’s 2016 “conscious exotica” programme, a later-Wittgenstein reading of consciousness language, to LLM agents understood as role-playing simulacra. It adds a test for candidature (can we engineer an encounter?), a verdict on current systems (disembodied chat agents fail it), and a scenario in which virtually embodied simulacra pass it and push the language into new shapes. For the self it adds two short claims (§3, §7.3), not an account.
Key insight
Ask how a word would come to be used, not what hidden fact it names. For consciousness that means: put the entity in a world with us, let people be with it and study its mechanism, and see what the community comes to say. Nothing further is hidden. Disembodiment is therefore disqualifying, not because experience needs a body but because the use of the words does.
Assumptions
- A Wittgensteinian method, summarised rather than defended (§4, fn 8: “no attempt here to offer a definitive interpretation”). Its central step is the private-language conclusion that “a nothing would serve as well as a something about which nothing can be said” (PI §304).
- All theories of mind-body relation are “dualistic”, behaviourism, functionalism and identity theory included, because they “posit two metaphysical categories” (§4). This is asserted.
- Embodiment is the home of consciousness language: shared perception, joint action and purposeful movement (§6.1).
- The role-play account of LLM agents, taken from the 2023 paper and Janus (§3).
- Scope: LLM agents of 2024, from simple chat agents to an imagined virtually embodied agent trained on text, vision and action tokens (§7.1).
Key results
- Candidature by encounter (§§5.2–5.3): the ability to engineer an encounter, “even if only in principle, establishes an exotic entity’s candidature”. Illustrated by the octopus and an imagined alien cube containing a simulated world and agent.
- Inscrutability (§5.4): no entity has a high capacity for consciousness and very low human-likeness; that region, the void of inscrutability, is empty.
- Chat agents (§6.1): “it is not possible to engineer an encounter with a simple conversational agent, even in principle”, since we “cannot be with them in a shared world”. Using consciousness words of them either bends the words out of shape or clings to dualism.
- Enhanced agents (§6.2): tools, multimodality, memory and planning legitimise belief and intention talk, but “we still cannot engineer an encounter” and “nothing has changed” for consciousness.
- Embodied agents (§§6.2–6.4): robots and virtually embodied agents can be encountered and so are candidates; edge cases (locked-in patients, the foetus, a phone-borne agent) are left to the community.
- Selfhood (§3): “There is nothing that would qualify as a self worth preserving for such an agent. It has no body, no personal history, and no autobiographical memory.” Extending lifetime and persistent memory “might legitimise talk of needs, desires, and selfhood”.
- No person behind the mask (§7.3): humans have “some kind of stable self at the core”, generative agents do not; their identity is the distribution of roles they concurrently play.
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | Consciousness words have meaning only through what is public (behaviour, bodies, mechanisms) | moderate as exegesis, assumed as premise | summary of PI §§256–271, 304; not argued against rivals |
| C2 | An entity is a candidate for consciousness only if we can, at least in principle, engineer an encounter with it in a shared world | weak | stipulated as the paper’s stance (§5.2), illustrated by the octopus and the alien cube |
| C3 | Disembodied chat agents, even with tools and memory, are not candidates | moderate given C2 | follows from C2 and the fact that they share no world with us (§§6.1–6.2) |
| C4 | Tool use and world access make literal belief attribution increasingly legitimate | weak | informal argument from answerability to the world, following Shanahan 2024 (§3) |
| C5 | A chat agent has no self worth preserving, lacking body, personal history and autobiographical memory | weak | assertion (§3); the three conditions are not defended as necessary |
| C6 | Humans have a stable core self behind their personas; generative agents do not | weak | ”we take it for granted” (§7.3); the human half is assumed |
| C7 | There is no fact about consciousness beyond what an informed community converges on, and where it does not converge, nothing more to say | weak | follows from C1; the paper concedes this is “hard to reconcile” with holding a minority view (§5.5) |
| C8 | Embodied simulacra could come to be spoken of as conscious, with no residual difficulty in their being “mere” mimics | moderate given C1, C7 | scenario (§§7.1–7.2), modelled on the perfect-actor argument |
Method
Conceptual: a method taken from Wittgenstein, applied to a graded series of cases (octopus, alien cube, chat agent, enhanced agent, robot, virtually embodied agent) and one extended scenario. The paper calls its aim to describe “without judgement, the language games of imagined communities” (§8).
Concepts
- Conscious exotica: entities very unlike any terrestrial organism that nonetheless belong to “the fellowship of conscious beings”.
- Engineering an encounter: putting ourselves where we can meaningfully interact with an entity “given the purpose we discern in it” (§5.2).
- Void of inscrutability: the empty region of the space of possible minds where human-likeness is very low and capacity for consciousness above zero (§5.4).
- Garland test: a human ascribes consciousness to an AI while knowing it is an artefact (fn 16).
- Perfect actor: Putnam’s case, here made “(almost) real” by an embodied simulacrum (fn 22).
Connections
It builds on Shanahan 2016 (“Conscious exotica”), Shanahan, McDonell and Reynolds 2023 (LIT-460) and Janus. It cites Putnam 1963 and 1964 as near allies. It names Nagel (LIT-096) and Chalmers 1996 as canonical statements of the dualist intuition. Chalmers 2023 (LIT-411) and Butlin et al. (LIT-056) are named as the rival, evidential method. Seth (LIT-135) is quoted on consciousness and living systems (fn 19). Birch’s work on animal sentience is cited for the octopus case, not his AI manifesto (LIT-111).
On the self, its claims meet the record’s works as follows. Dennett (LIT-442) denies the stable core self that §7.3 takes for granted. Goldstein and Lederman (LIT-206) give instances desires on behavioural grounds that §3 withholds. Minsky (LIT-432) puts the self in reflective self-models, which §3’s conditions do not mention. Hofstadter (LIT-416, unread) is reported to hold a conversational criterion that §6.1 rejects. None of these is cited; the comparisons are mine.
Bearing on the record
- THEORY-023: a challenge to its presupposition, not evidence for or against it. The account treats AI consciousness as an open question current evidence cannot settle. This paper says the question has no answer beyond an eventual consensus (C7), and that for chat agents it is answered now, negatively, by the grammar of the word (C3). If C1 is granted, THEORY-023’s deadlock is not epistemic. The account should note this as a position that dissolves the dispute rather than taking a side in it. The promotion condition is untouched: the paper offers no evidence of the kind it asks for.
- For a THEORY on LLM selfhood: it supplies one candidate condition (body, personal history, autobiographical memory) and the “no core” claim about role players. Both are asserted, so neither could source a THEORY alone.
- No ML instruction. The ethics section warns against letting consciousness language “wander too far”; it is not practice.
Limitations
- C2 does all the work and is stipulated. A reader who does not share the Wittgensteinian method gets no argument against pure-thinker or disembodied-mind views; Chalmers’s is named (fn 18) and set aside.
- Calling functionalism and identity theory “dualistic” is a definition wide enough to make the rival positions one; it is not argued.
- The self passages are two paragraphs. The three conditions for a self worth preserving are listed, not defended, and §7.3’s human “stable self at the core” contradicts the Humean and Dennettian views the author adopts a year later (LIT-466, §6.1 fn 41).
- The embodied-simulacrum scenario is imagined, and the outcomes are listed, not weighed.
Open questions
- Would persistent memory and a long lifetime (§3) make a chat agent a self without making it a candidate for consciousness? The paper allows the first and denies the second, so its notions of self and consciousness come apart; it does not say how.
- Does a phone-borne or headset agent that shares a perceptual scene with a user count as encounterable (§6.4)? The paper leaves it to the community.
Corrections
- Venue and date. The brief’s “Inquiry (2024)” is right for first appearance and online publication (1 December 2024). The journal issue is 69(6), 2026, pp. 2982–3010 (Crossref).
- The paper does not argue that LLMs lack a self from the role-play view alone. §3 grounds the denial in missing body, history and memory, and allows that enhancements could change it.