LIT-411: Could a Large Language Model be Conscious?

David J. Chalmers (2023), arXiv:2303.07103 (v1 4 March 2023, v3 18 August 2024). Published in Boston Review, 9 August 2023. An edited version of his invited talk at NeurIPS, 28 November 2022.

Key takeaways

• Method (§§2–3). A claim for or against LLM consciousness should name a feature X, give reasons that LLMs have or lack it, and give reasons that having or lacking it makes consciousness probable or improbable.
• For (§2). Self-report is fragile and trained on human talk about consciousness. Seeming conscious is cheap, as ELIZA showed. Conversational ability matters only as a sign of general intelligence. General intelligence gives “some limited reason to take the hypothesis seriously”.
• Against (§3). Six X’s: biology, senses and embodiment, world models and self models, recurrent processing, a global workspace, unified agency. The strongest are recurrence, workspace and agency. All but biology are “temporary rather than permanent”: each names a research programme, and LLM+ systems (multimodal, tool-using, embodied) may meet them “within the next decade or two”.
• Numbers (§4), “for illustrative purposes”. Give at least 1/3 credence to each of the six requirements. If they were independent, a system lacking all six would have under a 1/10 chance of consciousness, hence “somewhere under 10 percent” for current paradigmatic LLMs. For sophisticated LLM+ systems within a decade: over 50% that they will be built, times 50% that they would be conscious, gives “25 percent or more”.
• The theory-balanced approach (fn 30). Weigh several theories’ verdicts by credences, perhaps taken from expert surveys. Chalmers presents this as distinct from Birch’s theory-heavy, theory-neutral and theory-light approaches.
• Challenges (§4): four foundational (benchmarks, theory, interpretability, ethics), seven engineering, and a twelfth: “If that’s not enough for conscious AI: What’s missing?”. He offers them as a roadmap or as red flags. He is “not asserting that we should pursue this research program”.

Standing in the record

Filed on 2026-10-02 at the owner’s request, as one of the Chalmers works he asked for. NOTE-345 is the close reading. Active: it is the paper the AI-consciousness works in this record react to. Butlin et al. (LIT-056) build on it (NOTE-052), and Schwitzgebel (LIT-191, NOTE-104) reports its 25% figure.

How it sits against THEORY-023. THEORY-023 says current evidence cannot settle whether an AI system is conscious. Mimicry undercuts behavioural evidence, and architectural indicators presuppose the disputed computational functionalism. This paper agrees on the first half, is an example of the second, and adds one more voice to the contested prior.

• Behaviour: agrees, independently and earlier. Self-reports are “fragile” and the model “has learned to imitate those claims” from a corpus of people talking about consciousness, so “the evidence is much weaker”. That is the undercutting move of Birch’s gaming problem (LIT-111) and Schwitzgebel’s Mimicry Argument (LIT-191), made in 2022. Chalmers’s third challenge, an LLM that describes features of consciousness it was not trained on (after Schneider and Turner), is a candidate for the “theory-neutral marker … immune to training on human output” that THEORY-023 names as a refuter. He proposes it; he does not supply it.
• Architecture: an instance of the Janus problem, not a way out. Recurrence, workspace, self models and unified agency are all theory-derived requirements, and he treats building them into LLM+ systems as progress toward consciousness. That is the functionalist face of Birch’s Janus problem. The biological naturalist’s face, Seth’s (LIT-135), is set aside: Chalmers says he has “argued that these views involve a sort of biological chauvinism”, that “silicon is just as apt as carbon”, and then “I’ll set this issue aside”. He does give biology at least 1/3 credence and multiplies it in. So the numbers are not conditional on functionalism alone. But the multiplication is a credence under assumed theories, which THEORY-023’s promote_when says “cannot settle it”. The twelfth challenge, asking a doubter to name a missing X “and could that X be built into an AI system?”, frames the burden as a functionalist would. For Seth the missing X is being alive, which is not an engineering item.
• Priors: a fifth position. THEORY-023 lists Seth (unlikely), Birch (cannot rule out), Butlin et al. (no strong current candidate) and Schwitzgebel (we will not know). Chalmers is low for current LLMs (under 10%) and significant for LLM+ systems within a decade (25% or more). In fn 29 he says his own views “lean somewhat more to consciousness being widespread”, so he would go higher. His own term for the stance is “theory-balanced”.
• Where it falls short of the account’s bar. Nothing here is evidence that both parties accepted in advance. It is a structured credence exercise, and Chalmers says so (“specious precision”).

On THEORY-043. Two points only. First, a global workspace “gathering information from numerous non-conscious modules” is offered as a possible requirement. That is the subsystem-driven architecture Schwitzgebel used to answer Chalmers’s correspondence objection (NOTE-131), and here Chalmers does not press that objection against it. Second, Chalmers suggests that one LLM “can support an ecosystem of multiple agents”. That is the multiplicity of minds in one system that his 1996 implementation paper (LIT-404) allowed, here offered as a reply to the disunity objection. It is a passing remark, not an argument.

NOTE-131’s correspondence principle is not stated here.

Not the extended mind. “LLM+” and “extended large language models” mean models with added modalities, actions and tools. They are not the extended mind of LIT-097. Tool use (database queries, code execution) is the natural place where the two would meet, and the paper does not make the link. Among the core consciousness works filed alongside this one, the hard problem is named here only as the second foundational challenge (“we don’t understand consciousness. That’s a hard problem, as they say”).

Boundary. Not held in the anthology, so ADR-013 does not apply. Its engineering challenges (“build LLM+s with a global workspace”) are a research map offered equally as red flags, not an instruction for machine-learning practice, and no anthology topic holds AI consciousness. So it stays here and is not tagged anthology-candidate.