Source
Chalmers (2023), “Could a Large Language Model be Conscious?”, arXiv:2303.07103 (v3, the Boston Review text), from a NeurIPS 2022 invited talk, LIT-411, read in full in NOTE-345, with v1 compared for its conclusions.
What was actually shown
The form (§§2–3). A claim for or against consciousness in a language model should name a feature X, with reasons that models have or lack X and reasons that having or lacking it makes consciousness probable or improbable.
Evidence for (§2). Self-report is fragile: a one-word prompt change reverses it, and the model “has learned to imitate those claims” from a corpus of people talking about consciousness, so “the evidence is much weaker”. Seeming conscious is cheap, as ELIZA showed. Conversation matters only as a sign of general intelligence, which gives “some limited reason to take the hypothesis seriously”.
Evidence against (§3). Six features: biology, senses and embodiment, world and self models, recurrent processing, a global workspace and unified agency. Transformers are “almost entirely feedforward”; models are “chameleons” without stable goals; self models are “especially limited”. The strongest are recurrence, workspace and agency. “For all of these objections except perhaps biology … the objection is temporary rather than permanent”: each names a research programme, with simple systems that already have the feature (LSTMs and external memory, Perceiver-style modular interfaces, agent models trained on one individual).
The arithmetic (§4). Give at least 1/3 credence to each of the six being required. If independent, a system lacking all six has under a 1/10 chance of consciousness, since (2/3)⁶ ≈ 0.088; dependence pushes it up and unknown requirements down, giving “somewhere under 10 percent”. Over 50% that sophisticated extended models with these features exist within a decade, times at least 50% that they would be conscious, gives “25 percent or more”. The figures are “for illustrative purposes”, with “specious precision” conceded.
What could have come out otherwise. An objection other than biology could have turned out permanent: a feature no engineering programme could supply, or one whose simple instances clearly lack what the objection needs. The paper finds none, by surveying the features one at a time.
What this does not say
- That current models are conscious, or that their successors will be. It says the case against the successors is weaker. The second credence is the product of two stipulated 50% figures.
- That biology has been answered. It is set aside: Chalmers has “argued that these views involve a sort of biological chauvinism” and “I’ll set this issue aside” (C7, asserted here). It enters the numbers at 1/3, so the credences are not conditional on functionalism alone, but they are credences under assumed theories, which THEORY-023 says cannot settle the question.
- That the features are sufficient. The twelfth challenge, “If that’s not enough for conscious AI: What’s missing?”, frames the burden as a functionalist would. For a biological naturalist the missing feature is being alive, which is not an engineering item (THEORY-023).
- That the survey figures are credences. The theory-balanced approach (fn 30) reads acceptance of, or finding promising, global workspace, recurrence and higher-order theories in an expert survey as collective credence, and leaves the conversion to “further work”.
- That the empirical claims are current. They are dated 2022–23; the July 2023 Afterword already says GPT-4 shortens the timelines. v1 gave different weights (25% and 50%) and reached “at most a 5% or so chance” before adjustment (NOTE-345, Corrections).
Connections
- THEORY-023 (current evidence cannot settle AI consciousness). This paper makes that account’s behaviour argument independently, in 2022, and is an instance of its architecture half: the obstacles that do the work are theory-derived, and building them counts as progress only for a functionalist. It adds a fifth position on the prior. Its tenth challenge, a model that describes features of consciousness it was not trained on, is a proposal for the training-immune marker THEORY-023 names as a refuter, not an instance of one.
- Butlin et al. (LIT-056) build their indicator method on this paper’s theory-balanced approach (NOTE-052).
- Shanahan (THEORY-606) names this paper as the evidential approach and rejects the frame: for a disembodied chat agent, the question is answered by the grammar of the word, not by features.
- Unified agency. The suggestion that one model “can support an ecosystem of multiple agents” is offered as a reply to the disunity objection, not argued. It meets THEORY-490 (Shanahan on what a chat agent’s “I” refers to) and THEORY-043’s remark that functionalism lets one system host two minds.