Contribution
A thought experiment that puts a mind inside the implementation of a program, so that one can ask the implementer whether running the program produced understanding. It turned the strong-AI claim from a slogan into a target with a fixed shape, and its catalogue of replies, labelled by the universities that offered them, organised the debate that followed.
Key insight
If a program were sufficient for understanding, then anyone who ran it would understand. A person can run it by hand and not understand. So the understanding, where there is any, comes from something other than the program: for Searle, from the causal powers of the stuff that runs it.
Assumptions
- Proposition 1 (abstract): intentionality is a product of causal features of the brain, “an empirical fact”.
- The man is a full implementation. Following the rules by hand, he is “simply an instantiation of the computer program”. That his slowness and the scale of the task do not matter is assumed, not argued.
- First-person authority settles the case. “It seems to me quite obvious in the example that I do not understand a word of the Chinese stories.” The verdict on the system is read off the verdict the man gives about himself.
- Clear cases suffice. Degrees and kinds of understanding are granted and set aside: “there are clear cases in which ‘understanding’ literally applies and clear cases in which it does not”.
- Programs are purely formal and can be “carved off” from any particular causal realisation.
Key results
- The two-part conclusion (§§1–2 of the argument): the program is not sufficient for understanding; and “not the slightest reason has been given to suppose” it is necessary or contributory.
- Six replies and Searle’s answers:
- Systems (Berkeley): the man internalises rules and data; “there isn’t anything in the system that isn’t in him”, so the system doesn’t understand. A Chinese subsystem would be “simply a part of the English subsystem”. The reply’s only motivation is passing the Turing test, and the adequacy of that test is the point at issue. Accepting it makes stomach, heart and liver “understanding subsystems”.
- Robot (Yale): adds causal relations to the world, and so concedes that formal symbol manipulation is not enough; but the man inside the robot still understands nothing.
- Brain simulator (Berkeley and M.I.T.): an odd reply for a functionalist; water pipes simulating the synapses do not understand, and the man can internalise the pipes.
- Combination (Berkeley and Stanford): we would rationally ascribe intentionality to a humanlike robot “as long as we knew nothing more about it”, and withdraw it on learning it runs a formal program.
- Other minds (Yale): the question is what is attributed, not how it is known; “it is no answer to this argument to feign anesthesia”.
- Many mansions (Berkeley): concedes the thesis by redefining AI as whatever produces cognition artificially.
- What could think. “We are precisely such machines”; an artifact with a nervous system “sufficiently like ours” could; a digital computer could, in the sense that everything instantiates some program; but nothing thinks “solely in virtue of being a computer with the right sort of program”.
- Three breaks in “mind is to brain as program is to hardware”. Programs have absurd realisations (toilet paper and stones, water pipes, wind machines); intentional states are defined by content, not form; mental states are a product of the brain, a program is not a product of the computer.
- Simulation is not duplication. Simulated fires do not burn, and “to confuse simulation with duplication is the same mistake, whether it is pain, love, cognition, fires, or rainstorms”.
- Diagnosis. “Information processing” equivocates; AI keeps a residual behaviourism; and strong AI presupposes a dualism in which the brain does not matter.
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | Instantiating a program is never by itself sufficient for understanding or intentionality | moderate | the room; rests on the man’s first-person verdict standing for the system’s |
| C2 | Nothing shows a program is necessary for, or contributes to, understanding | weak | absence of reasons; Searle: “I have not demonstrated that this claim is false” |
| C3 | The systems reply fails because the man can internalise the system | weak | asserted: “the system is just a part of him”; whether an internalised system is a part whose states are the man’s is the question the reply raises |
| C4 | The systems reply would make digestive organs understanding subsystems | weak | the analogy ignores that the Chinese system’s outputs answer the questions; Searle answers that information is in the eye of the interpreter in both cases |
| C5 | A humanlike robot would rationally be credited with intentionality until we learn it runs a formal program | strong as a report of ascription practice | combination reply, and the comparison with apes and dogs |
| C6 | Only systems with causal powers equivalent to the brain’s can have intentionality | assertion | follows from Proposition 1 plus C1; which powers are required is left empirical |
| C7 | Programs have syntax but no semantics | assertion | stated as the reason C1 holds; argued only through the room |
| C8 | Strong AI presupposes a form of dualism | moderate | the argument that programs are independent of realisation, so the mind would be independent of the brain |
| C9 | The Turing test is “unashamedly behavioristic and operationalistic” | assertion | one sentence in the diagnosis |
Concepts
- Strong AI — the claim that the appropriately programmed computer literally has cognitive states and that programs explain human cognition. Weak AI — the computer as a tool for studying the mind.
- Intentionality — “that feature of certain mental states by which they are directed at or about objects and states of affairs in the world” (n. 3). Beliefs, desires and intentions have it; undirected anxiety does not.
- Understanding — as used here, possession of the intentional states, not their truth (n. 2).
- Formal — identifiable “entirely by their shapes”.
- Causal powers — the brain’s capacity to produce intentional states; what a simulation of “the formal structure of the sequence of neuron firings” leaves out.
Connections
- Turing (LIT-394, read in NOTE-355). Searle’s “other minds reply” is Turing’s answer to Jefferson, and Searle gives it “only a short reply”: the issue is what is attributed, not how it is known. Turing’s paper does not claim understanding for a passing machine; it claims the word “think” will come to be used of it.
- Block (LIT-407, read in NOTE-366). The China brain and the Chinese room both build a functional duplicate out of people. Block doubts its qualia and grants a Psychofunctional homunculi-head beliefs; Searle denies even the intentional states. Block’s elementary-particle case is the mirror of Searle’s internalisation: there the tiny people are parts of you, and Block says your pains survive.
- Nagel (LIT-096). Searle’s reference list has Nagel’s bat only as a commentator’s citation; the target article argues about intentionality and does not use the bat.
- Seth (LIT-135, NOTE-169) separates his biological naturalism from Searle’s later formulation. This paper is the earlier, causal-powers version, and it already leaves open whether another chemistry would do.
- Chalmers. His implementation paper, being filed now by other readers, answers Searle by finding two computational structures, and so two minds, in the room.
Bearing on the record
- THEORY-043 (Proposed): extends it with a third route. The account finds two ways to accept a mind of mindless parts while refusing a mind of minded parts: anti-nesting, and architecture. Searle refuses the downward move itself, and on neither ground: by causal powers that a formal organisation cannot carry. His argument would equally refuse a society of mind, a nation, or a single brain described as a program, so it does not separate Minsky from Schwitzgebel at all; it rejects both functionalist moves. The account should name this as the position that sits outside its dispute rather than as a way across it.
- THEORY-023 (Proposed): consistent. C5 shows that for Searle behaviour does license ascription until architectural knowledge defeats it. That is THEORY-023’s two-level structure, with Searle taking the biological side of the architectural dispute.
- NOTE-176: Šekrst’s use of the Chinese room for the claim that current models lack understanding is within the paper’s scope, but the paper’s conclusion is that no program suffices, not that a given system lacks understanding. NOTE-176 records that Šekrst later disavows the argument as one against the possibility of machine understanding, which matches what Searle himself grants.
- No new THEORY is indicated. No ML instruction; nothing for the anthology.
Limitations
- The decisive step is asserted. The internalisation answer to the systems reply assumes that whatever the man runs in his head is a part of him whose understanding he could report. That is the question.
- Introspective access is assumed to be authoritative about all understanding the man’s head might host.
- Scale and speed are idealised away. No estimate is given of how long the man would take, or whether that bears on the verdict.
- Consciousness is not argued. The paper mentions producing “consciousness, intentionality, and all the rest of it”, but its argument concerns intentionality and understanding.
- Which causal powers matter is left open (“an empirical question”).
- Only the target article was read. The commentaries, including Dennett’s and Hofstadter’s, and Searle’s response were not.
Open questions
- What test, if any, would show a system has the brain’s relevant causal powers, since by C5 behaviour cannot and the program cannot?
- Does the internalisation reply survive if the internalised system’s states are not accessible to the man’s introspection, as the subsystem version of the systems reply supposes?
Corrections
- none to a seeded skim (there was no seed)
- Page range. Citations of “BBS 3:417–457” give the whole treatment with commentaries and response. Crossref gives the target article as pp. 417–424.