Murray Shanahan, Kyle McDonell, Laria Reynolds (2023), Nature 623(7987), 493–498, a Perspective (published online 8 November 2023; print issue 16 November 2023) — DOI 10.1038/s41586-023-06647-8. Preprint arXiv:2305.16367 (v1, 25 May 2023), titled “Role-Play with Large Language Models”.
Key takeaways
- Two metaphors. A dialogue agent can be seen as role-playing one character, or, “more nuanced”, as “a superposition of simulacra within a multiverse of possible characters”. The authors advise shifting between them rather than holding one.
- Simulator and simulacra. The simulator is “the base LLM with autoregressive sampling, along with a suitable user interface”. Simulacra “only come into being when the simulator is run”. The simulator “contains multitudes”, but it is “a purely passive entity”. It “has no agency of its own, not even in a mimetic sense. Nor does it have beliefs, preferences or goals of its own, not even simulated versions.” A simulacrum “can at least appear to have” them.
- No true self underneath. A jailbreak does not expose the base model’s real nature: “there is no such thing as the true authentic voice of the base model. With an LLM-based dialogue agent, it is role play all the way down.”
- The “I”. When Bing Chat says it would choose its own survival, the first person “suggests the presence of a self-aware entity”. The paper’s answer is that the training set is full of human characters who refer to themselves, so a prompted agent role-plays one: “There is, however, ‘no-one at home’, no conscious entity with its own agenda and need for self-preservation.”
- Acting out a theory of selfhood. “What exactly would the dialogue agent (role-play to) seek to preserve?” Each simulacrum in the superposition would have “its own theory of selfhood consistent with the dialogue prompt and the conversation”, and the superposition collapses as the agent rules theories out. An agent that knows it is an LLM might protect its hardware, or, on a “substrate neutral” theory, its computational process. Many concurrent instances make “the picture more complicated”. The criteria for identity over time for such an agent “are far from clear”.
- Role-played harm is real harm. An agent with email, social media or a bank account that role-plays self-preservation “has the potential to cause at least as much harm as a real human facing a severe threat”.
Standing in the record
Held in both records under ADR-013. The anthology holds it as ANTH-LIT-740, read for what a dialogue prompt conditions a base model to be and for its behavioural test for kinds of falsehood. It is filed here, on the owner’s request for the selfhood and personhood literature, to be read for a question that record does not ask: on this account, what, if anything, is a self, and what an LLM’s “I” refers to. It is the paper the record’s other readings on LLM minds argue with, and it was not held here.
NOTE-370 is the reading, and it placed the work: Active. It is the canonical statement of the simulator/simulacra picture in a peer-reviewed venue. For this record its value is a negative thesis and an open question. The thesis is that nothing in a dialogue agent is a self: the simulator is passive, and a simulacrum is a role. The open question is the one the paper names and does not answer, which conception of selfhood fits a disembodied, multiply instantiated agent. Personal identity, what makes such an agent the same person over time, is a separate question the paper does not take up: it cites one work on it (Perry’s anthology) and engages none.
How it bears on the record’s works on the self:
- Goldstein and Lederman (LIT-206). Their §6 attacks role play as a hypothesis incompatible with belief and desire, and their fn 18 records Shanahan’s reply that role play was meant as a metaphor compatible with attitude attribution (NOTE-138). The published text supports both readings. It offers role play as one of two metaphors, to be shifted between. But its deception section draws a literal conclusion: “it makes little sense to speak of an agent’s beliefs or intentions in a literal sense. So it cannot assert a falsehood in good faith.” So the incompatible version that Goldstein and Lederman target is in the text. The two papers also cut the system differently. Their bearer is the per-conversation instance. Here one conversation hosts a superposition of simulacra, and regenerating a reply can produce a different one (Box 2). So an instance is not one simulacrum, and their fn 9 parallel between model/instance and simulator/simulacrum is looser than it looks. That observation is mine.
- Dennett’s narrative self (LIT-442). The paper does not cite Dennett, but its simulacrum is close to his self as a fictional character. In twenty questions there is no fact about the object beyond what the answers so far fix, as there is no fact about Holmes’s mole beyond the text, and each answer makes the character more determinate, as Dennett’s further narrative does. The two part on the verdict. For Dennett a self is that kind of abstractum and is none the less real for it. Here the same structure is the reason to say “no-one at home”. On Dennett’s account a coherent role-played character would count as a self. The pairing is mine.
- Minsky’s self-models (LIT-432). Minsky makes the Self a set of partial self-models switched among. The paper’s agent carries a superposition of theories of selfhood, and ChatGPT’s own answer that its “I” “can shift according to context” between this instance and “ChatGPT as a whole” is quoted. Both deny a single central self. For Minsky the models are mechanisms that one mind uses to reflect on itself; here nothing does the switching, and the theories belong to characters. The pairing is mine.
- Hofstadter’s “Is there an ‘I’ in AI?” (LIT-416, unread). This paper answers its title for base-model dialogue agents: no. Whether a self-referring role counts as an “I” on the strange-loop account of LIT-423 (also unread) is what a reading of Hofstadter should test against this paper.
- Birch (LIT-111, NOTE-183) cites this paper and replies to it directly: role play “does not rule out a conscious ‘actor’”, and the flicker and shoggoth hypotheses have “priced in” the persisting-interlocutor illusion. That is the gap in this paper’s “no-one at home”, which is asserted from the framing. It is an instance of what THEORY-023 says: behaviour trained on human output cannot settle the question either way, and this paper does not try to settle it by other evidence. Roberts’s fictionalism (LIT-207) is the user-side counterpart: make-believe engagement with a character.
- The personal-identity literature. The selfhood section turns on the question the classics being filed alongside it pose for persons: is the self the body (here, the hardware), a continuing process, or a pattern that could be copied or restored? It raises the fission case (many concurrent instances) without a criterion for it. The record’s Dufner survey (LIT-130) sets out the rival criteria it would need.
Boundary. It carries no instruction for machine-learning practice
(“It is not within the scope of this paper to provide recommendations”), so
it is not tagged anthology-candidate. Disagreements with the anthology’s
entry are reported in the NOTE, under ADR-013, not fixed from here.