Contribution
It turns Dennett’s choice of stance into a computation. Given an interaction history, compare the evidence for “this is an agent pursuing some goal” against “this is a device mapping inputs to outputs”, with priors over both mixtures.
Key insight
“devices” are described “in terms of an input-output mapping, while ‘agents’ are described in terms of the function they optimise”. Agency is then whichever description compresses the behaviour better, for this observer.
Assumptions
- An identified system. “we assume a clearly identified system that takes a sequence of inputs and returns a sequence of outputs at discrete time steps”. Where the system ends is given, not found.
- Observer-relative priors. The question “is observer-relative, since it depends the observer’s prior knowledge”. The paper offers Solomonoff and speed priors, and “A more observer-dependent prior could also be considered”.
- Approximately rational agents. Agents are ε-greedy toward an optimal policy for some utility, with ε integrated out.
Key results
- Dennett’s three examples, informally. A stone is best predicted physically. The thermostat is “on the knife edge”, describable “either as a reactive device … or as an agent … using descriptions of comparable simplicity”. AlphaGo’s policy network invites the intentional stance even though “it is entirely reactive”.
- Gridworld results, from the prose.
- Running in circles and following walls are better described as devices.
- Going straight to a goal, or reaching one suboptimally, is better described as an agent.
- Random behaviour is poorly explained by both.
- With a switching prior, a system that changes goals mid-trajectory is still an agent: “agents can change their mind and switch goals—and still be considered agents, as long as the switches are rare”.
- A prior that makes everything an agent. Under the speed prior, “if any device can be represented as an agent then everything may look like an agent because the penalty for devices is too large”. The fix proposed is to penalize agents more.
- The conclusion. “Every physical system can be described as either an agent (which pursues goals) or a device (which responds mechanically to its inputs)”.
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | Agency can be defined as posterior probability under agent-versus-device model comparison | definitional, well-formed | the formalism, Section 3 |
| C2 | The verdict is observer-relative, through the prior | follows from C1 | stated, Sections 1 and 3.1 |
| C3 | Goal-directed trajectories score as agents and reactive ones as devices | illustrated | toy gridworld, seven trajectories, no human study |
| C4 | Rare goal switches are compatible with agency | illustrated | switching prior, Fig. 5–6 |
| C5 | The choice of prior can make everything look like an agent | argued | speed-prior discussion |
Connections
- The admission test (CLAIM-174). C1 is its model-comparison half, and CLAIM-154 supplies the Occam pricing in the record’s own terms.
- The circularity objection (CLAIM-168). The identified-system assumption is the presupposition that the threshold criterion (CLAIM-157) would discharge.
- Time-varying drive (CLAIM-166, now CLAIM-316). C4 says an agent can change what drives it, rarely, and stay an agent. That is the owner’s non-constancy, bounded.
- C5 is the permissiveness objection (CLAIM-010) in formal dress. Whether everything is an agent depends on the prior.
- Dennett (LIT-398) is the source of the stances. THEORY-036 records that his compression criterion “admits almost every non-random pattern”, the same worry as C5.