nucleation

THEORY-1236: Agency is relative to an observer: an identified system is an agent, for that observer, to the degree that a description of the function it optimises explains its behaviour better under the observer's prior than any description of its input-output mapping

1 min read

Properties6
StatusProposed
Tagsagency, model-comparison
SourceLIT-911 — Agents and Devices: A Relative Definition of Agency
ExtendsTHEORY-624 — Orseau, McGill and Legg cast whether an identified system is an agent or a device as Bayesian model comparison between descriptions of the function it optimises and descriptions of its input-output mapping, so that the verdict is relative to the observer's prior; on seven toy gridworld trajectories goal-seeking comes out agent-like and wall-following device-like, and they argue that under a speed prior everything may look like an agent
RivalsTHEORY-527 — If agents are the systems that would adapt their policy were their actions to affect the world differently, then in mechanised causal games meeting Kenton et al.'s five assumptions (one agent per connected component of the decision–utility graph, best response to every mechanism intervention, a fixed tie-breaking order, terminal edges only into decision mechanisms, and mechanism interventions able to set any deterministic function) the game graph, with its decisions, utilities and agents, is recovered exactly from interventions on mechanisms, THEORY-801 — Agents are the systems that would adapt their policy were their actions to affect the world differently, so the agent is wherever that adaptation is: a thermostat is not an agent while it together with its design process is, an RL training process is while its learnt policy generally is not, and humans are
Bears onQUESTION-032 — Relative to a level of description and a target, what makes a factor of a system an agent, and what further makes it a willed agent?

Graph View

Source

Orseau, McGill & Legg (2018), “Agents and Devices: A Relative Definition of Agency”, LIT-911, read in full in NOTE-705.

Where the evidence stands

What the paper does and argues is THEORY-624, Active, which this entry extends. The comparison is illustrated on seven toy gridworld trajectories, with the prior chosen and not justified, and under a speed prior everything may look like an agent. Kenton et al. (THEORY-527, THEORY-801) object that a compression test may count accidentally optimal systems as agents; QUESTION-228 asks for that test. The promote_when above is THEORY-624’s condition before the twenty-fourth status review promoted it, carried here unchanged, because it asks what would make the thesis true and not what the source says.

Backlinks