nucleation

THEORY-801: Agents are the systems that would adapt their policy were their actions to affect the world differently, so the agent is wherever that adaptation is: a thermostat is not an agent while it together with its design process is, an RL training process is while its learnt policy generally is not, and humans are

2 min read

Properties8
StatusProposed
Version2
Tagsagency, causality
SourceLIT-909 — Discovering Agents
ExtendsTHEORY-527 — If agents are the systems that would adapt their policy were their actions to affect the world differently, then in mechanised causal games meeting Kenton et al.'s five assumptions (one agent per connected component of the decision–utility graph, best response to every mechanism intervention, a fixed tie-breaking order, terminal edges only into decision mechanisms, and mechanism interventions able to set any deterministic function) the game graph, with its decisions, utilities and agents, is recovered exactly from interventions on mechanisms
AnswersQUESTION-228 — Run on real systems, does Kenton et al.'s interventional procedure give the verdicts its definition of agency predicts, for a thermostat with and without its design process, an RL policy with and without its training loop, and a human, and does it separate an accidentally optimal input-output system from an agent where a compression test does not, under a principled choice of variables?
Bears onQUESTION-032 — Relative to a level of description and a target, what makes a factor of a system an agent, and what further makes it a willed agent?
Cited as “rivals” byTHEORY-1236 — Agency is relative to an observer: an identified system is an agent, for that observer, to the degree that a description of the function it optimises explains its behaviour better under the observer's prior than any description of its input-output mapping

Graph View

Source

Kenton et al. (2022), “Discovering Agents”, LIT-909, read in NOTE-707. §1 and §4 give the verdicts on thermostats, design processes, RL training and learnt policies; §5.1 says that “what is a decision or a utility node is frame-dependent”.

What is asserted, and by whom

That agency is the disposition to adapt one’s policy to how one’s actions affect the world, and that on this definition the agent is located at the adaptation: the design process with the thermostat, the training loop rather than the frozen policy, which “derive[s] much of [its] goal-directedness from the process that creates [it]”. The authors argue the verdicts in prose from the definition.

What the record holds toward it

  • THEORY-527 (Active): under the five assumptions, the game graph, with its decisions, utilities and agents, is recovered exactly from interventions on mechanisms (Theorem 1, re-derived). That shows the definition is operational in principle, not that it picks out agency.
  • Against it: the verdicts depend on the chosen variables (§5.1–5.2), the frame problem THEORY-249 presses, and mechanism interventions are not available for most systems of interest. THEORY-1236 is a rival account, and THEORY-624 reports the paper that proposes it.
  • The owner’s question on conditional theorems whose promote_when moves to a QUESTION (THEORY-185, -133, -419, -527) is not decided by filing this.

What would confirm or reject it

The procedure run on real systems with the verdicts as stated, as the promote_when says. QUESTION-228 asks the same thing as a question.

Backlinks