nucleation

CASE-187: A proposed study: agents matched on competence over a declared family of goals, one planning with an explicit transition model, one a directly learned goal-conditioned policy, one a distributed predictive-state controller, compared on the original goals, on on-demand predictive and interventional queries, under targeted internal interventions, and after environmental change and damage to their predictive machinery, to find which world-modeling achievements are linked and which come apart under matched resources

2 min read

Properties6
StatusActive
Standingstipulated
Kindproposed-study
Tagsagency, cognition, representation-learning
Linedistributed-agency
IllustratesCLAIM-514 — World-modeling is a family of resource-bounded capabilities for producing and using predictions about an environment's possibilities and consequences, each relative to a declared family of questions; a world model in the representational sense is one possible realization of them, competent behaviour can determine environmental structure without identifying any internal mechanism, and producing, maintaining and revising world-modeling capabilities are further achievements, not prerequisites of agency in general, CLAIM-519 — Competent behaviour does not establish an internal world model: over a fixed interface and a finite horizon, an agent with no model, whose actions are looked up from the history so far, reproduces the interaction distribution of an agent that has one exactly; a policy can identify predictive structure without any particular internal component realizing it, so compression and generalization support comparing candidate architectures, and establishing an internal mechanism needs evidence that discriminates candidate implementations, such as targeted interventions, CLAIM-513 — Across a family of goals, a world model is not an optional explanation of competent behaviour: in a fully observed, finite, communicating, stationary environment, a goal-conditioned policy within a factor 1 − δ of optimal on every composite goal of depth n fixes every transition probability to within √(2P(1 − P)/((n − 1)(1 − δ))), so observers reading the same policy must ascribe the same world model; what it leaves open is whether a predictive component is realized inside the agent, since a lookup table over histories and goals has the same policy

Graph View

The design

There are three architectures, matched on competence over a declared goal family:

  • an explicit transition model with planning;
  • a goal-conditioned policy learned directly;
  • a distributed predictive-state controller.

Each is tested four ways, independently specified:

testwhat it isolates
the original goal familyordinary behavioural competence
on-demand predictive and interventional queriesoperational world-modeling competence, in CLAIM-514’s sense
targeted interventions on internalsthe realized mechanism
environmental change, and damage to the predictive machinerymodeling corrigibility and reconstruction

Matching. Comparisons must account for memory, computation and training history. A lookup-table controller can be far more expensive than a compact learned model, so equal competence at unequal cost is itself a result to report, not a confound to hide.

What it would show

The point is not which architecture does best. It is which achievements are linked and which come apart, under matched conditions.

  • If competence on the goals predicts query competence across all three architectures, the two are linked in this class.
  • If one architecture answers queries no better than chance while matching the others on goals, they come apart.
  • If targeted interventions separate architectures that the first two tests do not, mechanism is underdetermined by capability, as CLAIM-519 holds.
  • Recovery after damage to the predictive machinery tests the maintenance-and-revision row of QUESTION-362.

Ordinary competence identifies less than targeted interventions on the machinery that sustains it (CLAIM-414, CASE-161).

Not tested. The design does not test Richens et al.’s theorem, which is about full goal-conditioned policies in fully observed environments. It asks what that theorem leaves open.

Backlinks