nucleation

THEORY-1191: In a simulated active inference agent, a habit learned by counting the agent's own state transitions and policy choices emerges from goal-directed behaviour, and once acquired it persists, and reinforces itself, after the reward is devalued, while an agent devalued before acquisition switches at once

2 min read

Properties5
StatusProposed
Tagslearning-and-conditioning, probabilistic-modeling, neuroscience
SourceLIT-1476 — Active inference and learning
ExtendsTHEORY-923 — In one simulated T-maze task with hand-set parameters, run once per condition, an active inference agent's habit, learned by counting its own state transitions and policy choices, emerged from goal-directed behaviour, and once acquired it persisted, and reinforced itself, after the reward was devalued, while an agent devalued before acquisition switched at once
Bears onQUESTION-313 — When a habit is modelled as a prior over policies that grows by counting, do the counts fitted to an animal's or a person's own choice history predict when behaviour becomes insensitive to devaluation, and when it still switches at once?

Graph View

Source

Friston, FitzGerald, Rigoli, Schwartenbeck, O’Doherty & Pezzulo (2016), LIT-1476, read in full in NOTE-1138. What its one simulated T-maze task, with hand-set parameters and run once per condition, showed is THEORY-923, which this entry extends.

The claim

The paper states its result as a property of the scheme, not of the one task. In active inference a habit is one more policy, a state-to-state mapping whose transitions and prior grow by counting what the agent does. The paper asserts that such a habit forms from the agent’s observation of its own goal-directed behaviour, and that once acquired it persists after devaluation, while before acquisition behaviour switches at once (NOTE-1138, C1–C2). It goes further and says that, to the extent one accepts the variational formulation, model-free learning cannot be learned apart from model-based behaviour (C7). The authors also concede that at the level of behavioural predictions the scheme probably offers nothing new (p. 876).

What would settle it

The promote_when, which is QUESTION-313: choices of animals or people fitted by this scheme, with the counts estimated from their own history, predicting when behaviour becomes insensitive to devaluation and when it still switches at once. A simulation in another task cannot settle it.

Backlinks