nucleation

LIT-1476: Active inference and learning

2 min read

Properties5
StatusActive — read in full 2026-10-24 (NOTE-1138) from the CC BY version of record; worth reading as the active inference account in which a habit is one more policy, its transitions and prior probability learned by counting the agent's own behaviour, so that habits form from goal-directed choice and, once formed, survive devaluation. It gives the record a published analogue of a learned log prior beside a precision-weighted evaluation. Simulation of one T-maze task only; its learned prior grows by frequency, not by reward, its reaction times are simulation run times, and the authors concede that it makes probably no new behavioural predictions.
Version3
Tagslearning-and-conditioning, neuroscience, probabilistic-modeling
Cited as “paper” byNOTE-1138 — Friston et al. — Active inference and learning
Cited as “source” byTHEORY-1191 — In a simulated active inference agent, a habit learned by counting the agent's own state transitions and policy choices emerges from goal-directed behaviour, and once acquired it persists, and reinforces itself, after the reward is devalued, while an agent devalued before acquisition switches at once, THEORY-923 — In one simulated T-maze task with hand-set parameters, run once per condition, an active inference agent's habit, learned by counting its own state transitions and policy choices, emerged from goal-directed behaviour, and once acquired it persisted, and reinforced itself, after the reward was devalued, while an agent devalued before acquisition switched at once

Graph View

Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., O’Doherty, J. & Pezzulo, G. (2016). Active inference and learning. Neuroscience & Biobehavioral Reviews 68: 862–879. doi:10.1016/j.neubiorev.2016.06.022.

Key takeaways

From the reading, NOTE-1138. Page numbers are the journal’s.

  • A habit is one more policy. It maps each state to the next whatever the time or context, and “Because habits are just another policy, the arbitration among habits and (sequential) policies rests on their posterior probability” (p. 869).
  • Learned by counting. The habit’s transitions and the prior over policies grow with how often they occur: “if a particular state or policy is encountered frequently, it will come to dominate posterior expectations” (p. 869).
  • Formed from goal-directed behaviour. Habits “emerge naturally (and autodidactically)” from the agent observing itself (p. 862).
  • Insensitive to devaluation once formed. “Despite this switch, the habit persists and, indeed, reinforces itself with repeated executions”; devalued before the habit forms, the agent switches at once (p. 874).
  • Belief-based against belief-free. The authors argue this, not model-based against model-free, is the distinction that matters, and that dynamic programming is the special case without ambiguity (pp. 870–871, 876).
  • Limits the authors state. “At the level of behavioural predictions, the answer is probably no” (p. 876).

Why this is here

The exchange cited it at turn U11 for active inference’s treatment of habits as learned priors over policies: a repeated action becomes the expected one, and so the default. The abstract supports the habit claim; the phrase “prior over policies” is the paper’s mechanism as standardly cited, not the abstract’s wording.

Corrected 2026-10-24, from the reading. Right in substance, with one correction. The paper has two learned things, not one: a habit, which is a separate state-to-state policy whose transitions are learned, and a prior over all policies, learned by counting which policies were pursued. Both grow by frequency, not by reward. The habit competes with the sequential policies by posterior probability on every trial, so it sits beside online evaluation as CLAIM-443 says; it shows persistence, not saved computation. Filed from it: THEORY-923.

Backlinks