nucleation

LIT-577: Sophisticated Inference

4 min read

Properties4
StatusActive — read in full 2026-10-03 (NOTE-454), from the arXiv preprint, whose displayed equations did not survive text extraction. Worth reading as the active-inference planning scheme the record's free-energy holdings otherwise lack: expected free energy evaluated recursively over future belief states, a deep tree search pruned by probability, with the Bellman equation as its formal analogue. The evidence is small simulations (a T-maze rat and an 8 × 8 grid). Flagged anthology-candidate: it is a planning algorithm, compared with reinforcement learning, and the anthology's agents-and-environments topic may hold it.
Tagsprobabilistic-modeling, behavioral-integration, motivation, information-theory, anthology-candidate
Cited as “paper” byNOTE-454 — Sophisticated Inference
Cited as “source” byTHEORY-1245 — Evaluating expected free energy recursively over the beliefs an agent would hold after each plausible outcome makes active inference a pruned tree search over belief states, formally analogous to Bellman recursion, which lets the agent plan through short-term costs that one-step agents cannot; that the pruning keeps deep search tractable beyond toy problems is argued, not shown, THEORY-426 — Evaluating expected free energy recursively over the beliefs an agent would hold after each plausible outcome makes active inference a pruned tree search over belief states, formally analogous to Bellman recursion; in single deterministic runs on a T-maze and an 8 × 8 maze it planned through short-term costs that agents with a shorter horizon did not; that the pruning keeps deep search tractable beyond toy problems is argued, not shown

Graph View

Karl Friston, Lancelot Da Costa, Danijar Hafner, Casper Hesp and Thomas Parr (2021), Neural Computation 33(3):713–763; arXiv:2006.04120, 7 June 2020.

Authors and year, verified. The brief gave “Friston et al. (2021; arXiv 2020)” and asked for a check. arXiv and Crossref agree on the five authors in this order. The preprint appeared on 7 June 2020, which is the published: date under the record’s rule; the journal issue is March 2021.

Key takeaways

  • The move. “From simply considering beliefs about ‘what would happen if I did that’ to ‘what would I believe about what would happen if I did that’.” The expected free energy of an action is its own risk and ambiguity plus the average expected free energy of subsequent actions under the beliefs that each plausible outcome would produce.
  • A tree search over beliefs, pruned. Paths are followed only while the action’s prior probability and the outcome’s predictive probability stay above a threshold (1/16 in the simulations). In the maze, horizons of four and six would mean about 1.05 × 10¹⁰ and 1.07 × 10¹⁵ paths; “usually several hundred” were evaluated.
  • Bellman, with beliefs. The recursion is formally like the Bellman equation, but over functionals of beliefs rather than functions of states. In the zero-temperature limit it recovers Bayes-adaptive reinforcement learning; the difference claimed is that expected free energy also values information gain.
  • What the simulations show. In a three-arm T-maze, the sophisticated agent visits the informative cue even when the cue is made mildly aversive; the one-step agent stays put. In an 8 × 8 maze, a planning horizon of four is needed to escape a local minimum behind an aversive barrier. With novelty, an agent with no location preferences covers nearly every location in 64 moves; a Bayesian-risk agent sits still.

Standing in the record

Filed on 2026-10-03 at the owner’s request, in the free-energy part of the agency batch.

The boundary. The brief expected none of this batch to belong in the anthology. This one might. It is an action-selection algorithm, evaluated in simulated environments, positioned against reinforcement-learning methods (Bellman, Q-learning, SARSA, Bayes-adaptive RL) and a deep-RL baseline cited from Tschantz et al., with a co-author from Google Brain. The anthology’s agents-and-environments topic (“percept–action loops, world models … what acting rather than only predicting changes about learning”) could hold it. Against that, the paper is presented as theoretical neurobiology, its simulations are rats in mazes, and it carries no instruction a practitioner would follow. It is filed here with the anthology-candidate flag (ADR-005), carried last, so the transfer question is asked rather than settled. The anthology holds no paper on active inference.

Where it bears on what the record holds:

  • Friston, “Life as we know it” (LIT-526). The paper grounds its “first principles” in the statistical physics of systems at nonequilibrium steady state and cites LIT-526 for it. The critiques filed beside it bear on that grounding, not on the planning scheme. Raja et al. (LIT-598) argue that active-inference models presuppose perception and action in the generative model they are given; this paper’s rat is handed its A, B, C and D matrices, so the critique applies to its simulations as stated.
  • Self-determination theory (LIT-559, LIT-558). The paper equates its “intrinsic value”, expected information gain, with “intrinsic motivation” and cites Ryan & Deci (1985). In self-determination theory intrinsic motivation is doing an activity for its inherent satisfaction and is tied to needs for autonomy and competence; it is not information gain. The identification is the paper’s, by name, not by argument. It is tagged motivation because it offers a formal account of curiosity and novelty seeking that a reader of that topic would expect to find, not because it supports the SDT filings.
  • Constructed emotion (LIT-542). The record’s other active-inference holding, used by THEORY-045. Nothing here bears on affect.
  • Behavioral integration. It is tagged for action selection: how an agent arbitrates between exploiting preferences and resolving uncertainty, and how depth of planning changes the choice.

Backlinks