nucleation

QUESTION-313: When a habit is modelled as a prior over policies that grows by counting, do the counts fitted to an animal's or a person's own choice history predict when behaviour becomes insensitive to devaluation, and when it still switches at once?

1 min read

Properties5
StatusOpen
Tagslearning-and-conditioning, probabilistic-modeling, neuroscience
Linewill-organization
RefinesQUESTION-088 — Does modelling psychological organization as processes that allocate influence to one another predict habit, rumination, rigidity, akrasia and insight better, with one shared set of organizational variables, than models built for each phenomenon or around a central executive?
Cited as “bears on” byTHEORY-1191 — In a simulated active inference agent, a habit learned by counting the agent's own state transitions and policy choices emerges from goal-directed behaviour, and once acquired it persists, and reinforces itself, after the reward is devalued, while an agent devalued before acquisition switches at once, THEORY-923 — In one simulated T-maze task with hand-set parameters, run once per condition, an active inference agent's habit, learned by counting its own state transitions and policy choices, emerged from goal-directed behaviour, and once acquired it persisted, and reinforced itself, after the reward was devalued, while an agent devalued before acquisition switched at once

Graph View

Why it is a question

THEORY-923 shows, in one simulated T-maze task with hand-set parameters, a habit learned by counting that persists after devaluation once acquired, while an agent devalued earlier switches at once. Nothing is fitted to behaviour, and the authors concede that the scheme probably makes no new behavioural predictions. Whether the count-driven prior is how habits form is a part of what QUESTION-088 asks of process models of habit.

What would answer it

An empirical test, not another simulation: animals or people whose choices are fitted by this scheme, with the habit’s prior and transitions estimated from their own history, showing that the amount of prior practice the fitted counts imply predicts insensitivity to devaluation, and that the same fits predict immediate switching when devaluation comes before that point. The account of habit is refuted if devaluation sensitivity is lost under training regimes where the fitted policy counts stay low, or kept where they are high. A simulation in another task cannot settle it.

Backlinks