nucleation

THEORY-1190: In a Markov decision process, an agent that evaluates actions as if it will fail to avoid danger at later steps, while in fact choosing optimally at each step, spreads threat value to states far from the danger and avoids early and widely, which reproduces qualitatively exaggerated threat appraisal, fear generalization and several laboratory findings associated with anxiety

2 min read

Properties5
StatusProposed
Tagspsychopathology-and-treatment, learning-and-conditioning
SourceLIT-1496 — Anxiety, Avoidance, and Sequential Evaluation
ExtendsTHEORY-915 — In Zorowitz, Momennejad and Daw's simulations of a Markov decision process, an agent that evaluates actions as if it will fail to avoid danger at later steps, while in fact choosing optimally at each step, spreads threat value to states far from the danger and avoids early and widely, which reproduces qualitatively exaggerated threat appraisal, fear generalization and several laboratory findings associated with anxiety
Bears onQUESTION-325 — Is persistent avoidance in anxiety maintained by an inflated uncertainty estimate about the avoided states, as Brown, Price and Dombrovski propose, or by a pessimistic evaluation of one's own future avoidance, as Zorowitz, Momennejad and Daw propose, rather than by an inflated threat value?

Graph View

Source

Zorowitz, Momennejad and Daw (2020), Computational Psychiatry 4: 1–17, LIT-1496, read in full in NOTE-1136. What its simulations showed is THEORY-915, which this entry extends.

The claim

The paper argues from the Bellman equation itself. Under the optimal equation, opportunities for reward propagate back to earlier states and avoidable dangers do not, “because the return-maximizing action is to avoid”: “If danger can be avoided later, it poses less threat now”. Replace the max over next actions with a mixture of best and worst, so that the agent expects to fail later while still choosing optimally now, and the asymmetry goes: threat value spreads backward across the state space, and the agent avoids early and widely. The paper presents this as a property of any such agent, and as an account of exaggerated threat appraisal, fear generalization and several laboratory findings in anxious people, in which the misbelief concerns one’s own future control and the danger is represented correctly.

THEORY-915 holds what the simulations showed, in a gridworld, the balloon analogue risk task, a sleeping-predator task and a two-branch task. The general statement, and the match to anxiety, are the paper’s.

What would settle it

The promote_when, carried over unchanged, which is also the THEORY-915 part of QUESTION-325’s “What would answer it”: people’s choices in a sequential avoidance task fit better by a pessimistic weight on their own future actions than by inflated threat values or altered transition beliefs, with the weight related to anxiety, or a free-choice premium reduced or reversed in anxious people. More simulations matched to published effects cannot settle it.

Backlinks