Topics — What a work is about. Nucleation holds whatever reading the owner files here, whether or not an Anthology of the SOTA topic could also hold it; instruction for ML practice belongs to the anthology (ADR-048). The words are seeded from the clusters the reading-time triage of 2026-09-25 found, and grow by decision.
Learning and conditioning — how organisms learn from contingencies and consequences — Pavlovian and instrumental conditioning, reinforcement and reward learning in brains, habit and goal-directed control, behaviour analysis and relational responding. Not machine learning, which is learning-theory or the anthology (ADR-025).
28 of 2004 LIT documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| LIT-541 | Relational Frame Theory: Some Implications for Understanding and Treating Human Psychopathology | Barnes-Holmes, Barnes-Holmes, McHugh & Hayes (2004), International Journal of Psychology and Psychological Therapy 4(2):355–375. RFT stated briefly: humans relate events bidirectionally and arbitrarily, as a generalized operant under contextual control, so words take on the functions of what they are related to. Frames combine into networks, networks are related to networks (metaphor), rules are networks that regulate behaviour, and the self is content, process and context of verbal relations. Psychopathology is read as verbal behaviour, and ACT’s acceptance, defusion and values are given RFT interpretations. | Active — read in full 2026-10-03 (NOTE-430), in place of the RFT book, LIT-549; worth reading as a short statement of Relational Frame Theory by two of the book’s editors, and as the paper that shows how RFT is meant to become a theory of psychopathology. Humans, unlike the animals of conditioning research, relate events bidirectionally, so a word acquires the functions of what it names. Relating is a generalized operant learned across exemplars, and its frames share mutual entailment, combinatorial entailment and the transformation of function. Rules are relational networks, and the self is three verbal relations to one’s own behaviour. ACT’s acceptance, defusion and values are then reinterpreted in those terms. Its clinical examples are illustrations, not data, and it says its Part 3 is “no more than a heuristic”. |
| LIT-549 v2 | Relational Frame Theory: A Post-Skinnerian Account of Human Language and Cognition | Hayes, Barnes-Holmes & Roche (eds.), Kluwer Academic/Plenum, 2001. The founding book of Relational Frame Theory: human language and cognition are learned, generalized operant behaviour, the arbitrarily applicable relating of events (mutually, in combination, and with transformation of their functions) under contextual control. Eight chapters state the theory, from derived relations to rules and the self; five apply it to development, education, social processes, psychopathology and religion. Unread: registered from Crossref’s chapter records, with LIT-541 read in its place. | Unavailable — registered 2026-10-03 from Crossref’s book and chapter records, not read; LIT-541, a 21-page statement of the same theory by two of its editors, was read in its place. No lawful full text of the book was found. Springer’s book page returned a “Client Challenge” bot page, Unpaywall and OpenAlex list no open copy, and Maynooth University’s repository (where Barnes-Holmes and Roche deposited many RFT papers) holds no chapter of it. A copy on an Ethiopian national-library server was not used, because it is not an author, institutional or publisher deposit. The editors’ own fourteen-page chapter 8, “Relational Frame Theory: A Précis” (pp. 141–154, DOI 10.1007/0-306-47638-x_8), is the part a human reader should take first; a reading of the whole book would move this to Active. |
| LIT-556 v3 | Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis | Cameron & Pierce (1994), Review of Educational Research 64(3):363–423. A hierarchical meta-analysis of about 96 experiments concluding that rewards do not, overall, decrease intrinsic motivation; that verbal rewards increase it; and that the only reliable negative effect is on free-choice time when expected tangible rewards are given for merely doing a task. It called for “abandoning cognitive evaluation theory”. Unread: registered from the account of it in LIT-557 and its authors’ restatement in LIT-562. | Unavailable — registered 2026-10-03 from Crossref and from the two papers that analyse it, not read. The article is closed access at SAGE (Review of Educational Research), and Unpaywall finds no open copy. A web search found it only on Scribd, a third-party upload, which was not opened. Its authors’ reply to Deci, Koestner and Ryan, Cameron, Banko and Pierce (2001, LIT-562), has since been read and restates this paper’s results, but not as a substitute for reading it. A reader with SAGE access could promote this. |
| LIT-557 v2 | A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation | Deci, Koestner & Ryan (1999), Psychological Bulletin 125(6):627–668. A hierarchical meta-analysis of 128 experiments (101 with free-choice, 84 with self-report measures) on interesting tasks. Tangible rewards undermined free-choice intrinsic motivation (d = −0.34) and self-reported interest (−0.07); expected rewards did, by every task-related contingency; unexpected and task-noncontingent rewards did not; positive feedback enhanced both measures, but not children’s free choice. Read as strong support for cognitive evaluation theory and against Eisenberger and Cameron’s behavioural account. | Active — read in full 2026-10-03 (NOTE-438), from an OCR of the scanned published article the authors post on selfdeterminationtheory.org; worth reading as the largest meta-analysis of the reward-undermining effect and the evidential base SDT cites for it. Across 128 experiments on interesting tasks, expected tangible rewards undermined free-choice intrinsic motivation (d = −0.36), engagement-, completion- and performance-contingent rewards all did (−0.40, −0.36, −0.28), unexpected and task-noncontingent rewards did not, and positive feedback enhanced it (0.33), though not for children’s free choice. It reanalyses Cameron and Pierce (LIT-556) study by study. Its categories are CET’s, and the rival authors reply, by the abstract of Cameron, Banko and Pierce (LIT-562, unread), that the negative effects are circumscribed, not pervasive. |
| LIT-562 v2 | Pervasive negative effects of rewards on intrinsic motivation: The myth continues | Cameron, Banko & Pierce (2001), The Behavior Analyst 24(1):1–44. The authors of the 1994 rival meta-analysis re-analyse 145 studies, adopt Deci, Koestner and Ryan’s split by initial interest, and code reward contingencies by procedure rather than by cognitive evaluation theory. Rewards for merely doing an interesting task undermine free choice (−0.35, K = 57), as in Deci et al. Rewards for surpassing a score do not, and rewards for beating others enhance. Every effect either way is small. | Active — read in full 2026-10-05 (NOTE-587), on the owner’s instruction, from the article as hosted by the Cambridge Center for Behavioral Studies; the same article is free in PubMed Central (PMC2731358). Worth reading as the strongest case against pervasive reward undermining, by the authors of the rival meta-analysis, and as the paper that shows the coding scheme decides the result. Re-coding 145 studies by procedure, it finds no overall effect on free choice and enhancement on low-interest tasks. On interesting tasks it finds undermining only for expected tangible rewards loosely tied to performance or earned at less than the maximum, and neutral or positive effects for rewards tied to a score or to beating others. The core effect agrees with Deci et al.; the per-contingency sizes are in figures with no text layer. |
| LIT-580 | Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control | Daw, Niv & Dayan (2005), Nature Neuroscience 8(12):1704-1711. Habitual and goal-directed control are model-free (“caching”) and model-based (“tree-search”) reinforcement learners in dorsolateral striatum and prefrontal cortex. Behaviour follows whichever controller’s value estimate is less uncertain under approximate Bayesian versions of each. Simulations reproduce the devaluation data: over-training yields devaluation-insensitive habit, except for actions near reward or in tasks with more actions and outcomes. Both controllers serve the same ends; neither is the “irrational” one. | Active — read in full 2026-10-03 (NOTE-459), supplementary methods not read. Worth reading as the founding statement of model-based versus model-free control as two brain systems, and of arbitration between them by relative uncertainty. Goal-directed (devaluation-sensitive) behaviour is a prefrontal “tree-search” controller, statistically efficient but noisy to compute. Habit is a dorsolateral-striatal “caching” controller, cheap but data-hungry and blind to revaluation. Each controls behaviour when its value estimate is the more certain, which reproduces why over-training produces habits unless the task is complex or the action near the reward. Both controllers pursue “identical rational ends”: a rejection of the impulsive-limbic-versus-rational-prefrontal picture. |
| LIT-581 | Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective | Botvinick, Niv & Barto (2009), Cognition 113(3):262-280. Hierarchical RL groups actions into reusable “options” with their own policies, subgoals and pseudo-reward. That eases the scaling problem by cutting the number of decisions to explore and learn. The paper builds an actor-critic version and maps option identifiers to DLPFC, option-specific policies to dorsolateral striatum, and option-specific values and option-length prediction errors to OFC. Unsuitable options produce negative transfer, so how useful options are discovered is the central open problem. | Active — read in full 2026-10-03 (NOTE-451), from the PubMed Central author manuscript; the online appendix with the equations was not read. Worth reading as the bridge between hierarchical reinforcement learning (temporally abstract “options”) and the psychology and neuroscience of hierarchically organized behaviour. Options ease the scaling problem of RL by shrinking search and learning to a few decision points. An actor-critic version maps option identifiers to dorsolateral PFC, option-specific policies to dorsolateral striatum, and option-specific values and extended prediction errors to orbitofrontal cortex. The problem it puts first is option discovery: the wrong options cause negative transfer, as in human problem solving. |
| LIT-583 v2 | Breakdown of Will | Ainslie (2001), Cambridge University Press. The book-length statement of picoeconomics: hyperbolic discounting makes a person a population of bargaining interests, and will is the recursive bundling of choices into precedents. Unread: registered from the publisher’s description and the précis, LIT-586, which stands in for it. | Unavailable — registered 2026-10-03 from the publisher’s description and from the outline of the book given in its précis, not read. Not open access: Unpaywall reports no open copy, and no author deposit of the book was found (the author’s site, picoeconomics.org, carries the précis and articles, not the book). The owner suggested the précis instead, and it is read and filed as LIT-586 (NOTE-443), which follows the book’s three sections and twelve chapters. The book is kept as the citation others reach for. A human with library access could read chapters 5-8 (the bargaining model and its evidence) against the précis. |
| LIT-584 | Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation | Dayan & Berridge (2014), Cognitive, Affective, & Behavioral Neuroscience 14(2):473-492. Pavlovian prediction is not only model-free. A cue that had predicted disgusting salt became instantly “wanted” when a never-experienced salt appetite was induced, before the salt was retasted, which needs a prediction of the outcome’s identity revalued against the current bodily state. Over-training may “defocus” outcome representations rather than hand control to a model-free system. Dopamine may carry model-based evaluations. No algorithm for the revaluation is given. | Active — read in full 2026-10-03 (NOTE-463), from the PubMed Central author manuscript. Worth reading as the argument that Pavlovian prediction, usually modelled as purely model-free, is also model-based. In the “Dead Sea salt” experiment a lever that had predicted a disgusting salt solution became, on first presentation after a novel sodium appetite was induced and before any salt was retasted, nearly as attractive as a sucrose lever, with mesolimbic activation. A cached value cannot do that. The CS must predict the outcome’s identity, and its value must be recomputed against the current bodily state. The paper adds “defocusing” of outcome representations as a model-based route to apparent habit, and argues that dopamine can carry model-based evaluations. |
| LIT-586 | Précis of Breakdown of Will | Ainslie (2005), Behavioral and Brain Sciences 28(5), the précis of Breakdown of Will (2001). People and other animals discount delayed rewards hyperbolically, so preferences reverse as a smaller, sooner reward comes near, and a person becomes a population of reward-seeking “interests” that must forestall one another. Willpower is the bundling of future choices by treating the present one as a precedent: an intertemporal repeated prisoner’s dilemma, not an organ or a top-down ego. The same recursion predicts the will’s costs: legalism, circumscribed “lapse districts”, motivated blindness to lapses, and rules that favour countable goals over subtle ones. | Active — read in full 2026-10-03 (NOTE-443), as the substitute for the 2001 book (LIT-583), which the owner suggested and which the précis follows chapter by chapter. Worth reading as the shortest complete statement of picoeconomics in Ainslie’s own words. Hyperbolic discounting makes preference reverse as a smaller, sooner reward draws near, so a person behaves like a population of “interests” that bargain across time. Will is not a faculty above them but a recursive self-prediction: treating each choice as a precedent “bundles” future rewards, an intertemporal repeated prisoner’s dilemma in which “credibility is power”. The same mechanism predicts the will’s side effects: legalism, “lapse districts”, motivated misperception and compulsive over-control. |
| LIT-754 v2 | Effects of reward on intrinsic motivation—Negative, neutral, and positive: Comment on Deci, Koestner, and Ryan (1999) | Eisenberger, Pierce & Cameron (1999), Psychological Bulletin 125(6):677–691, a comment on Deci, Koestner and Ryan’s meta-analysis. By Cameron, Banko and Pierce’s account it re-analyses the performance-contingent studies by reward procedure and by the kind of control group, and reports that reward contingencies raise perceived self-determination. Unread: closed at APA, no abstract reachable. | Unavailable — registered 2026-10-05 from Crossref and PubMed records and from the account of it in Cameron, Banko and Pierce (2001, LIT-562), not read. The article is closed at APA (Psychological Bulletin); Unpaywall, OpenAlex and Semantic Scholar list no open copy, and neither PubMed nor Crossref carries its abstract. The only web copy found is on ResearchGate (not used). The author’s University of Houston lab page, which may post it, answered with 503s and connection resets and was not reached. A reading would test the comparison THEORY-049’s refutation clause needs: performance-tied reward against a control given the same feedback. |
| LIT-906 v3 | The effect of brief mindfulness training on momentary impulsivity | Dixon, Paliliunas, Belisle, Speelman, Gunnarsson & Shaffer (2019), Journal of Contextual Behavioral Science 11: 15–20. Twenty-three participants were randomised to a 5-minute mindful-breathing video or a music video. Monetary delay discounting decreased after the mindfulness video and did not change after the control. Small and brief. Unread beyond the abstract’s summary. | Unavailable — unread, registered 2026-10-11 from Crossref metadata and the abstract as summarised on the Missouri State BearWorks record page. The article is closed. Filed as direct evidence on whether mindfulness changes discounting, which is the fork the record’s mindfulness claim turns on. |
| LIT-1184 v2 | Factorized embedding of goal and uncertainty in the lateral prefrontal cortex guides stably flexible learning | Sung, Rigotti & Lee (2025). A reanalysis of 20 people’s fMRI from a two-stage Markov task with experimenter-cued goals and uncued block-wise transition noise. With a specific goal cued, transition noise did not lower choice optimality or consistency. Unfitted model-free agents showed a flexibility–stability trade-off (r = −0.964) that model-based agents and humans did not. Goals and, under specific goals only, transition uncertainty were weakly decodable from vlPFC, dlPFC and OFC. Nonlinear dichotomies decoded worse than goal and linear ones, and a goal decoder transferred across noise levels: the paper’s “factorized embedding”. Across subjects, goal separability and transfer in lateral PFC correlated with flexibility and performance; uncertainty separability correlated with nothing. | Active — read 2026-10-16 (NOTE-953) in full from the open-access article, with the supplementary tables for the shattering comparisons and the brain–behaviour correlations. Worth reading as a clear human-fMRI application of Bernardi et al.’s geometry tests (shattering dimensionality, cross-condition generalization) to goals and transition noise. Read it knowing that it is a reanalysis of 20 subjects with effects a few points above chance; that “uncertainty” is block-wise transition noise, coded only when a specific goal is cued; that it tested only whether the goal code is invariant to uncertainty, never the converse; that the task is action–outcome learning with experimenter-cued goals; and that goal-code invariance does not correlate with stability once corrected for multiple comparisons, although the main text says it does. Registered at the owner’s request (U9 of the exchange of 2026-10-16 on active inference). |
| LIT-1199 v2 | Brain–computer interfaces and communication in paralysis: Extinction of goal directed thinking in completely paralysed patients? | Kübler & Birbaumer (2008). Across 35 patients trained with EEG BCIs, paralysis explained performance (η² = .40) only because of the seven in the completely locked-in state, none of whom, having entered training already in CLIS, reached yes/no communication; without them η² = .12, n.s. Evoked potentials showed hearing and some semantic processing intact. The authors propose that lost perception of the contingency between a response and its consequences, with reward blunted by artificially held homeostasis, extinguishes goal-directed thinking; predict that a contingency learned before CLIS and carried into it should prevent this; and say one CLIS patient who learns to communicate would disprove it. | Active — read 2026-10-16 (NOTE-948) in full, from the NIH author manuscript in PubMed Central. Worth reading as the source of the extinction hypothesis for the completely locked-in state: none of seven patients who began BCI training already in CLIS reached yes/no communication, and the authors propose that losing the perceived contingency between a response and its consequences, with reward blunted by artificially held homeostasis, extinguishes goal-directed thinking while passive sensory processing stays intact. Read it knowing that the hypothesis rests on seven heterogeneous patients and an analogy with curarized rats whose result failed to replicate, that it is stated as “might be responsible”, and that it predicts its own exception: a contingency learned before CLIS and carried into it should prevent extinction. |
| LIT-1200 v2 | Spelling interface using intracortical signals in a completely locked-in patient enabled via auditory neurofeedback training | Chaudhary et al. (2022). A man with ALS who had used an eye-movement BCI with an auditory speller until February 2019, and was judged completely locked-in from a few weeks before implantation, was given two microelectrode arrays in motor cortex in March 2019. Attempted and imagined movements gave no usable signal; from day 86 he learned to move an auditory tone driven by firing rate, and from day 106 spelled with the speller he had used before CLIS, producing intelligible sentences on 44 of 107 attempted days at about one character per minute. | Active — read 2026-10-16 (NOTE-951) in full, from the open-access article at nature.com; the Supplementary Information was not read. Worth reading as the one report of a patient classed as completely locked-in who spelled self-chosen sentences through a BCI, by learning to raise or lower motor-cortex firing rates against an auditory tone after attempted and imagined movements gave no usable signal. Read it knowing that he had used an eye-movement BCI with the same speller until weeks before CLIS, that he had been in CLIS for weeks rather than years, that control was intermittent (intelligible output on 44 of 107 days) and rated by the authors, and that the authors concede residual muscle control was not excluded. As published, the article carries no editor’s note, correction, expression of concern or retraction (nature.com and Crossref, 2026-10-16). |
| LIT-1434 | The power of goal-directed processes in the causation of emotional and other actions | Moors, Boddez & De Houwer (2017), Emotion Review 9(4):310–318. Standard dual-process models make stimulus-driven processes the default and give goal-directed ones a late, regulatory role, in action research and in emotion research alike. The authors reverse the default. Goal-directed processes, which select the action option with the highest expected utility, can be fast and automatic too, so most of the competition that decides behaviour is between goal-directed processes, settled by the values and expectancies of outcomes. Emotional action tendencies can be caused this way. Appraisal-to-action-tendency links are sophisticated stimulus-driven processes. Evidence for stimulus-driven control rests on null results open to hidden goals. | Active — read 2026-10-23 (NOTE-1117): in full, from the submitted manuscript deposited in Ghent University’s repository. Worth reading as the shortest statement of Moors’s goal-directed account of emotional action, and as the nearest prior statement of the record’s reading of appraisal as negotiation. Goal-directed processes, which weigh the utility of action options by the values and expectancies of their outcomes, are the default cause of action, emotional action included. Most of the competition that decides behaviour is between goal-directed processes, and its winner is fixed by values and expectancies. Control precedence is what high goal relevance produces, so an emotional episode is a decision process with higher stakes. The competition is run on a common currency, expected utility, which is the point where the record parts from it. |
| LIT-1461 | Liking, wanting, and the incentive-sensitization theory of addiction | Berridge & Robinson (2016), American Psychologist 71(8):670–679. ‘Wanting’ (incentive salience, mesolimbic and dopamine-related) and ‘liking’ (hedonic impact, small opioid and endocannabinoid hotspots) are dissociable. Repeated, especially intermittent high-dose, drug use can sensitize the ‘wanting’ system, making it hyper-reactive to drug cues and contexts, lastingly and perhaps permanently, while ‘liking’ need not rise and may fall. Sensitized ‘wanting’ can oppose cognitive desire, run without report, and grow during abstinence. Its scope is said to extend, on emerging evidence, to behavioural addictions and to the compulsions of dopamine dysregulation syndrome. | Active — read 2026-10-23 (NOTE-1109): in full, from the NIH author manuscript in PubMed Central (PMC5171207, NIHMS816820). Worth reading as the authors’ own short statement, a quarter-century on, of the incentive-sensitization theory of addiction. Drugs, and perhaps unpredictable non-drug rewards, can make mesolimbic ‘wanting’ systems lastingly hyper-reactive to cues and contexts, and ‘liking’ need not rise with it. Sensitized ‘wanting’ can oppose a person’s cognitive desire, operate without report, persist for years and grow during abstinence. It is a review of the authors’ programme. Filed for the record’s question whether potential of a proxy for maintenance is valence, and the form addiction and metric fixation share (QUESTION-267), to which it gives a mechanism on the personal side. |
| LIT-1465 | A taxonomy of technology design features that promote potentially addictive online behaviours | Flayelle, Brevers, King, Maurage, Perales & Billieux (2023), Nature Reviews Psychology 2(3):136–150. Reviews evidence linking design features to loss of control in online gaming, gambling, cybersexual activity, shopping, social networking and streaming. Proposes a taxonomy based on model-free and model-based learning and control. Model-free: reinforcement schedules (random-ratio rewards such as loot boxes and likes) and personalized triggers (push notifications, recommendations). Model-based: overvaluation of outcomes, features that interfere with deliberation (autoplay, countdowns), partial goal fulfilment (infinite scroll, levels, progress bars) and features that strengthen cognitive biases (near-misses, losses disguised as wins). Providers keep whatever raises use or revenue, and informational asymmetry lets them personalize it. | Active — read 2026-10-23 (NOTE-1096): in full, from the typeset article as deposited in the University of Lausanne’s repository (SERVAL/IRIS). Worth reading as a compact, theory-driven catalogue of the design features of games, gambling, pornography, shopping, social-networking and streaming services that may weaken users’ control. Features are sorted by whether they work on model-free learning (reinforcement schedules, personalized cues) or on model-based choice (overvaluation, features that interfere with deliberation, partial goal fulfilment, cognitive biases). It also names the selection process on the provider’s side: features that raise use, time-on-device or spending are kept, with no psychology needed. The evidence reviewed is mostly cross-sectional, qualitative or indirect, and the authors warn against pathologizing common behaviour. Filed for the record’s question whether potential of a proxy is valence (QUESTION-267), because it describes how a firm optimizing its own metric can install proxy-keyed potential in its users. |
| LIT-1476 v3 | Active inference and learning | Friston, FitzGerald, Rigoli, Schwartenbeck, O’Doherty & Pezzulo (2016). An active inference account of choice and learning, on how goal-directed and habitual behaviour contextualise each other. Habits emerge naturally from sequential policy optimisation in agents equipped with state-action policies, as learned prior beliefs over policies; epistemic, ambiguity-resolving behaviour enables pragmatic behaviour and then habit. The important distinction is between belief-free and belief-based schemes, not model-free and model-based; the scheme covers dopamine transfer, reversal learning, habit formation and devaluation, and reduces to a Bellman scheme without ambiguity. | Active — read in full 2026-10-24 (NOTE-1138) from the CC BY version of record; worth reading as the active inference account in which a habit is one more policy, its transitions and prior probability learned by counting the agent’s own behaviour, so that habits form from goal-directed choice and, once formed, survive devaluation. It gives the record a published analogue of a learned log prior beside a precision-weighted evaluation. Simulation of one T-maze task only; its learned prior grows by frequency, not by reward, its reaction times are simulation run times, and the authors concede that it makes probably no new behavioural predictions. |
| LIT-1477 v3 | A systematic review of human avoidance learning: Cognition, computation, and methods | Mancinelli & Bach (2026; preprint 2025). A systematic review of 116 studies of avoidance learning in healthy humans with primary reinforcers, analysed for the phenomena a theory must explain and their power to tell theories apart. The evidence contradicts two-factor and other associative theories and favours expectancy and inference-based accounts, though residual avoidance after the outcome is visibly removed contradicts a purely inferential one. An available avoidance response protects threat expectancy from extinction, more so the more effective it is. The common paradigms are simpler than real-world avoidance, so tasks with larger computational demands and more action options are needed. | Active — read in full 2026-10-24 (NOTE-1134) from the PsyArXiv preprint, not the closed version of record; worth reading as the only systematic map of human avoidance learning, and for the finding, rated robust, that an available avoidance response protects threat expectancy from extinction, the more so the more effective it is. Avoidance is learned without ending the warning signal and transfers by inference, against two-factor theory; but some avoidance survives visible removal of the outcome, against a purely inferential account. Healthy volunteers with shock or noise only; nothing here is about anxiety disorders. |
| LIT-1479 v3 | Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy | Brown, Price & Dombrovski (2023; preprint 2022). A narrative review and proposal, with no new data or model run. Cognitive-behavioural accounts put exaggerated threat expectancy behind maladaptive anxiety, but the learning literature (seven computational studies, chosen without a systematic search) fits better a disorder of uncertainty learning: altered adjustment of learning rate to noise and volatility, mostly for aversive outcomes. They propose that in aversive settings avoidance of uncertain states keeps their uncertainty high and so perpetuates itself, that anxiety feeds this by misreading noise as change, and that exposure therapy works by reducing the uncertainty. For fear-based disorders, not GAD. | Active — read in full 2026-10-24 (NOTE-1120) from the NIH author manuscript in PubMed Central, not the typeset version of record; worth reading as the clearest statement that anxious avoidance can be kept going by uncertainty that avoidance itself leaves unreduced, with no excess of learned threat needed. A narrative review and proposal: seven computational studies, chosen without a systematic search, show altered adjustment of learning rate to uncertainty, mostly for aversive outcomes; the avoidance loop and the account of exposure are argued in words and schematic figures, not modelled or tested. For fear-based disorders; by the authors’ statement it does not explain GAD. |
| LIT-1483 v3 | The influence of anxiety on exploration: A review of computational modeling studies | Chou, Wilson & Smith (2024). A narrative review of computational modelling studies of how anxiety influences exploration, mostly in reward tasks and sub-clinical samples. In reinforcement-learning tasks directed, but not random, exploration may be raised by trait or cognitive anxiety and lowered by state or somatic anxiety; in foraging tasks anxiety is consistently associated with less exploration, including, in one study, less checking for a predator. Some differences may come from how anxiety modulates learning rates. Results are inconsistent and unreplicated, and tasks, anxiety measures and methods need to be made consistent. | Active — read in full 2026-10-24 (NOTE-1127) from the CC BY PsyArXiv preprint, not the closed version of record, whose abstract differs; worth reading as the map of how anxiety relates to exploration once both are split into kinds. In reward bandits directed exploration tends to rise with trait or cognitive anxiety and fall with state or somatic anxiety, random exploration shows no link, and in foraging, and in one task with a predator to check for, anxiety goes with less exploration. A narrative review of mostly sub-clinical, unreplicated studies with inconsistent tasks and measures. |
| LIT-1490 v3 | Impaired action-safety learning and excessive relief during avoidance in patients with anxiety disorders | De Kleine, Hutschemaekers, Hendriks, Kampman, Papalini, Van Minnen & Vervliet (2023). In a computer-based avoidance task, 47 healthy controls were compared with 33 patients with OCD, 40 with panic disorder with agoraphobia and 66 with PTSD, before exposure treatment. OCD and PTSD patients avoided more during the safety cue and kept a higher expectancy of the aversive picture after the safety cue and after effective avoidance; PDA and PTSD patients reported more relief. The groups did not differ in avoiding the danger cues. The authors propose that low confidence in the safety that avoidance brings makes omissions surprising, and the extra relief reinforces avoidance. | Active — read in full 2026-10-24 (NOTE-1123) from the CC BY version of record; worth reading as, by its own count, the third experimental study of avoidance learning in anxiety patients, and the first to track what they learn from avoiding as well as how often they avoid. Against 47 controls, OCD and PTSD patients avoided a safety cue more and, after effective avoidance, still expected the aversive picture more; PDA and PTSD patients reported more relief at omissions; nobody differed on the danger cues. Cross-sectional, with no cost to avoiding; relief was already higher before avoidance was possible, and patients rated the pictures as more aversive. That the excess relief reinforces avoidance is the authors’ interpretation, not a finding. |
| LIT-1494 v3 | Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules | Gerstner, Lehmann, Liakoni, Corneil & Brea (2018). Behaviour unfolds over seconds and spikes over milliseconds, so learning rules must bridge the two. In neoHebbian three-factor rules, co-activation of pre- and postsynaptic neurons sets an eligibility trace at the synapse, which changes the weight only if a third factor (reward, punishment, surprise or novelty, carried by neuromodulators or special inputs) arrives while it is set. The review sets out four recent experiments supporting eligibility traces on the scale of seconds. | Active — read in full 2026-10-24 (NOTE-1130) from the CC BY version of record; worth reading as the review that set out what the first direct experiments (2014–2017) showed about eligibility traces on the scale of seconds: in striatum about 1 s, in cortex 3–10 s with the modulator setting the sign, in hippocampus about 2 s. Mostly slices, one laboratory per region, by theoreticians whose models assumed the traces; they grant that other mechanisms could bridge the same gap, and what is well established is the hour-scale tag of consolidation. |
| LIT-1496 v3 | Anxiety, Avoidance, and Sequential Evaluation | Zorowitz, Momennejad & Daw (2020; preprint 2019). A decision-theoretic analysis and reinforcement-learning simulations of maladaptive avoidance, with no data fitted. In sequential evaluation the chance to avoid danger later should make it less threatening now; a pessimistic false belief that one’s own later avoidance will fail, with the danger represented correctly, propagates fear and avoidance to situations far before the threat. This one bias reproduces qualitatively exaggerated threat appraisal, fear generalization, persistent avoidance and laboratory findings on approach–avoid conflict and loss avoidance, and predicts a reduced free-choice premium in anxiety. | Active — read in full 2026-10-24 (NOTE-1136) from the CC BY version of record; worth reading for the derivation that avoidance far from a threat needs no overestimate of the threat, only an expectation that one will fail to avoid it later, which spreads threat value backward across a decision process. Simulations only, matched qualitatively to published findings (earlier cash-out, behavioural inhibition, aversive pruning); the predicted loss of the free-choice premium is untested, and the persistence of avoidance after the misbelief is corrected is argued, not simulated. |
| LIT-1500 v4 | Maximizing exposure therapy: An inhibitory learning approach | Craske, Treanor, Conway, Zbozinek & Vervliet (2014). A clinical translation of the inhibitory-learning model of extinction: the original threat association survives extinction, beside a competing inhibitory one, so fear can return; fear reduction during exposure does not predict outcome; and exposure should be designed to violate stated expectancies, not to reduce fear in session. Strategies: expectancy violation, deepened extinction, occasional reinforced extinction, removal of safety signals, variability, retrieval cues, multiple contexts and reconsolidation, with affect labeling as a further regulation strategy. Cognitive restructuring before exposure is held to blunt learning. Five illustrative cases; no new data. | Active — read in full 2026-10-24 (NOTE-1137) from the NIH author manuscript in PubMed Central, the accepted unedited version, not the closed version of record; worth reading as the standard clinical statement of the inhibitory-learning account of exposure: extinction adds a competing inhibitory association and leaves the original intact, fear reduction during exposure does not predict outcome, and exposure should be built to violate expectancies. It reports no data. Its strategies are cited to evidence of uneven strength, which the authors flag, and it reports the evidence against removing safety behaviours as well as for it. |
| LIT-1505 | Learning Through Reinforcement and Replicator Dynamics | Börgers & Sarin (1997; working paper 1993). For two players each running Cross’s version of Bush–Mosteller learning with payoffs in (0, 1), the expected change of each player’s mixed strategy equals the discrete replicator step of a two-population biological model, and in the continuous-time limit the learning process converges in probability to the continuous (Taylor) replicator dynamic at every finite time. Asymptotically it does not: the discrete process is absorbed in a pure strategy profile with probability one for every learning speed. With negative payoffs the limit adds a probability-matching term to a replicator term in “benefits”. | Active — read in full 2026-10-24 (NOTE-1149) from the authors’ working paper of October 1995, as held in Levine’s Working Paper Archive; worth reading as the proved source of “Cross learning is the replicator dynamic”: a single Bush–Mosteller learner with positive payoffs moves in expectation exactly as a population under the discrete replicator dynamic, and in the continuous-time limit follows the continuous replicator dynamic on any finite horizon. Two limits matter as much as the result. Asymptotically the discrete learner is absorbed in a pure strategy profile with probability one at every learning speed, even in matching pennies where the replicator dynamic cycles; and with negative payoffs a second force, probability matching, enters beside the replicator term. |
| LIT-1846 | Experimental reductions of delay discounting and impulsive choice: A systematic review and meta-analysis | Rung & Madden (2018). A systematic review and category-wise meta-analysis of 92 papers that tried to reduce delay discounting or impulsive choice by non-pharmacological means, in humans and animals. Most kinds of manipulation reduced discounting (Q(9) = 224.68, I² = 64%). Learning-based procedures such as bundling and delay fading gave the largest and longest-lasting effects. Framing and priming moved one-off choices. Clinical interventions moved discounting little: mindfulness and ACT (two studies) only at trend level, g = 0.32, p = .06. Formal mediation analyses were rare, so how any of these manipulations works is largely unknown. | Active — read in part 2026-10-30 (NOTE-1440, Skimmed) from the NIH author manuscript in PubMed Central (PMC6112163, NIHMS963171), fetched as PMC XML through NCBI’s E-utilities after the PMC web page put up a reCAPTCHA; worth reading as the systematic review and meta-analysis of 92 papers on reducing delay discounting by non-drug manipulations, and for its finding that mindfulness and ACT moved discounting only at trend level. Read it knowing that effect sizes are pooled only within categories, that publication bias was not assessed, and that its search ended on 1 March 2016. |