learning-and-conditioning.
23 of 1538 NOTE documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| NOTE-430 | Relational Frame Theory: Some Implications for Understanding and Treating Human Psychopathology | RFT stated for clinicians. Human relating is bidirectional and arbitrarily applicable, a generalized operant under contextual control, with mutual and combinatorial entailment and transformation of function. Relating relations explains metaphor, rules are networks whose coordination with events regulates behaviour, and the self is content, process and context. These are offered as the reason direct-contingency behaviour therapy is “inadequate”, and ACT’s acceptance, defusion and values are given RFT readings explicitly labelled heuristic. | Read |
| NOTE-438 v2 | A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation | 128 experiments on interesting tasks, separate meta-analyses for free-choice behaviour and self-reported interest. Tangible rewards expected for engaging in, completing or doing well at a task undermine free-choice intrinsic motivation (d from −0.28 to −0.40), and less so self-reports; unexpected and task-noncontingent rewards do not; positive feedback enhances, except for children’s free choice. The pattern fits cognitive evaluation theory and contradicts Eisenberger and Cameron’s helplessness and learned-industriousness account. | Read |
| NOTE-443 | Précis of Breakdown of Will | Hyperbolic discounting makes preference reverse with delay alone, so a person is a population of interests that forestall one another. Will is the recursive bundling of choices by treating each as a precedent, which gives an intertemporal repeated prisoner’s dilemma and makes Kavka’s toxin rational to drink. The same recursion predicts legalism, “lapse districts”, motivated misperception and compulsive over-control, and premature satiation of appetite limits what will can do for emotional reward. | Read |
| NOTE-451 | Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective | The paper introduces hierarchical RL (options) to psychology and neuroscience as a response to RL’s scaling problem. An actor-critic options model learns faster with useful options (doorways in the rooms task) and slower or worse with unhelpful ones (windows; a shortcut ignored). It proposes DLPFC for option identity, DLS for option-specific policies, and OFC for option-specific values and option-level prediction errors. It predicts phasic dopamine at subtask boundaries and a possible neural pseudo-reward. Option discovery is named as the central problem. | Read |
| NOTE-459 | Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control | Habit and goal-directed action are model-free (cached values) and model-based (tree search) reinforcement learners, arbitrated by which controller’s Bayesian posterior over action value has the lower variance. Over-training shifts control to the cache because tree search accrues computational noise per step. Task complexity and proximity to reward keep the tree in control. It also predicts that incentive learning should become unnecessary when the caching system is lesioned. | Read |
| NOTE-463 | Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation | The paper argues that Pavlovian prediction has a model-based form. A cue that predicted disgusting salt was instantly “wanted” in a novel salt appetite, before retasting, with mesolimbic activation. That needs identity-specific prediction revalued by current state, which no model-free cache provides. It proposes “defocusing” of the predicted outcome as a model-based source of apparent habit. It argues dopamine can carry model-based evaluations. A kappa-transformation model fits the data but is described as phenomenological, and no algorithm is offered. | Read |
| NOTE-587 | Pervasive negative effects of rewards on intrinsic motivation: The myth continues | The authors of the 1994 rival meta-analysis re-analyse 145 studies. They accept several of Deci, Koestner and Ryan’s corrections, and code reward contingencies by what participants were told rather than by cognitive evaluation theory’s categories. Result: no overall effect on free choice; enhancement on low-interest tasks; on high-interest tasks, undermining only for expected tangible rewards loosely tied to performance, or earned at less than the maximum; neutral or positive effects for rewards tied to a standard or to beating others. All of it is small. | Read |
| NOTE-948 | Kübler & Birbaumer — BCI and extinction of goal-directed thinking in CLIS | Of 35 patients trained with EEG BCIs in Tübingen, degree of paralysis explained BCI performance (η² = .40) only because of the seven in the completely locked-in state, none of whom reached the 70% needed for yes/no communication; without them the relation vanished (η² = .12, n.s.). All seven were already in CLIS when training began. The authors propose, by analogy with the curarized rat, that lost perception of the contingency between a response and its consequences extinguishes goal-directed thinking, while “passive sensory information processing is intact”, and predict that a contingency learned before CLIS and carried into it should prevent extinction. | Read |
| NOTE-951 | Chaudhary et al. — intracortical spelling in a completely locked-in patient | A man with ALS (born 1985), who had used an eye-movement BCI with an auditory speller until February 2019 and was judged completely locked-in from a few weeks before implantation in March 2019, was given two 64- channel arrays in left motor cortex. Attempted and imagined movements gave no usable signal; from day 86 he was given auditory feedback of firing rate, matched target tones from day 98, and from day 106 spelled with the same speller he had used before CLIS. Over days 106–462 he produced intelligible text on 44 of 107 days on which spelling was attempted, at about 1 character per minute. | Read |
| NOTE-953 | Sung, Rigotti & Lee — factorized goal and uncertainty codes in lateral PFC | A reanalysis of 20 people’s fMRI from a 2014 two-stage Markov task with cued goals and block-wise transition noise. Goals and transition uncertainty are linearly decodable, weakly, from vlPFC, dlPFC and OFC, in a geometry whose nonlinear dichotomies decode worse than goal and linear ones (“factorized”), and a goal decoder trained under one noise level transfers to the other. Goal separability and transfer in lateral PFC correlate with choice flexibility and optimality across subjects; uncertainty separability does not. Unfitted model-free agents show a flexibility–stability trade-off that model-based agents and humans do not. | Read |
| NOTE-1096 | Flayelle et al. — A taxonomy of technology design features that promote potentially addictive online behaviours | A narrative review and a taxonomy. Design features of six kinds of online service are linked, mostly by cross-sectional, qualitative or indirect evidence, to loss of control. They are sorted by the learning and control process each exploits. Model-free: reinforcement schedules and personalized triggers. Model-based: overvaluation, interference with deliberation, partial goal fulfilment, cognitive biases. Providers select features by their effect on use and revenue, and personalize them under informational asymmetry. For the record it maps closely onto the parts of the potential account: features that strengthen a cue-keyed ranking, move its reference course, falsify the user’s reading of progress, or pre-empt the other perspectives. | Read |
| NOTE-1109 | Berridge and Robinson — Liking, wanting, and the incentive-sensitization theory of addiction | An invited review of the incentive-sensitization theory by its authors. ‘Wanting’ and ‘liking’ have dissociable brain systems. Drugs can sensitize the ‘wanting’ system so that it is lastingly hyper-reactive to drug cues and contexts while ‘liking’ stays flat or falls. Sensitized ‘wanting’ can oppose cognitive desire, run without report, and grow in abstinence, and the theory is being extended, on weak evidence, to behavioural addictions. For the record it is a mechanism for proxy-keyed potential in persons. It also shows that the addict case CLAIM-433 settles holds only if the sign of valence is read along the proxy. | Read |
| NOTE-1117 | The power of goal-directed processes in the causation of emotional and other actions | A reversal of the default in dual-process models of action and emotion. Goal-directed processes, which select the action option with the highest expected utility, can be fast and automatic, so they, not stimulus-driven habits or appraisal-to-action-tendency links, are the default cause of action. Most of the competition that decides behaviour is between goal-directed processes, and values and expectancies settle it. Emotional actions are those whose tendencies have control precedence because the goals at stake are highly valued, so emotional episodes are decision processes with higher stakes. | Read |
| NOTE-1120 | Brown, Price & Dombrovski — Anxiety as a disorder of uncertainty | A narrative review and theoretical proposal, with no new data and no simulation. Computational learning studies of anxiety (seven, not a systematic search) find altered adjustment of learning rate to kinds of uncertainty, mostly with aversive outcomes, rather than inflated threat learning or valuation. The authors propose that in aversive environments avoidance of uncertain states keeps their uncertainty high and so perpetuates itself, that anxiety amplifies this by misreading noise as change, and that exposure works by reducing the uncertainty, not the threat. It covers fear-based disorders and says it does not explain GAD. | Read |
| NOTE-1123 | De Kleine et al. — Impaired action-safety learning and excessive relief during avoidance | A cross-sectional comparison of 47 healthy controls with 139 treatment-seeking patients (33 OCD, 40 panic disorder with agoraphobia, 66 PTSD) in a picture-based avoidance task. OCD and PTSD patients pressed the avoidance button more to the safety cue and kept a higher expectancy of the aversive picture after effective avoidance; PDA and PTSD patients reported more relief when it was omitted. Relief was already higher before avoidance was available, and patients rated the pictures as more aversive. That low confidence in avoidance makes omissions surprising and the relief reinforces avoidance is the authors’ interpretation, not something the study tested. | Read |
| NOTE-1127 | Chou, Wilson & Smith — The influence of anxiety on exploration | A narrative review of computational modelling studies of anxiety and exploration, almost all in reward tasks and sub-clinical samples. In bandit tasks, directed exploration tends to rise with trait or cognitive anxiety and fall with state or somatic anxiety; random exploration shows no link; in foraging tasks, and in one task with a predator to check for, anxiety goes with less exploration. Results are inconsistent across tasks and measures and none has been replicated. | Read |
| NOTE-1130 | Gerstner et al. — Eligibility traces and plasticity on behavioral time scales | A review by theoreticians. In a three-factor rule, pre- and postsynaptic co-activation sets a decaying synaptic flag, the eligibility trace, and the weight changes only if a third signal (reward minus expected reward, surprise, novelty, or a neuron-specific error) arrives while it is set. Five experimental papers (2014–2017), mostly in slices, give traces of about 1 s in striatum, 3–10 s in cortex, about 2 s and about a minute in hippocampus. The authors grant that other mechanisms could bridge the same gap. | Read |
| NOTE-1134 | Mancinelli & Bach — A systematic review of human avoidance learning | A PRISMA systematic review, not a meta-analysis, of 116 studies of avoidance learning in healthy humans with shock or noise (1939–2024). Avoidance is learned without terminating the CS, transfers by inference to stimuli never paired with harm, and persists when it can be performed, longest when it is most effective; this contradicts two-factor theory and favours expectancy and inference. But some avoidance persists after the US is visibly removed, which a purely inferential account does not predict. No clinical samples were included. | Read |
| NOTE-1136 | Zorowitz, Momennejad & Daw — Anxiety, avoidance, and sequential evaluation | A decision-theoretic argument with simulations, no data. In sequential evaluation the ability to avoid danger later should make it less threatening now. An agent that computes values as if it will fail to avoid later (a β-pessimistic Bellman equation, weight w) spreads negative value to states far from the threat and avoids early, which reproduces qualitatively exaggerated threat appraisal, generalization, earlier cash-out in risk tasks and aversive pruning, and predicts a reduced free-choice premium. The misbelief concerns the agent’s own future control, not the danger. | Read |
| NOTE-1137 v2 | Craske et al. — Maximizing exposure therapy: an inhibitory learning approach | A clinical translation, not a study. It summarizes evidence that extinction leaves the original threat association intact and adds a competing inhibitory one, so fear can return. It argues that exposure should aim to violate expectancies, not to reduce fear within a session, and that fear reduction during exposure does not predict outcome. It sets out strategies for strengthening and retrieving inhibitory learning, with five invented case examples. Cognitive restructuring before or during exposure is held to blunt learning by lowering expectancy. The evidence on safety behaviours is reported as mixed. | Read |
| NOTE-1138 | Friston et al. — Active inference and learning | A discrete-state active inference scheme in which a habit is one more policy, a state-to-state transition model learned by counting observed transitions, and the prior over policies is learned by counting which policies were pursued. In simulated T-maze foraging the agent first explores, then exploits, then acquires a habit; after the habit is acquired it persists when the reward is devalued, and before it is acquired behaviour switches at once. Simulations only; the authors say the scheme makes probably no new behavioural predictions. | Read |
| NOTE-1149 | Börgers & Sarin — Learning through reinforcement and replicator dynamics | A short theoretical paper. One Bush–Mosteller learner with payoffs in (0, 1), Cross’s model, moves in expectation as a population under the discrete replicator step, and in the continuous-time limit follows the Taylor replicator dynamic at any finite time. For every learning speed, though, the discrete process is absorbed in a pure strategy profile with probability one. With negative payoffs a probability-matching force enters beside a replicator force. Proofs are by Euler’s method and Norman’s theorems on slow learning; the negative-payoff limit is stated with its proof omitted. | Read |
| NOTE-1440 | Rung & Madden — Experimental reductions of delay discounting | A systematic review and category-wise meta-analysis of 92 papers that tried to lower delay discounting without drugs or lesions. Most kinds of manipulation worked, learning-based ones such as bundling and delay fading best. Clinical interventions moved discounting little, and mindfulness and ACT, two studies, only at trend level (g = 0.32, p = .06). Mediation was almost never tested, so how any manipulation lowers discounting is unknown. | Skimmed |