Peter Dayan & Kent C. Berridge (2014), Cognitive, Affective, & Behavioral Neuroscience 14(2):473–492 — DOI-10.3758/s13415-014-0277-8
Key takeaways
- Pavlovian prediction can be model-based. In the Dead Sea salt experiment, rats learned to turn away from a lever that predicted an intensely salty squirt. After drugs mimicking salt deprivation, a state they had never been in, the lever on its first re-presentation was approached and nibbled nearly as avidly as the sucrose lever. Engagement rose more than tenfold, before any salt was tasted. A model-free cache would still have held a negative value.
- The model needs outcome identity and current state. The cue must predict “saltiness”, not just “bad”, and that prediction must be revalued against the present physiological state. The authors call this the most basic expression of a model-based mechanism.
- Defocusing is an alternative to habit. Persistence of responding after devaluation need not mean model-free control. With extended training, the predicted outcome’s representation may become generalized (“tasty food”), and so escape a devaluation that is specific to its identity.
- Dopamine is not only model-free. The ventral tegmental area and nucleus accumbens were strongly activated by the salt cue only in the new appetite state. The authors take this as evidence that dopamine release can reflect model-based evaluation, so mesolimbic dopamine is not only a temporal-difference teaching signal.
Standing in the record
Filed on 2026-10-03 at the owner’s request, for the record’s coverage of
behavioural control and motivation. It carries Daw, Niv and Dayan’s
distinction (LIT-580) from instrumental action to Pavlovian
prediction. That is why it declares extends. The paper takes the
model-based/model-free distinction and its uncertainty-based arbitration
from that work. It also reinterprets that paper’s account of incentive
learning: the instrumental model-based system knows the new value but is
uncertain about it, because the state is novel. The paper builds on that
framework throughout and could not stand without it.
The boundary. The paper is about how brains predict reward and how bodily state transforms motivation. Its computational content (temporal difference learning, the κ transformation, model-based evaluation) explains rats and dopamine, and it gives no instruction for machine-learning practice. It belongs here.
What it adds to the record:
- Value is computed against the body’s current state. This bears on the record’s account of felt affect, THEORY-045, which holds that affect reports the body’s regulatory state. Here the bodily state enters the computation of a cue’s incentive value directly and instantly, without new experience of the outcome. The paper does not discuss felt affect. But its “liking” reactions to the cue itself (conditioned alliesthesia) are an affective response that follows the state.
- Wanting and liking are separable. The paper’s incentive salience is “wanting”, a motivational attribution distinct from hedonic “liking”. Ainslie’s précis (LIT-586) uses Berridge and Robinson’s wanting-without-liking for its own model of urges, but rejects reading them as conditioned. This paper defends Pavlovian prediction as a distinct process with its own model-based form, so the two disagree on that point.
No instruction for machine-learning practice; nothing here belongs in the anthology.