Contribution
A reanalysis meant “to resolve differences in previous meta-analytic findings and to provide a meta-analysis of rewards and intrinsic motivation that permits tests of competing theoretical explanations” (p. 1). It accepts the frame Deci, Koestner and Ryan (LIT-557) insisted on. Tasks are split by initial interest first, dissertations are included, missing effect sizes are imputed as zero, and studies without a proper control are dropped. It then changes one thing: reward contingencies are coded from the procedures the original studies used, in seven categories (Table 1, p. 12), instead of cognitive evaluation theory’s four. With that change, the theory’s “completion-contingent” and “performance-contingent” categories each turn out to pool procedures with opposite effects (Figure 2, p. 18).
Key insight
A meta-analysis inherits the categories of whoever codes it. “The problem with organizing studies according to a particular theoretical stance is that each theory could be used to organize the literature and, using categories appropriate to the theory, each theory could gain support” (p. 10). Coding by procedure is offered as the way out: it assesses “the effects of the actual contingencies rather than those presumed to be an effect by any theoretical perspective” (p. 10). Under that coding, what predicts harm is how loosely the reward is tied to performance, and whether the reward amounts to failure feedback (pp. 22–23).
Assumptions
- Procedural categories are theory-neutral. Assumed, and only partly true. The seven categories are themselves choices: “doing well” with no stated standard is kept apart from “surpassing a score”, for instance. Coding reliability on 32 expected-tangible studies was 97% (p. 12).
- Both control types count for performance-tied rewards: a control told the performance objective and given feedback (“complete”), or not (“partial”). Eisenberger, Pierce and Cameron (1999, LIT-754) found one small difference between them, in the same direction, so both were pooled (p. 13).
- Imputed zeros for missing effect sizes, as Deci et al.; each analysis was also run without them, and both are reported where they differ (p. 13).
- Free choice combines time and performance measures, as Deci et al.; time alone gave “no significant differences” (p. 10).
- Low-interest tasks belong in the analysis, because practitioners and critics of rewards do not distinguish them (pp. 8–9).
Key results
- Sample (p. 11). 145 independent studies, 21 of them dissertations; 115 with a free-choice measure, 100 with self-report. Boal and Cummings (1981) is dropped because “all participants (including the control group) received monetary payments”, and Boggiano and Hertel (1983) because its measure preceded the task.
- All rewards (Table 2, p. 14). Free choice −0.08, not significant; self-report 0.12. Both heterogeneous.
- Low-interest tasks (p. 15). Free choice 0.28 (K = 12, homogeneous); self-report 0.12, not significant. Expected tangible rewards on low-interest tasks: free choice 0.26; offered for doing the task, 0.26 (K = 9).
- High-interest tasks, overall (p. 15). Free choice −0.09; self-report 0.12. Both heterogeneous.
- Verbal rewards (pp. 15–16). Free choice 0.31, homogeneous; children 0.22 (K = 10), adults 0.36 (K = 15), both significant. Self-report 0.32 after the same three outliers Deci et al. removed.
- Tangible rewards (p. 16). Free choice −0.17; self-report 0.08. Unexpected: no effect. Expected: −0.18 and 0.08, both heterogeneous.
- Expected tangible rewards by contingency, free choice (pp. 16–17,
Figure 1).
- No effect: task-noncontingent rewards, rewards for finishing, and rewards for surpassing a score.
- Negative: rewards for doing the task, for doing well, and per unit solved. For doing the task, K = 57, −0.35 (CI −0.43 to −0.27) before outliers. Seven outliers were removed to reach homogeneity, with no common factor found, and the effect stayed significantly negative. Its post-removal value is in the figure only.
- Positive: rewards for meeting or exceeding others’ performance.
- Expected tangible rewards by contingency, self-report (p. 18,
Figure 3).
- Doing the task: small negative, −0.13; −0.22 without the 14 imputed zeros (K = 24).
- Task-noncontingent: no effect.
- Every other contingency: positive. Per unit, 0.19 with all studies (K = 22), before two outliers.
- Per unit, maximum versus less than maximum reward (pp. 19–20).
- Maximum reward: −0.03, not significant (K = 6).
- Less than the maximum: −0.22 (K = 14).
- The authors read this as time pressure or failure feedback, “not reward” (p. 23).
- Durability (pp. 23–24).
- Of seven between-groups studies of rewards for doing the task with a second free-choice measure weeks later, two showed undermining at the second (Morgan 1983, Exps. 1–2).
- Five single-subject operant studies (no control groups) found no decline after the reward ended, against baseline.
- The authors read this as “transitory effects”.
- Magnitude (pp. 24–25). Rewards for doing or doing well average about −0.30. With Pretty and Seligman’s pooled SD of 2.6 min, that is “about 47 s less” in an 8-min free period. The authors flag that this rests on one study’s SD and on extrapolating beyond 8 minutes.
- Theory (p. 26). Social cognitive theory (Bandura 1986) and a behavioural account fit the pattern. A strict reading of cognitive evaluation and overjustification theories does not, because rewards “tied to level of performance (surpassing a score, exceeding others) do not undermine”. The authors grant that CET could fit if competence information “override[s] the controlling aspect of reward”. They cite Eisenberger, Pierce and Cameron (LIT-754) for reward contingencies raising perceived self-determination.
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | Rewards in general do not reduce free-choice intrinsic motivation | weak as stated: the overall category is heterogeneous by the paper’s own test, and the authors say so (p. 21) | Table 2; p. 21 |
| C2 | Rewards raise free choice on low-interest tasks | moderate (K = 12, homogeneous) | p. 15 |
| C3 | On interesting tasks, expected tangible rewards for merely doing the task undermine free choice | strong; agrees with Deci et al. in direction and roughly in size | pp. 16–17 |
| C4 | Rewards tied to surpassing a score or exceeding others do not undermine free choice, and the latter enhances it | moderate: small K per category, control types pooled, figure values not in the text | pp. 16–17, 23 |
| C5 | Per-unit undermining comes from earning less than the maximum: failure feedback or time pressure | weak to moderate (K = 6 against 14; the interpretation is offered, not tested) | pp. 19–20, 23 |
| C6 | Undermining is transitory | weak: seven studies counted by significance, not pooled; five single-subject studies without controls | pp. 23–24 |
| C7 | Deci et al.’s categories pool procedures with different effects, and they omitted positive self-report effects | moderate: documented in Appendix A, skimmed here; not conceded by Deci et al. | pp. 9–10, 25–26, App. A |
| C8 | Social cognitive and behavioural theory fit the pattern better than CET | interpretive: no mediators are measured here | p. 26 |
Concepts
- procedural reward contingency: one of seven categories defined by what
participants were told the reward was for (Table 1, p. 12):
- task noncontingent;
- doing the task;
- doing well;
- finishing;
- per unit solved;
- surpassing a score;
- exceeding a norm.
- maximum versus less than maximum reward: whether rewarded participants met every requirement and received the full reward. Less than maximum typically arises from a time limit (p. 13).
- complete versus partial control: a control group that was, or was not, told the performance objective and given feedback (p. 13).
Connections
Cameron & Pierce (1994), LIT-556. This is the authors’ own earlier analysis, restated on pp. 6–7. It keeps that paper’s conclusion while adopting corrections it was criticised for. Its database is merged with Deci et al.’s (p. 11).
Deci, Koestner & Ryan (1999), LIT-557. This is the paper it compares itself against. The agreements are larger than either side’s rhetoric:
- unexpected and task-noncontingent rewards do nothing;
- verbal rewards enhance;
- expected tangible rewards for merely doing an interesting task undermine free choice: −0.35 here before outliers, −0.40 there (NOTE-438);
- rewards earned at less than the maximum undermine.
The disagreements are about rewards tied to standards or to others’ scores, about self-report, about scope (all tasks or only interesting ones), and about durability.
Self-determination theory’s summaries. LIT-559, LIT-558 and LIT-560 state the 1999 conclusion as settled. This paper shows it is contested on coding, though not on the core effect.
Bearing on the record
- THEORY-046 names this reading as its promotion condition: “a first-hand reading of the reply … finds the same direction for expected tangible rewards given for engaging in an interesting task”. It does, −0.35 on 57 studies, significant after outliers. The durability result (C6) is weaker evidence than THEORY-046’s refutation clause asks for, since it counts significant results instead of pooling them.
- THEORY-049 (cognitive evaluation theory) named this reading as something that “could move this either way”. Rewards for exceeding others raise free choice (C4). That is the case the account’s refutation clause describes, but only if the control group received the same performance feedback. Here the two control types are pooled, and the complete-control result is reported only from Eisenberger, Pierce and Cameron (1999, LIT-754), filed but unread. So it does not refute.
- THEORY-154 states this paper’s own account, that the tightness of the stated contingency decides the effect, as the rival to THEORY-049.
- No instruction for machine-learning practice; nothing here belongs in the anthology.
Limitations
- Its key numbers are images. The per-contingency effect sizes in Figures 1 and 3 after outlier removal are not in the text, so the record has the direction and significance of each contingency, but sizes only where the prose gives them.
- It measures no mediators. Self-efficacy, perceived competence and felt control are inferred from the pattern.
- The two control types are pooled. The most diagnostic contrast between it and CET, performance-tied reward against a control given the same feedback, is therefore not isolated here.
- Small categories: “few studies” for finishing and for doing well on self-report, by the authors’ own account (pp. 22–23).
- It is an adversarial reanalysis by the authors of the result under dispute, as Deci et al.’s was.
Open questions
- Does a performance-tied tangible reward raise free choice against a control given identical performance feedback? A preregistered experiment with both controls would close the question between this account and CET.
- Is “less than maximum” undermining an effect of failure feedback, of time pressure, or of reward? A design crossing time limits with reward would separate them.
- Does the undermining by rewards for doing the task persist? Pooled delayed measures with controls would decide what C6 only counts.