Source
Tilman Börgers and Rajiv Sarin, “Learning through reinforcement and replicator dynamics”, Journal of Economic Theory 77(1):1–14 (1997), LIT-1505, read in full in NOTE-1149 from the working paper of October 1995 (Propositions 3–5, pp. 11–17). The journal version was not read.
What was actually shown
- Expected motion. A player who plays j and receives Ũ ∈ (0, 1) moves a fraction Ũ of the way to the vertex j. The conditional expected change is p_j(e_j Ũ q − p Ũ q), the discrete replicator step (Proposition 3). I re-derived it.
- Finite horizons. With steps scaled by θ, as θ → 0 and nθ → t the process converges in probability to the solution of the continuous Taylor replicator equation at t (Proposition 4, from Norman’s theorem on slow learning).
- Infinity. For every θ > 0 and every initial state, the process converges almost surely to a pure strategy pair (Proposition 5), and from an interior start every pure pair can be the limit (footnote 20). In matching pennies the replicator dynamic cycles around the mixed equilibrium, and the learner is absorbed in a corner. The authors’ word for the gap is that convergence is “pointwise, but not uniformly on the complete real line” (p. 17).
What could have come out otherwise: the slow-learning limit could have carried the replicator dynamic’s long-run behaviour with it. It does not, for any step size.
What this does not say
- That learning in general is replicator dynamics. One rule, with payoffs read as reinforcement strengths in (0, 1). With negative payoffs the limit changes (THEORY-934).
- That the long-run behaviour of the replicator dynamic predicts a learner’s. The theorem says the reverse.
- That absorption is fast. No rate is given. At small θ the process follows the replicator orbit for a long time before absorption.
- Anything about populations of imitators. The analogy is inside one learner; imitation is set aside as complementary (p. 6).
Connections
- THEORY-899. Soma et al.’s Proposition 1 rests on this. The macro-agent identity is of expected motion, and inherits this theorem’s limit at infinity.
- CASE-155 and CLAIM-444. The replicator reading of authority holds for finite horizons. Rest points, cycles and thresholds read off the mean dynamic say what a realized organization does on the way, not where a single learner ends.
- CLAIM-443 and CLAIM-446. Absorption with no feedback from evidence or authority: reinforcement of what was chosen, with no return path, suffices for lock-in. It narrows the claims’ mechanism and supports their worry.
- THEORY-905. The single-learner counterpart of loss of types in a finite imitating population.