nucleation

LIT-1513: Multi-agent Reinforcement Learning in Sequential Social Dilemmas

2 min read

Properties4
StatusActive — read in full 2026-10-24 (NOTE-1151) from arXiv v1, the AAMAS 2017 paper; worth reading as the paper that moved social dilemmas from matrix games to learned policies in Markov games. Two two-player games, Gathering (apples, and a beam that removes the rival for a while) and Wolfpack (lone or joint capture of prey), both yield Prisoner's Dilemma payoff matrices when the learned policies are played off, yet respond in opposite directions to agent capacity: a larger network makes Gathering agents more aggressive and Wolfpack agents more cooperative, because aggression is the harder policy to implement in one and cooperation in the other. Agents learn aggression where apples are scarce. It is a descriptive simulation with independent learners, no figures of spread, and no protective institution or depletable commons.
Tagsgame-theory, social-science, learning-theory, anthology-candidate
Cited as “paper” byNOTE-1151 — Leibo et al. — Multi-agent reinforcement learning in sequential social dilemmas
Cited as “source” byTHEORY-935 — In two two-player Markov games whose learned cooperative and defecting policies, played against each other, both give Prisoner's Dilemma payoffs, independent deep Q-learners responded oppositely to capacity: more hidden units raised aggression in a fruit-gathering game, where tagging a rival is the harder policy to implement, and lowered lone hunting in a wolfpack game, where coordination is; in the gathering game, aggression was learned where apples were scarce or a tagged rival stayed out long

Graph View

Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J. & Graepel, T. (2017). Multi-agent reinforcement learning in sequential social dilemmas. Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017). arXiv:1702.03037.

Key takeaways

From the reading, NOTE-1151. Page numbers are arXiv v1’s.

  • Cooperation is a property of policies. “Cooperativeness is a property that applies to policies, not elementary actions” (p. 1). A Markov game is a sequential social dilemma when policies classed as cooperative and defecting, played off against each other, give an empirical payoff matrix that satisfies the social dilemma inequalities.
  • Scarcity breeds aggression. In Gathering, agents trained where apples are scarce or a tagged rival stays out long learn to tag far more. “When learning does change beam-use rate, it is almost always to increase it” (p. 5).
  • Same matrix, different games. Both games’ empirical matrices are mostly Prisoner’s Dilemmas (Wolfpack also gives Chicken and Stag Hunt), yet a larger network raises defection in Gathering and lowers it in Wolfpack: what is harder to learn to implement is less often done.
  • Descriptive, not prescriptive. Independent learners, each treating the other as environment; the model “only addresses whatever part of cooperative behavior arises “by habit”” (p. 6).

Why this is here

Filed on 2026-10-24 as the lead’s testbed for protection and the commons (CLAIM-445). The reading narrows that (NOTE-1151): these are two-player games with no protective regime and no depletable resource, and conditions are fixed during training, so nothing here measures whether aggression or protection recedes when scarcity does. The method, tracking a social behaviour metric while manipulating the environment, could be turned to the claim’s test. Filed THEORY-935. It carries anthology-candidate as an ML method, and no instruction for practice.

Backlinks