THEORY-935: In two two-player Markov games whose learned cooperative and defecting policies, played against each other, both give Prisoner's Dilemma payoffs, independent deep Q-learners responded oppositely to capacity: more hidden units raised aggression in a fruit-gathering game, where tagging a rival is the harder policy to implement, and lowered lone hunting in a wolfpack game, where coordination is; in the gathering game, aggression was learned where apples were scarce or a tagged rival stayed out long

Source

Leibo, Zambaldi, Lanctot, Marecki & Graepel (2017), LIT-1513, read in NOTE-1151 from arXiv:1702.03037v1: sections 5.1–5.3 and Figures 4, 6 and 7 (pp. 4–7).

What was actually shown

• Scarcity and aggression. Beam use after 40 million training steps is high with scarce apples or long removal, low with plentiful apples or short removal (Figure 4A), and once changed during learning almost always rises.
• Embedded dilemmas. Policies from the two kinds of environment, played against each other, give mostly Prisoner’s Dilemma matrices in Gathering and Prisoner’s Dilemma, Chicken and Stag Hunt in Wolfpack (Figure 6).
• Capacity. A larger network raises aggression in Gathering and lowers lone-wolf capture in Wolfpack (Figure 7, right); the authors explain it by which policy needs more coordination to implement. Less discounting raises defection and a larger replay batch lowers it in both games.

What this does not say

That the effects are robust: no seeds or spread are reported, and the settings in Figure 7 differ from those in the text. That the policies are cooperative or defecting by any measure independent of the environment they were trained in. Anything about more than two agents, a depletable resource, institutions, or what happens when scarcity changes during learning. Or anything about human cooperation, which the authors offer only as a modelling analogy for habit.