nucleation

LIT-1501: A selection-mutation model for Q-learning in multi-agent systems

1 min read

Properties3
StatusUnavailable — filed 2026-10-24 for the owner's question how the record stands on multi-agent reinforcement learning; a reading was attempted the same day and failed on access. The ACM Digital Library (doi:10.1145/860575.860687) answered every request from this client with a Cloudflare challenge (403). Unpaywall and OpenAlex list no open copy. The repository records OpenAlex names hold no file this client could reach: the VUB repository (biblio.vub.ac.be) page is a script-only catalogue entry, ULB's DI-fusion record shows no attached full text, the Hasselt document server returned 503 and a search result describes its record as "No Fulltext", and CiteSeerX reset the connection. Web searches found no author copy. The model the paper derives is read instead through Kaisers & Tuyls 2010 (LIT-1511, NOTE-1140), which restates its equation and shows that it describes frequency-adjusted, not plain, Q-learning. It stays Unavailable until a copy is supplied. No NOTE is filed.
Tagsgame-theory, learning-theory, anthology-candidate
Cited as “source” byTHEORY-932 — The replicator-plus-entropy model of Boltzmann Q-learning in two-player stateless games is, in the small-step limit, the expected dynamics of a learner that updates every action's value each step, and so of frequency-adjusted Q-learning wherever every action's probability is at least its adjustment parameter; it is not that of Q-learning, whose expected update carries an extra factor of each action's selection probability, so that its policy dynamics depend on the Q-values; in Kaisers and Tuyls's three 2×2 games at one temperature, compared by eye, Q-learning's trajectories departed from the model depending on initialization and the frequency-adjusted learner's followed it

Graph View

Tuyls, K., Verbeeck, K. & Lenaerts, T. (2003). A selection-mutation model for Q-learning in multi-agent systems. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ‘03), 693–700. ACM. doi:10.1145/860575.860687.

Not read. The summary is drawn from the abstract and from Kaisers & Tuyls (2010), LIT-1511, which restates the paper’s equation. A reading should check it first.

Why this is here

Filed 2026-10-24 when the owner asked how the record stands on multi-agent reinforcement learning, as the source of “Q-learning is replicator dynamics plus an exploration term that removes absorbing states”. The later paper by one of its authors, read in its place, says the model describes Q-learning only after a frequency adjustment (NOTE-1140).

Backlinks