Matthew D. Hoffman, Andrew Gelman (2011), Journal of Machine Learning Research 15:1593–1623 (2014) — ARXIV-1111.4246
Key takeaways
- NUTS removes HMC’s trajectory-length parameter L by building trajectories recursively (doubling) until they begin to turn back on themselves. With step size tuned automatically by Nesterov-style dual averaging, it matches or beats well-tuned HMC with no hand tuning.
Seeded from the abstract and a skim, before the reading in NOTE-081. What follows is what the work says about itself.
Hamiltonian Monte Carlo uses gradient information to avoid random-walk behaviour and handle correlated parameters, so it converges on high-dimensional targets much faster than random-walk Metropolis or Gibbs sampling. Its performance, however, depends heavily on the step size ε and the number of steps L. NUTS builds a set of candidate points spanning a wide swath of the target by a recursive algorithm, and stops automatically when the trajectory starts to double back, so L is no longer needed. Empirically NUTS is at least as efficient as well-tuned HMC. A primal-dual averaging scheme adapts ε during warm-up, so NUTS runs with no hand tuning, which makes it suitable for “turnkey” automatic inference engines such as BUGS-style systems.
Standing in the record
Filed from the survey of 2026-09-25 of work the anthology set aside as out of scope (tier C): 370 seconds of active reading over 3 sessions in the papers-feed tracker. It was filed Deferred, unread. NOTE-081 is the close reading of 2026-09-25, and it placed the work: Active — This is the source of the default sampler in Stan, PyMC and NumPyro, and it is worth reading for its derivation. The algorithm as run today, with multinomial rather than slice selection and adapted mass matrices, is documented elsewhere. A reader who wants current practice should read this paper together with that later account.
When seeded, before the reading in NOTE-081, the entry gave a priority for a deeper reading: medium — Load-bearing and widely used, with moderate owner time (t=370 s over 3 sessions). The skim captures what a note needs. The algorithmic detail in §3 is only needed if the note explains the mechanism.
What the seeding asked a deeper reading to check (NOTE-081 is that reading):
- NUTS is the default sampler in Stan, PyMC and NumPyro, so this is the root citation for “use NUTS with adapted step size and mass matrix” in a probabilistic-modeling note. The survey is right that samplers are in the blurb.
- Current practice has moved on in details: multinomial rather than slice sampling in Stan, diagonal or dense mass-matrix adaptation, and newer ESS estimators. A note should cite Betancourt (2017) or Stan’s documentation for the current form, not treat the 2011/2014 algorithm as what is run today.
Access when seeded: I read the arXiv abstract page (v1 18 Nov 2011) and the full arXiv PDF (30 pp.): §1, the §3 structure, §4 (Empirical Evaluation setup, §4.1–4.2) and §5 (Discussion), plus the ESS appendix. JMLR’s page confirms vol. 15, pp. 1593–1623 (2014). JMLR issues no DOI. The batch gives only the short title; the full title is above.