John Paisley, Chong Wang, David M. Blei, Michael I. Jordan (2012), IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 256-270 (2015); first posted as arXiv preprint — ARXIV-1210.6738
Key takeaways
- Letting each word follow its own path through a shared topic tree, via document-specific distributions over paths (a global nCRP as base for per-document nCRPs), fixes the single-path restriction of the nested CRP and, with stochastic variational inference, scales hierarchical topic models to millions of documents.
Seeded from the abstract and a skim, before the reading in NOTE-023. What follows is what the work says about itself.
The paper introduces the nested hierarchical Dirichlet process for learning tree-structured topic hierarchies. Unlike the nested Chinese restaurant process, where a document uses topics along one root-to-leaf path, the nHDP gives each document its own distribution over paths in a shared tree, so individual words can take different paths and documents can mix themes from different branches. The authors derive a stochastic variational inference algorithm for it and demonstrate it on 1.8 million New York Times articles and roughly 2.7 million Wikipedia pages.
Standing in the record
Filed by the reading-time triage of 2026-09-25: 1,245 seconds of active reading over 2 sessions in the papers-feed tracker. It was filed Deferred, unread. NOTE-023 is the close reading of 2026-09-25, and it placed the work: Active — the reference construction of the nHDP and a compact worked example of conjugate-exponential SVI for a tree-structured nonparametric model. Two things about it: the global update equations (23, 26–27) as printed contradict the paper’s own §4.1.2 derivation, and the large-scale results never compare against the nCRP.
When seeded, before the reading in NOTE-023, the entry gave a priority for a deeper reading: medium — a well-cited Blei/Jordan-group model (owner t=1245 s); the skim captures the idea and results, and a deeper read is only worthwhile for the SVI derivation in §4.
What the seeding asked a deeper reading to check (NOTE-023 is that reading):
- The advantage over nCRP grows with corpus and document size in Table 2, but the Gibbs-nCRP beats variational nHDP on the smallest corpus; check how much of the large-scale gain is due to the model versus the inference method.
- Large-scale evaluation is held-out likelihood plus qualitative trees; no human evaluation of hierarchy quality was seen in the skim.
- ML link: a canonical example of stochastic variational inference applied to a deep Bayesian nonparametric model; relevant to history of scalable VI rather than to current practice.
Access when seeded: arXiv abs page and full v4 PDF (15 pp., 2 May 2014) read via arxiv.org; DOI confirmed via Crossref (TPAMI vol. 37, pp. 256-270, issue dated Feb 2015). Keywords are the paper’s Index Terms. Discrepancy noted: the arXiv listing abstract says 3.3 million Wikipedia documents, while the v4 PDF abstract and experiments say 2.7 million; the PDF figure is used here.
Note of 2026-10-26: the nested Chinese restaurant process, now held
The nested HDP takes a global nested Chinese restaurant process tree as
the base of every document’s tree, and the single path of the nCRP
returns in its β → 0 limit. Blei, Griffiths and Jordan’s JACM paper on the
nCRP and hLDA was registered and read on 2026-10-26 (LIT-1542,
NOTE-1176), for the exchange “Experimental Infrastructure Review”,
whose T21 moved from hLDA to the nested HDP because hLDA gives each
document one path. So this entry now extends it. The reading confirms
the restriction the nested HDP relaxes: in hLDA two documents share a
topic only by sharing the whole path above it (THEORY-969). Status
and summary are unchanged.