nucleation

LIT-1948: Latent Dirichlet Allocation

1 min read

Properties6
StatusActive — read in full 2026-11-03 (NOTE-1519), from the JMLR PDF; worth reading as the source of the per-document Dirichlet mixture that CLAIM-536 makes the difference between the owner's H1 and H2, and for the authors' own contrast with the one-topic-per-document mixture of unigrams. The structure is definitional and checked; the empirical superiority rests on perplexity on two corpora.
Version2
Tagsprobabilistic-modeling
Cited as “grounds” byCLAIM-536 — A categorical model of perspectives and an LDA-style model differ in where the mixture lives: the first draws every act's perspective from one fixed categorical distribution, the second gives each episode its own mixture, drawn from a Dirichlet, and draws each act from it; the Dirichlet is a distribution over mixtures, not the perspectives themselves, and with symmetric concentration a over K perspectives two acts in one episode share a perspective with probability (a + 1)/(Ka + 1) against 1/K across episodes, so the concentration runs from one perspective per episode, as in persona selection (a → 0), to a fresh independent draw for every act (a → ∞), CLAIM-543 — What separates a language model's distribution over personas from a distribution over organizational perspectives is not whether the latent model is hierarchical, since a mixture or an LDA-style admixture already is, but whether some latent components condition which others are available or consequential: a model whose components are exchangeable given their weights is organizationally flat however many levels its prior has, and dependencies among components count as organization (organization as relational arrangement) only where interventions, not fit, support them
Cited as “paper” byNOTE-1519 — Blei, Ng & Jordan — Latent Dirichlet allocation
Cited as “source” byTHEORY-1304 — In latent Dirichlet allocation each document draws its own topic mixture from a Dirichlet shared across the corpus and draws a topic afresh for every word, a three-level model of conditionally independent assignments; the mixture of unigrams, which draws one topic per document, is its two-level comparison, and the Dirichlet is a distribution over mixtures, not over topics

Graph View

Blei, D. M., Ng, A. Y., Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research 3(Jan): 993–1022. https://jmlr.org/papers/v3/blei03a.html.

Read in full on 2026-11-03 (NOTE-1519; THEORY-1304).

Key takeaways

  • θ_d, the topic mixture, is drawn once per document from Dir(α); a topic is drawn for every word (p. 996). Topics are the categories; the Dirichlet is over mixtures.
  • The authors’ own comparison is the two-level “Dirichlet-multinomial clustering model”, one topic per document (p. 997), i.e. the mixture of unigrams (§4.2).
  • Given θ_d the topic assignments are conditionally i.i.d.; nothing in the model lets one topic condition another.

Standing in the record

The model the owner’s U12 says H2 is. The record holds hLDA (LIT-1542), the nested HDP (LIT-002) and LIT-677, an ELBO paper about LDA, but not LDA itself. Grounds CLAIM-536.

Backlinks