Contribution
The journal version of the author–topic (AT) model of UAI 2004 and KDD 2004. Authors are distributions over topics, topics distributions over words, and a document with several authors is a uniform mixture of its authors’ topic distributions. It adds to the conference papers a systematic comparison with LDA, a topic-free author model and AT with a fictitious author per document, an information-retrieval experiment, the Enron corpus, and a reading of all three models as matrix factorizations (Appendix C).
Key insight
Put the mixture weights on the authors instead of on the documents, and documents by the same author borrow strength from each other. So the AT model predicts a document it has barely seen better than LDA can, and loses to LDA once enough of the document is seen, because a document is “assumed to be generated by a mixture of the authors’ topic distributions and nothing else.” The trade between member-level pooling and document-level flexibility is the paper’s central empirical finding, and a fictitious author per document is its way of buying some of both.
Assumptions
- Bag of words, stop words removed by a standard list.
- Authorship is observed. “For the purposes of estimation, we assume that the set of authors of each document is observed. This leaves unresolved the issue of having unobserved authors”. a_d is the set of authors of document d.
- A fixed number of topics T. Bayes factors or nonparametric priors are mentioned as ways to choose it, and not used.
- Generative process. θ_a ~ Dirichlet(α) for each author, φ_t ~ Dirichlet(β) for each topic; for each word of document d, an author x ~ Uniform(a_d), a topic z ~ Discrete(θ_x), and the word w ~ Discrete(φ_z).
- Fixed hyperparameters, α = 50/T and β = 0.01 in every experiment.
- Each author’s θ_a lies on the simplex: an author’s topics are a mixture, and their weights sum to one.
Key results
- Gibbs conditional (3). For a ∈ a_d, P(x = a, z = t | rest) ∝ (C^WT_wt + β)/(Σ_w′ C^WT_w′t + Wβ) · (C^TA_ta + α)/(Σ_t′ C^TA_t′a + Tα), counts excluding the current word. Derived in Appendix A by integrating out Φ and Θ; re-derived here from Dirichlet–multinomial conjugacy. Author and topic are sampled jointly (“blocked”), which mixed better in their runs. Cost per iteration O(N A_max T).
- Point estimates (6)–(7). Posterior means of φ_t and θ_a from the counts of one sample, and predictions averaged over the last samples of S = 10 independent chains. Expectations of nonlinear functions are approximated by plugging in the means, “exact when f is linear”.
- Identifiability (§3.2). “Like any mixture model, the author-topic model suffers from a lack of identifiability: the posterior probability of Φ and Θ is unaffected by permuting their columns.” So samples from different chains are not averaged column by column.
- Corpora (§4). NIPS 1987–1999 (D = 1,740, A = 2,037, N = 2,301,375, W = 13,649); CiteSeer (D = 150,045, A = 85,465, N = 10,810,003, W = 30,799); Enron (D = 121,298 emails, A = 11,195, N = 4,699,573), with quoted replies and attachments removed. Ten chains of 2,000 iterations each: 12 hours for NIPS at T = 100, about 200 hours for CiteSeer at T = 300, 23 hours for Enron at T = 200.
- Convergence and stability (§4.1–4.2). Test perplexity on 7,502 CiteSeer documents levels off after about 100 iterations, and averaging 10 chains lowers it markedly. Greedy alignment of two 100-topic NIPS samples by symmetrized KL: “Roughly 80 of 100 topics have a reasonable degree of correspondence that would be associated with the same subjective interpretation”, and the worst-aligned pair “shows no correspondence at all”.
- Enron (§4.4, Figs. 7–8). Topics on energy business, the California energy crisis, Texas sports and Christianity. Institutional accounts are nearly single-topic (0.942 for Enron General Announcements, 0.991 for the Outlook Migration Team). Three individuals, unnamed, are described from their four highest topics, and “the author-topic model can learn about the roles and interests of different individuals from text that they have written.” Judged by inspection.
- Unusual papers (§5.1, Table II). Scoring each CiteSeer abstract under M Jordan, D Koller, T Mitchell and S Russell as sole author: the most perplexing were by namesakes (Mick Jordan, Daniel and David Koller, Toby Mitchell, Stephen Russell).
- Parsing a two-author pseudo-abstract (§5.2, Fig. 9). Words of a combined Schölkopf and Darwiche abstract were assigned to the right author 69% and 72% of the time.
- Perplexity against other models (§6.4, Fig. 11). NIPS, 1,557 training papers and 183 test papers (102 single-authored), every test author also a training author, T = 100, 10 chains. The author model is far worst. AT beats LDA up to 16 observed test words and loses above. AT with a fictitious author is “systematically better” than AT, and loses to LDA “after 64 words or so”.
- Retrieval (§6.5, Fig. 12). Extended NIPS 1987–2003 (D = 2,483), 200 queries against 2,283 documents, relevance by the 13 NIPS categories. With title and authors as the query, AT beats LDA and BM25. With the full text, LDA is marginally best.
- Factorization (Appendix C). The W × D matrix of word probabilities is P = ΦΘA, with A the A × D matrix taking 1/A_d where a ∈ a_d. LDA merges ΘA into one T × D matrix; the author model merges ΦΘ into one W × A matrix.
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | The AT model’s word distributions factor as P = ΦΘA, with LDA and the author model as the special cases that collapse one pair | strong | Appendix C; §6.1–6.2; re-derived in THEORY-972 |
| C2 | The model is unidentified up to permutations of topics | strong as far as it goes; the likelihood’s freedom is larger (see Limitations) | §3.2 |
| C3 | AT predicts sparsely observed documents better than LDA, and worse once more than about 16 words are seen | moderate | Fig. 11, one corpus, one split |
| C4 | A fictitious author per document improves AT throughout | moderate | Fig. 11 |
| C5 | With only titles and authors as queries, AT retrieves same-category NIPS papers better than LDA and BM25 | moderate | Fig. 12, one corpus |
| C6 | Topic solutions are “relatively stable across samples” | weak to moderate | one pair of NIPS runs, greedy alignment, a subjective threshold |
| C7 | The model learns roles and interests of individuals from what they write | weak | Enron examples by inspection, with no validation against the individuals’ actual roles |
Concepts
- Author: any observed source whose presence constrains the topics of a document. §7 generalizes it to citations, venues, years and affiliations by adding them to a_d.
- Fictitious author: one extra author unique to each document, giving it a document-specific topic mixture, between AT and LDA.
- Perplexity: exp of the negative mean log-probability per test word, conditioned on the test document’s known authors.
Connections
It extends LDA (Blei, Ng and Jordan 2003) and its collapsed Gibbs sampler (Griffiths and Steyvers 2004), and McCallum’s (1999) mixture of authors over words. It contrasts itself with stylometry, which identifies who wrote a text from style, and with network models of co-authorship, which use links and not words. McCallum et al.’s author–recipient–topic model for email is cited as the extension that conditions topics on recipients as well. The record holds the nonparametric author–topic model as the baseline of LIT-013’s Twitter-Network Topic Model (NOTE-008).
Bearing on the record
- QUESTION-276, condition 2 (overlap). This is the plain-text case of inferring members’ overlapping participation, and its overlap is of a particular kind. An author’s θ_a is a point on the simplex, so taking more part in one theme means taking less part in the others. T19 asked for participation with “no requirement that ∑_p w_ep,t = 1”, and the AT model cannot give it. An author strongly in two themes and an author weakly in two look the same if their proportions match. A feature allocation (T22’s Indian buffet prior, CASE-165) records having a theme separately from how much of one’s output it takes. The two are different models of overlap, and the question names both.
- QUESTION-276, “without their number being fixed”. T is fixed here. LIT-013’s model (NOTE-008) is the nonparametric form.
- QUESTION-276, condition 4, and T21’s falsification test. The likelihood depends on the words and the authorship only. Two organizations with identical messages and authorship and different authorization pathways get the same posterior, so the AT model must fail T21’s test, as T21 predicted of any content-only model. It is a candidate for the “communication relations alone” side of the comparison only in so far as authorship is a relation.
- CLAIM-148. Appendix C makes the AT model a three-factor nonnegative factorization with one factor known. CLAIM-148’s NMF gauge applies to the other two, and it is larger than the permutations §3.2 names: a continuum of column-stochastic changes of topic basis leaves the likelihood unchanged (THEORY-972). The empirical face of this is §4.2’s alignment, where about a fifth of the topics had no counterpart in a second run.
- CLAIM-135. The paper reaches the first achievement (fit, by perplexity) and argues stability across runs. It claims identification only up to permutation and does not show even that. Its Enron reading of “roles and interests” is the step to constitution, and it is made by inspection.
- CASE-165. The fictitious author is the AT model’s version of that case’s W^private, the per-employee term outside any shared feature. CLAIM-148’s note of 2026-10-26 found that a feature one employee holds is absorbed into the private map. Here, likewise, a topic used by only one document can be carried by its fictitious author or by a real author’s θ_a, and only the priors choose. Fig. 11 shows the fictitious author improving prediction. It does not show what it takes from the authors’ profiles.
- CLAIM-454 and TERM-072. An author’s topic distribution is what they write about. The paper’s Enron reading of three employees’ “roles” from their topics is the move the claim says an allocation cannot license by itself. The institutional accounts, nearly single-topic, are CASE-101’s kind of coactivity: many emails alike because one source sends them.
No instruction for machine-learning practice; nothing for the anthology.
Limitations
- Identifiability understated. §3.2 names permutations only. For the likelihood, any invertible R with column sums one, ΦR⁻¹ ≥ 0 and RΘ ≥ 0 gives the same P = ΦΘA. With every entry of Φ positive, as every posterior mean with β > 0 is, such R include a continuum near the identity (re-derived and checked in THEORY-972). Only the Dirichlet priors separate those parameter values.
- Authorship must be known. Unobserved or partial authorship, and any dependence between authors, are outside the model.
- Fixed T and fixed hyperparameters. No sensitivity analysis.
- Stability by one pair of runs and a subjective correspondence threshold.
- Evaluations on NIPS only for the model comparison and retrieval. Enron is used for illustration, with no quantitative test.
- Perplexity is not interpretability, and the readings of authors’ roles are by inspection.
Open questions
- Which conditions on the authorship matrix A and the topics make Φ and Θ identified up to permutation? A has many columns per author, which may help, as anchor-word conditions do for NMF. The paper does not ask.
- How much of a member’s θ_a does a fictitious author absorb, and does that change the ranking of members by theme?
- With participation recorded as features instead of proportions, does the advantage on sparsely observed documents survive?