nucleation

LIT-1628: Factorial Hidden Markov Models

1 min read

Properties6
StatusActive — read 2026-10-27 (NOTE-1203) in full, from the authors' copy of the Machine Learning article, checked against the publisher's PDF; worth reading as the origin of the factorial hidden Markov model, the parametric model of several independent hidden chains jointly producing each observation that the infinite factorial HMM makes nonparametric, for its account of why exact inference costs O(TMK^{M+1}), and for its structured variational approximation. Read it knowing that its comparison of inference methods is on small synthetic problems (THEORY-1045), that its Bach chorale result is read from charts, and that its footnote on when the chains decouple is wrong for a non-isotropic covariance.
Version2
Tagsprobabilistic-modeling, representation-learning
Cited as “extends” byLIT-1577 — The Infinite Factorial Hidden Markov Model
Cited as “paper” byNOTE-1203 — Ghahramani and Jordan — Factorial hidden Markov models
Cited as “source” byTHEORY-1045 — On synthetic data from factorial hidden Markov models with three or five hidden chains of two or three states, an unconstrained hidden Markov model with as many states as the chains have joint settings overfits, its test-set excess over the true model rising to 175 bits per observation at five three-state chains against 4.5 for the factorial model trained exactly, while factorial models trained with completely factorized or structured variational E steps are not significantly worse than exact training

Graph View

Ghahramani, Z. & Jordan, M. I. (1997). Factorial hidden Markov models. Machine Learning 29(2–3): 245–273. doi:10.1023/A:1007425814087. An earlier version: Advances in Neural Information Processing Systems 8 (1995).

Read in full on 2026-10-27 (NOTE-1203). One account was drawn from it: THEORY-1045.

Why this is here

The owner asked on 2026-10-27 to “file the next readings”. The infinite factorial HMM (LIT-1577) is this model with the number of binary chains left unbounded under a Markov IBP prior, and the record held it without its parametric original. CASE-165’s model of members’ features, each feature a two-state chain over days and the output a sum of the active features’ contributions, is a factorial HMM in structure, so this paper’s account of why the chains couple given the data, and of what exact and approximate inference cost, bears on what fitting that model would take.

Backlinks