nucleation

LIT-1577: The Infinite Factorial Hidden Markov Model

2 min read

Properties7
StatusActive — read 2026-10-27 (NOTE-1199) in full, from the NeurIPS proceedings' own PDF; worth reading as the origin of the Markov Indian buffet process, the IBP's extension to features that switch on and off over time, in which each feature has its own probability of turning on and, drawn independently of it, of staying on (THEORY-1034). Read it knowing that its sequential buffet misprints both of its predictive rules, that after the first time step the expected number of active features is no longer the IBP's α (THEORY-1019), that it models one sequence and not several objects over time, and that its experiments are two single runs on mixed speech.
Version3
Tagsprobabilistic-modeling, mathematical-statistics
ExtendsLIT-1623 — Infinite Latent Feature Models and the Indian Buffet Process, LIT-1628 — Factorial Hidden Markov Models
Compared againstLIT-1626 — Infinite Sparse Factor Analysis and Infinite Independent Components Analysis
Cited as “paper” byNOTE-1199 — Van Gael, Teh and Ghahramani — The infinite factorial hidden Markov model
Cited as “source” byTHEORY-1019 — Under the Markov Indian buffet process the expected number of features on at the first time step is α, as in the Indian buffet process, but it rises over time toward α·E[ln((2 − b)/(1 − b))] with b ~ Beta(γ, δ), which is 9.04 at γ = 10 and δ = 1, so after the first step the rows are not distributed as an Indian buffet process, THEORY-1034 — In the Markov Indian buffet process each feature's probability of staying on is drawn independently of its probability of switching on, so how long a feature persists once on does not depend on how rare it is; the process is defined for the time steps of one sequence, not for several objects over time

Graph View

Van Gael, J., Teh, Y. W. & Ghahramani, Z. (2008). The infinite factorial hidden Markov model. In D. Koller, D. Schuurmans, Y. Bengio & L. Bottou (eds.), Advances in Neural Information Processing Systems 21, pp. 1697–1704. Curran Associates.

Read in full on 2026-10-27 (NOTE-1199). Two accounts were drawn from it: THEORY-1034 and THEORY-1019.

Why this is here

The owner asked on 2026-10-27 to “get started on the seven works and markov IBP”. CASE-165 (item 2) names the Markov IBP as the established relative of the sticky transition in its organizational feature model, the one that “gives each feature its own probabilities of turning on and staying on”, and NOTE-1172 calls it the closest relative of that term. Both cite it only through the IBP review (LIT-1556), and the record held neither the paper nor its year correctly. It is the reference for whether feature persistence can be a per-feature parameter in an IBP-type prior, and for what such a prior does to the number of active features over time (QUESTION-276).

Note of 2026-10-27 (later): its antecedents, now held

The works this paper builds on were registered and read on 2026-10-27, at the owner’s request (“file the next readings”), and its relations now name them.

  • extends LIT-1623, Griffiths and Ghahramani’s NIPS 2005 paper, which introduces the Indian buffet process. The curation entry of 2026-10-27 02:06 said this paper should extend it once registered. The Markov IBP’s limit argument is that paper’s, with time steps for objects.
  • extends LIT-1628, Ghahramani and Jordan’s Factorial Hidden Markov Models. The infinite factorial HMM is that model with an unbounded number of binary chains (NOTE-1199: “a factorial hidden Markov model whose number of hidden chains is learned”).
  • compared_against LIT-1626, Knowles and Ghahramani’s infinite ICA. The speech experiment runs the same model with an IBP prior, which is that paper’s iICA, and reports nine speakers found against five.

It also uses the stick-breaking construction, LIT-1627, and its slice sampler with forward filtering, backward sampling per chain is the beam sampler’s scheme, LIT-1622 (THEORY-1037). These are uses of a method, not relations the vocabulary holds, so they are noted here only. The status is unchanged.

Backlinks