Turner, R. & Sahani, M. (2007). A Maximum-Likelihood Interpretation for Slow Feature Analysis. Neural Computation 19(4): 1022–1038. doi:10.1162/neco.2007.19.4.1022.
Not read. The summary is drawn from the abstract. A reading should check it first.
Key takeaways
- From the abstract only: there is a probabilistic model whose maximum- likelihood learning and inference, “in the limiting case”, give exactly slow feature analysis (LIT-1249), and the model is used “as a conceptual spring-board” for extensions of the algorithm.
- What the model is (the abstract does not say whether the latents are linear Gaussian autoregressive processes or how the observation map is set) and which limit is taken are not established here.
Why this is here
NOTE-971 read Wiskott and Sejnowski’s slow feature analysis and found no generative model in it. It named the probabilistic follow-ups as the place to look for when the slowest functions recover the latent variables of a generating process. This is the first of them. It is wanted for three things:
- QUESTION-217: under which generative assumptions slow features are coarse, predictively sufficient or quickly learned. If the model is a linear Gaussian state space with Markov latents, the reading should say whether the slow features are then the maximum-likelihood estimate of a Markov state, and so predictively sufficient for the observations, or only of each latent’s own past.
- CLAIM-331: whether slowness is a property the model assumes of its latents, which would make a slow feature a modelling choice rather than evidence that higher-level variables are slower.
- THEORY-733 and THEORY-257: whether the slow features of such a model coincide with the invariant subspaces of Aᵀ that make a linear coarse-graining of a VAR(1) exactly dynamically independent.