nucleation

LIT-1249: Slow Feature Analysis: Unsupervised Learning of Invariances

1 min read

Properties5
StatusActive — read 2026-10-20 (NOTE-971) in full, from Wiskott's own PostScript preprint; worth reading for the slowness objective stated exactly (minimize ⟨ẏ²⟩ under zero mean, unit variance and decorrelation), for its closed-form solution within a fixed nonlinear expansion by sphering and an eigendecomposition of the derivative covariance (THEORY-746), and for the demonstration that stacked modules learn translation, size, rotation, contrast and, less well, illumination invariance for 1-D patterns from the training stimulus alone. Read it knowing that optimality holds per module, on the training signal and within the chosen function space, and not for the greedy hierarchy; that "slower is more abstract" is its premise, not a finding, so a stack built this way is slower at higher stages by construction; that the slowest outputs include location, so slowness is not invariance; that no generative model and no learning dynamics appear; and that every stimulus is synthetic and one-dimensional.
Version3
Tagsrepresentation-learning, neuroscience, anthology-candidate
Cited as “paper” byNOTE-971 — Wiskott and Sejnowski — Slow feature analysis
Cited as “source” byTHEORY-746 — Over a fixed finite set of basis functions, slow feature analysis is a generalized eigenproblem: the outputs that minimize ⟨ẏ²⟩ under zero mean, unit variance and decorrelation are the bottom generalized eigenvectors of the derivative covariance against the covariance, in order of slowness, and a stack of such stages, each of whose function spaces contains its inputs, can only keep or lower each of its first m slowness values on the training signal, and may keep them, THEORY-775 — With a Gaussian random-walk prior and an instantaneous unit-variance Gaussian encoder, the KL half of the evidence lower bound is the unordered slowness loss plus a constant; with a linear Gaussian encoder and decoder on whitened input, the bound is maximized on the span of the slowest SFA directions whose lag-one autocorrelation exceeds one half, with the basis inside that span unidentified, and on unwhitened input the maximizer is neither SFA's subspace nor invariant to rescaling the input

Graph View

Wiskott, L. & Sejnowski, T. J. (2002). Slow Feature Analysis: Unsupervised Learning of Invariances. Neural Computation 14(4): 715–770. doi:10.1162/089976602317318938.

Read in full in NOTE-971. The closed form is re-derived in THEORY-746.

Why this is here

A2 recommended it as a route from temporal regularity to learned features, distinct from explicit relevance targets: under which generative assumptions are slow features also coarse, predictively sufficient or quickly learned? Bears on QUESTION-217. LIT-252 and NOTE-824 treat slow feature analysis as the graph-Laplacian case of spectral inference networks. Flagged anthology-candidate because it is a representation-learning method; it carries no instruction for practice.

Backlinks