Karl Friston and Will Penny (2011), NeuroImage 56(4):2089–2099 — DOI-10.1016/j.neuroimage.2011.03.062
Key takeaways
- One fit scores every reduced model. Define a model space by priors on one shared likelihood, each reduced model’s prior support inside the full model’s. Then p(y|m_i) = p(y|m_F) ∫ p(θ|y, m_F) p(θ|m_i)/p(θ|m_F) dθ (Eq. 3), and the reduced posterior is the full posterior reweighted by the same prior ratio (Eq. 4). Only the parameters whose priors change need integrating (Eq. 5).
- Savage–Dickey is the point-mass case. When the reduced prior fixes a subset at a value, Eq. 5 becomes the posterior-to-prior density ratio at that value (Eq. 6).
- Closed form under Laplace. With Gaussian prior N(η, Π⁻¹) and posterior N(μ, P⁻¹), the reduced posterior precision is P_i = P_F + Π_i − Π_F, its mean is C_i(P_Fμ_F + Π_iη_i − Π_Fη_F), and the reduced free energy is a log-determinant term plus quadratic forms plus F_F (Eq. 9). A parameter “removed” by shrinking its prior variance to zero stops contributing.
- Selection and optimisation are one operation. Choosing which parameters to switch off and tuning prior variances (ARD), empirical Bayes, ReML, MAP and ML are all maximisation of evidence over the hyperparameters of the prior (Eq. 13, Table 1). With no constraint on the prior, the optimum collapses to a point mass at the maximum-likelihood value.
Standing in the record
Filed on 2026-10-03 at the owner’s request, in the model-comparison batch. No anthology topic holds it. It is a statistical method presented through neuroimaging (dynamic causal models of brain networks), and it carries no instruction for machine-learning practice.
It is the founding paper of Bayesian model reduction. It extends Dickey (LIT-610) in the strict sense: its Eq. 6 recovers the Savage–Dickey ratio, which it cites to Dickey (1971) and Verdinelli & Wasserman (1995), as the point-mass limit of its own Eq. 5, and the rest of the paper is what the general identity buys beyond that limit. Dickey’s own statement (his Eqs. 3.8–3.9) is already general in the likelihood and in the number of parameters fixed; what this paper generalises is the reduced prior, from a point mass to any narrower prior. Wagenmakers et al. (LIT-609) give the same ratio as their Eq. 12. Their Eq. 12 and this paper’s Eq. 6 are the same identity reached from opposite ends, but their conditions are not alternatives. Dickey’s theorem uses both a likelihood shared by the two hypotheses and a nuisance prior under the null equal to the alternative’s conditional prior at the null value (his 3.8), which is Wagenmakers et al.’s continuity condition. Here the second is carried by the construction of the reduced prior, with the prior on the other parameters unchanged.
The review that extends it, Friston, Parr & Zeidman’s Bayesian model reduction (LIT-614), adds the Dirichlet, beta, gamma and multinomial forms. Those are what let BMR prune discrete generative models, as Smith et al. (LIT-615) do for concept learning.
For the record’s free-energy holdings: the “free energy” here is the variational bound on log evidence of Eq. 7, accuracy minus complexity. It is the same functional as in Friston’s Life as we know it (LIT-526), and there too it is variational, not thermodynamic. What this paper adds to the record’s reading of the free-energy literature is the statistical use of that bound, comparing models, which the record had not held. Nothing in it bears on the Markov-blanket critiques (THEORY-066, THEORY-073): no blanket appears.