Contribution
It puts Bayesian design on one footing, Lindley’s (1972) decision-theoretic formulation. Many criteria in use become special cases of maximizing expected utility, and others are shown to have no utility behind them. It surveys linear, analysis-of-variance and nonlinear designs within that frame and says which approximations are being made. It also says where practice lags: “Apart from Flournoy (1993) there are no true ‘case studies’”.
Key insight
A design is chosen before the data, so it is the one place in Bayesian theory where averaging over the sample space is legitimate: “Experimental design is the only situation where it is meaningful within the Bayesian theory to average over the sample space.” The right criterion is then fixed by what the experiment is for. “A design that is optimal for estimation is not necessarily also optimal for prediction.” So choosing among D, A, c and other criteria is choosing a utility, not a matter of taste.
Assumptions
- Lindley’s formulation (§1.3). A design η is chosen from a set ℋ, data y are observed, and a terminal decision d is taken, with utility U(d, θ, η, y). The design maximizes U(η) = ∫ max_d ∫ U(d, θ, η, y) p(θ | y, η) p(y | η) dθ dy.
- Approximate designs (§1.4). A design is a probability measure η on a compact design region X, with the integrality of nη_i relaxed. The information matrix is nM(η) = n Σ η_i x_i x_iᵀ.
- The linear model (§2.1). y | θ, σ² ~ N(Xθ, σ²I) with σ² known, and prior θ | σ² ~ N(θ₀, σ²R⁻¹), R known. The posterior covariance is σ²D(η) = σ²(nM(η) + R)⁻¹.
- Nonlinear models (§4.2). Normal approximations to the posterior, using the expected Fisher information nI(θ, η), with or without R, and the prior used as the predictive distribution of θ̂.
Key results
- Bayes D-optimality (§2.2). Expected Shannon information of the posterior, equivalently expected Kullback–Leibler divergence from prior to posterior, reduces in the linear model to maximizing φ₁(η) = det{nM(η) + R}. Spezzaferri’s scoring-rule utility and Tiao and Afonja’s two-valued utility lead to the same criterion. Re-derived in THEORY-956.
- Bayes A- and c-optimality (§2.2). Quadratic loss (θ − θ̂)ᵀA(θ − θ̂) gives expected utility −σ² tr{AD(η)}. With A = ccᵀ this is c-optimality, the criterion for one linear combination cᵀθ, or for prediction at a fixed point.
- E- and G-optimality “appear not to correspond to any utility function”, so their Bayesian justification “is unclear”.
- The prior’s weight (§2.2). D(η) = n⁻¹(M(η) + n⁻¹R)⁻¹, so Bayesian and non-Bayesian linear designs differ little when n is large or R small. A proper prior keeps nM + R nonsingular even when nM is singular.
- Unknown variance (§2.5). With a conjugate normal–inverse-gamma prior the D and predictive criteria have no closed form. Bayes A-optimality does not depend on what is known about σ², which makes it robust in that sense.
- ANOVA (§3). The Bayes A-optimal allocation to a control and t − 1 treatments tends to the square-root rule as the priors become vague, and can put no observations on the control when its prior is precise. Hierarchical (exchangeable) priors change the optimal designs: as the treatments are believed more alike, half the observations go to the control.
- Nonlinear criteria (§4). φ₁(η) = ∫ log det{nI(θ, η)} p(θ) dθ (Bayes D), and φ₂(η) = −∫ c(θ)ᵀ{nI(θ, η)}⁻¹c(θ) p(θ) dθ for a function g(θ) with gradient c(θ) (Bayes c). Versions with R added inside are also given. The authors prefer the versions without R: they correspond to designing with an informative prior and analysing with a noninformative one.
- Local optimality (§4.4) is the crude approximation that replaces the prior by a point mass at a best guess θ₀ (Chernoff).
- The equivalence theorem (§5.1). For a criterion that averages a concave local criterion over the prior, a design is optimal iff the directional derivative toward every one-point design is nonpositive. For Bayes D: d(η, x) = ∫ tr{I(θ, η_x) I(θ, η)⁻¹} p(θ) dθ − k ≤ 0. It holds under stated regularity conditions (a finite criterion, continuity on ℋ, continuous derivatives).
- Support points (§5.2). In linear problems Carathéodory bounds the number of support points, and D-optimal designs often have as many as there are parameters. For nonlinear Bayesian designs there is no such bound, and in the reported examples the number grows as the prior becomes more dispersed (Chaloner and Larntz).
- Sequential design (§5.5). In linear problems with σ² known there is “nothing to be gained by designing sequentially”, because the posterior utility depends on the design and not on the data. In nonlinear problems there should be a gain.
- Turning point (§7.2). For estimating −θ₁/(2θ₂) in a quadratic regression on [−1, 1] with best guess ½, the locally optimal design puts half its mass at 0 and half at 1. That gives a singular nM of rank 2, from which the quadratic, and hence the turning point, cannot be fitted: “The locally optimal design is therefore useless”. Re-derived in THEORY-954.
- Model discrimination (§8.3). Spezzaferri’s Bayesian criterion for choosing between two linear models minimizes the expected posterior probability of one model when the other is true. For nested normal models under diffuse priors it coincides with D-optimality for testing that the extra parameters are zero. For discrimination and estimation together, the criterion is a product of the smaller model’s information determinant and that expected posterior probability.
- Mixtures of models (§8.2). From a Bayesian standpoint one averages expected utilities, not efficiencies: “expected utility should be maximized, not expected efficiency.”
- Examples 2 and 3 (§§6.2, 6.4). The designs the laboratories actually used were close to Bayes-optimal under the more dispersed priors. For example, the six-dose bioassay had a φ₂ value 1.13 times the optimum under a uniform prior, against 1.52 under a Beta(4,4) prior. Designs efficient for one quantity, such as AUC, can be very inefficient for others (t_max, C_max).
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | Bayesian design is maximization of expected utility, and the criterion should follow from the experiment’s purpose | strong as a framework | Lindley’s formulation; the review’s organization |
| C2 | In the normal linear model with known variance, Shannon information gives Bayes D-optimality and quadratic loss Bayes A-optimality | strong | §2.2; re-derived in THEORY-956 |
| C3 | E- and G-optimality have no utility-based justification | moderate | stated, without a proof that none exists |
| C4 | Nonlinear Bayesian criteria are asymptotic approximations, and local optimality is the point-mass special case | strong | §4.2–4.4 |
| C5 | Locally optimal designs can make the target quantity inestimable | strong for the example | §7.2; re-derived in THEORY-954 |
| C6 | Nonlinear Bayesian optimal designs gain support points as the prior disperses | moderate | examples cited (Chaloner and Larntz 1986, 1989 and others), no general theorem; “It is unclear, however, under what circumstances this is so” |
| C7 | Sequential design gains nothing in linear known-variance problems and should gain in nonlinear ones | strong for the first, moderate for the second | §5.5 argument |
| C8 | Practice lags theory: almost no real experiment has been designed this way | moderate | the authors’ survey as of 1995 |
Concepts
- Design measure η: the proportions of observations at each design point. It is approximate when the integrality of nη_i is relaxed.
- Information matrix nM(η): XᵀX = n Σ η_i x_i x_iᵀ. The Fisher information is σ⁻²nM. In nonlinear models nI(θ, η) depends on θ.
- Bayes D-, A-, c-, E-, G-optimality: the criteria above, with R inside.
- Local optimality: optimal for a single best guess θ₀.
- Equivalence theorem: the directional-derivative check of global optimality over ℋ.
Connections
It follows Raiffa and Schlaifer (1961) and Lindley (1956, 1972), with Stone, DeGroot and Bernardo (1979) on Shannon information as utility. It gives the Bayesian side of Kiefer and Wolfowitz’s alphabetical theory (Fedorov 1972, Silvey 1980, Pukelsheim 1993) and of Elfving’s theorem (Chaloner 1984), and Whittle’s (1973) form of the equivalence theorem. DasGupta (1995) is named as a complementary review. Berry and Fristedt (1985) on bandit problems is the sequential literature it does not review.
Bearing on the record
- QUESTION-031’s note of 2026-10-25, checked. The note’s G_ξ = Σ_c ξ(c)G_c is the review’s nM(η) = n Σ η_i x_i x_iᵀ, generalized to the expected Fisher information nI(θ, η) of §4.2. Its log det is Bayes D-optimality (THEORY-956). A29’s point that the task fixes the criterion is the review’s central thesis. Distinguishing two effects, rather than estimating every parameter, is c- or Ds- optimality, or Spezzaferri’s discrimination criterion. The review does not answer the question as posed. Design chooses among probe families by a criterion. It gives no condition under which a family separates what a decision family needs.
- CASE-159’s design problem. The case is a nonlinear design problem in the review’s sense: its information depends on the share being estimated, I_rare(π) = 0.64/(p(1 − p)) with p = 0.1 + 0.8π. So a design must average over a prior on π (§4.2) or fix a best guess (§4.4). In the two-specialist case the answer does not depend on π: the routine context has zero information at every π, so every criterion that increases with information puts all observations on the rare context. With three processes the question is nontrivial. In an illustration computed here, with four binary contexts (one where all agree, two that separate different pairs, one nearly redundant) and π = (0.5, 0.3, 0.2), the D-optimal design puts weight ½ on each of the two separating contexts. Its sensitivities tr(G_ξ⁻¹G_c) are 2, 2, 0 and 0.01 against the dimension 2, which is the equivalence theorem’s certificate. Its log det is −0.42, against −1.80 for the uniform design and −5.02 for a design spending 85% on the routine context. Each binary context contributes information of rank one, so at least k − 1 contexts with non-parallel profile differences are needed. The construction is the record’s.
- Sequential probing. Because CASE-159 is nonlinear in this sense, choosing probes adaptively should gain over a fixed design (§5.5). That is the design-side counterpart of the exploration results in NOTE-1164 and NOTE-1159.
- CLAIM-130. The review treats ethical constraints only through the clinical examples (“the choice of the treatments to be compared is a central ethical issue”). It has nothing on a probe that changes the act it measures, the caution A29 raised.
- CLAIM-450, CLAIM-444. Nothing here bears on the garbling inequality. That nM(η) adds over independent observations is the additivity CASE-159’s sum over contexts assumes.
Statistical theory read for itself. No instruction for ML practice.
Limitations
- A 1995 review. Simulation-based and MCMC design (Müller and Parmigiani are named as undemonstrated) and everything after are absent.
- Mostly normal linear models. Nonlinear results are approximations whose accuracy is “not much known” in general.
- No general theory of support points for nonlinear Bayesian designs.
- Discrimination is brief: two linear models, Spezzaferri’s utility.
- Technical-report copy, with equations lost to OCR. It was not compared with the published version.
Open questions
- Which utility, if any, justifies E-optimality? The review leaves it open.
- For a mixture model whose components’ response profiles are known, as in CASE-159, what is the Bayes-optimal adaptive probing policy, and how much does it gain over the best fixed design?