nucleation

THEORY-1096: Blackwell's comparison of experiments, its easy direction: an experiment is a family of distributions indexed by a common parameter, and if one experiment is a garbling of another by a Markov kernel, every risk attainable with the garbling in any decision problem with bounded losses is attainable with the original, on any outcome spaces

3 min read

Properties5
StatusActive
Tagsmathematical-statistics, information-theory, mathematics
SourceLIT-781 — Equivalent Comparisons of Experiments
ExtendsTHEORY-156 — For experiments on a finite parameter set whose outcome spaces are standard Borel, being at least as informative for every decision problem is the same as being able to simulate the other by a randomisation
Cited as “grounds” byCLAIM-050 — For report fidelity, fidelity is relative to the communicative decisions receivers make, and Blackwell's order on rendering procedures, over a designed distribution of situations, makes precise the ceiling a rendering sets on those decisions, CLAIM-125 — Directed transport between empirical models whose covers change has two open problems: when it preserves decision-relevant information, and how it extends to signalling data, CLAIM-126 — Model stitching tests whether a transport preserves decision-relevant information only in its reconstruction form: a map fitted to reproduce the target representation certifies one direction of Blackwell sufficiency, while task-loss stitching compares one decision problem through a fixed head, and a failure to stitch is no evidence against sufficiency, CLAIM-199 — Blackwell's order bounds what a rendering makes possible for any receiver, and a given receiver's access is usable information relative to the rules it can apply, which a garbling can raise for that receiver but never above what the source makes possible, CLAIM-212 — Fidelity as a report and fidelity as a re-performance are different decision targets: Blackwell's order compares a rendering with its source as evidence about the source's situation, which both readers decide about, while a rendering that re-performs the act in the target's situation is compared only through a correspondence of situations, CLAIM-218 — The task-paired risk difference the record uses in place of Blackwell's order, D_Q = sup_q |R_q(U) − R_τ(q)(V)|, is a different quantity: it reduces each risk set to one number, needs the losses of paired tasks on one scale, and its absolute value counts improvement as distortion, CLAIM-225 — Blackwell's order measures what an ideal decision-maker could do, not what a given receiver can reconstruct, and the line's claims that a rendering can improve access describe garblings that real receivers do better with, which is the shape of the Blackwell thesis's own defeat condition, CLAIM-228 — D_obs restated over a rendering procedure, with one kernel family fitted on some source items and scored on held-out ones and the target term the readers' response averaged over the procedure's output, separates rendering procedures that receivers' task-indexed recovery separates, so restated it measures observational fidelity informatively, CLAIM-237 — The first open problem of directed transport is not yet posed: a contextual model is a menu of incompatible experiments, and under the natural order on menus any state-independent classical simulation is dominated automatically, so what is open is only the converse or a deficiency bound, CLAIM-258 — Blackwell's order compares experiments about one parameter, so it does not reach the normal translation case: source and target readers decide about situations with changed coordinates, and where the state is common a translator with context knowledge yields a rendering that is not a garbling of the source, which the order typically leaves incomparable with it and about which the direction the manuscript uses is silent, CLAIM-525 — Information in the sense the Simondonian readings give it, a disparity whose resolution reorders the system that takes it up, is in the line's terms a change in the interpreter or in the questions receivers can decide, not evidence about a fixed situation, so Blackwell's order leaves it out by construction and the two accounts are not rivals, CLAIM-535 — An organization that acts on what it observes has a policy acceptable in every circumstance exactly when, for each observation, the circumstances it cannot tell apart share an acceptable action; so every circumstance having an acceptable action does not give a policy, a finer observation never removes one, and where acceptable actions overlap the coarsest observations that suffice are several and incomparable, so which distinctions must be made legible depends on which acceptable action the organization commits to

Graph View

Source

Blackwell (1953), LIT-781, read in NOTE-595: the definitions in §1 and Theorem 3, “α ≻ β implies α ⊃ β”. The paper quotes the result from Blackwell (1951), unread here. The re-derivation below does not depend on either proof.

Definitions

  • Experiment. For a parameter set Θ, an experiment α is a family (α_θ)_{θ∈Θ} of probability measures on one measurable outcome space X. THEORY-156 quotes it: “an n-tuple of probability measures on a common space, one per state”. Two experiments compared must share Θ, and they need not share an outcome space.
  • Garbling. β on Y is a garbling of α on X when there is a Markov kernel K from X to Y with β_θ(B) = ∫ K(B | x) α_θ(dx) for every θ and every measurable B. Blackwell’s “α is sufficient for β”, written α ≻ β.
  • Decision problem and risk. An action space A, a loss L(θ, a) bounded and measurable in a, and a decision rule for β, which is a Markov kernel δ from Y to A. The rule’s risk at θ is R_β(δ, θ) = ∫∫ L(θ, a) δ(da | y) β_θ(dy). The risk vector is (R_β(δ, θ))_{θ∈Θ}.
  • Informativeness order. α ⊃ β when, in every decision problem, every risk vector attainable with β is attainable with α. THEORY-156 quotes the finite-Θ form: “for every closed bounded convex set A of loss vectors, every risk vector attainable with β is attainable with α”.

The claim, and its proof

If β is a garbling of α, then α ⊃ β.

Take any decision problem and any rule δ for β. Define a rule for α by first garbling and then applying δ: δ’(C | x) = ∫ δ(C | y) K(dy | x). This is a Markov kernel from X to A, being a composition of two kernels. Its risk at θ is

R_α(δ’, θ) = ∫ [∫∫ L(θ, a) δ(da | y) K(dy | x)] α_θ(dx) = ∫∫ L(θ, a) δ(da | y) [∫ K(dy | x) α_θ(dx)] = ∫∫ L(θ, a) δ(da | y) β_θ(dy) = R_β(δ, θ).

The exchange of integrals is Fubini’s theorem. It applies because L is bounded and measurable, and every kernel is a probability measure in its first argument. The second line is the definition of the garbling. So δ’ attains with α exactly the risk vector δ attains with β, at every θ. Since δ and the problem were arbitrary, α ⊃ β. ∎

Nothing in the proof uses finiteness of Θ, X, Y or A. It uses only that rules may be randomized, so that a composition of kernels is a rule, and that losses are bounded, so the integrals exist.

What it does not say

  • The converse. That α ⊃ β implies a garbling exists is THEORY-717 for finitely many states and outcomes (Active), and THEORY-156 in general (Proposed).
  • Anything about a restricted receiver. The decision-maker may use any rule. A receiver limited to a class of rules can do better with a garbling than with the original (CLAIM-199, CLAIM-225).
  • How much worse an incomparable experiment is. The order is qualitative. That is Le Cam’s deficiency, unread here (LIT-778).

Backlinks