nucleation

LIT-1162: Quantifying unique information

2 min read

Properties7
StatusActive — read in full 2026-10-17 (NOTE-937); worth reading as the bivariate partial information decomposition that the redundancy-measure-dependent numbers in the record can be checked against: unique information is the minimum of the conditional mutual information over all joints with the same source–target pair marginals, so it vanishes exactly when Blackwell's order says the other source knows everything this one does, and synergy is what the pair marginals cannot reveal. Read it knowing that it is bivariate only (it cannot be extended to the PI lattice for three sources, by its own §6), that its optimisation is ill-conditioned in practice (Example 31), and that Table 1's Unq and Copy rows conflict with its own Proposition 18. Filed as THEORY-705.
Tagsinformation-theory, mathematical-statistics, complex-systems
Extended byLIT-1202 — Reconsidering unique information: Towards a multivariate information decomposition
Compared againstLIT-1172 — Nonnegative Decomposition of Multivariate Information, LIT-1168 — A Bivariate Measure of Redundant Information, LIT-1201 — Measuring multivariate redundant information with pointwise common change in surprisal
RivalsLIT-1172 — Nonnegative Decomposition of Multivariate Information, LIT-1201 — Measuring multivariate redundant information with pointwise common change in surprisal
Cited as “paper” byNOTE-937 — Bertschinger, Rauh, Olbrich, Jost and Ay — Quantifying unique information
Cited as “source” byTHEORY-705 — For a target and two sources with finite alphabets, if unique information is to depend only on the two source–target pair marginals, then the minimum of the conditional mutual information over all joints with those marginals is an upper bound on every non-negative decomposition's unique information, and the shared and synergistic information it implies are lower bounds; this unique information vanishes exactly when the other source is Blackwell-sufficient, and the measures are non-negative, satisfy the bivariate partial-information axioms and the identity property, and do not extend to three sources on the partial-information lattice

Graph View

Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, Jürgen Jost and Nihat Ay (2014), “Quantifying unique information”, Entropy 16(4): 2161–2183 — ARXIV-1311.2852

Key takeaways

From the reading, NOTE-937.

  • The definition. UĨ(X : Y \ Z) = min over Q ∈ Δ_P of MI_Q(X : Y | Z), where Δ_P holds the joints with P’s (X,Y) and (X,Z) marginals. SĨ = max CoI_Q, and CĨ = MI(X : (Y,Z)) − min MI_Q(X : (Y,Z)). One convex programme gives all three (Lemma 4).
  • The motivating property (∗). Unique information should depend only on p(x) and the two channels X → Y and X → Z. Under (∗), these measures are upper bounds on unique information and lower bounds on shared and synergistic information for any decomposition (Lemma 3). They are the only ones for which the pair marginals cannot reveal synergy (∗∗).
  • Operational meaning. UĨ(X : Z \ Y) = 0 exactly when every decision problem about X is solved at least as well from Y as from Z: Blackwell’s order (Lemma 6, Corollary 7).
  • Properties. Non-negativity, the bivariate PI axioms, the identity property SĨ((Y,Z) : Y; Z) = MI(Y : Z) (Proposition 18), additivity over independent subsystems, and unique information that cannot grow when sources are added (Lemma 25).
  • Against I_min. I_min satisfies (∗), so I_min ≥ SĨ. It is “unreasonably large” in examples such as Copy, where it gives 1 bit of redundancy between independent sources.
  • Limits. Bivariate only. The identity property is incompatible with a non-negative decomposition on the PI lattice for three sources (§6, citing Bertschinger et al. 2013). The optimisation can be badly conditioned (Example 31).

Standing in the record

Filed on 2026-10-17 as the bivariate decomposition the record’s transfer-entropy accounts can be checked against. Every number in THEORY-712 and THEORY-707 that went past the two-term split was computed with I_min. Recomputed under this measure, the split for every transfer-entropy example the record uses is unchanged (NOTE-937). The split itself holds for any bivariate decomposition (Remark 14). NOTE-010’s reliance on “Bertschinger et al.’s” identity property now has its source. mathematical-statistics is for the Blackwell order, complex-systems for its use in synergy and emergence. It carries no instruction for machine-learning practice.

Backlinks