nucleation

LIT-1202: Reconsidering unique information: Towards a multivariate information decomposition

2 min read

Properties5
StatusActive — read in full 2026-10-17 (NOTE-965); worth reading as the short proof that the identity axiom and a non-negative decomposition on the Williams–Beer lattice cannot both hold once there are three sources: with Y1, Y2 independent bits, Y3 = Y1 ⊕ Y2 and the target their joint copy, identity, the Williams–Beer axioms and invariance under relabelling the target force the atom {12}{13}{23} to be at most −1 bit. So a multivariate extension of Bertschinger et al.'s unique information cannot be a partial information decomposition. Read it knowing that the theorem rests on the identity axiom applied to sources that are sets of variables and on an unstated invariance under relabelling the target, that its other results (monotonicity of unique information, failure of left monotonicity) are about the bivariate measures, and that the multivariate structure it calls for is left open. Filed as THEORY-713.
Tagsinformation-theory, mathematics
ExtendsLIT-1162 — Quantifying unique information
Cited as “paper” byNOTE-965 — Rauh, Bertschinger, Olbrich and Jost — Reconsidering unique information
Cited as “source” byTHEORY-713 — No redundancy measure on the Williams–Beer partial-information lattice can satisfy symmetry, self-redundancy, monotonicity, the identity axiom and non-negative atoms once there are three sources: for two independent bits, their XOR and a target that copies all three, identity forces three one-bit atoms under a two-bit total, so one atom is at most −1 bit, and a multivariate decomposition that keeps identity must give up non-negativity or the lattice

Graph View

Johannes Rauh, Nils Bertschinger, Eckehard Olbrich and Jürgen Jost (2014), “Reconsidering unique information: Towards a multivariate information decomposition”, 2014 IEEE International Symposium on Information Theory: 2232–2236 — ARXIV-1404.3146

Key takeaways

From the reading, NOTE-965.

  • The impossibility (Theorem 2). No I∩ satisfies symmetry, self-redundancy, monotonicity (with subset equality), local positivity and the identity axiom. In XORCOPY, Y1 and Y2 are independent uniform bits, Y3 = Y1 ⊕ Y2 and X = (Y1, Y2, Y3). Identity gives I∩(X: Yi; Yj) = 0 and I∩(X: Yi; YjYk) = MI(Yi; YjYk) = 1. Monotonicity then makes the bottom of the lattice zero and each {i}{jk} atom 1 bit. But I∩ at {12}{13}{23} is at most MI(X; Y1Y2Y3) = 2, so its atom is at most 2 − 3 = −1.
  • Two silent premises. Identity is applied with a set of variables as one source, and I∩(X: …) is taken to equal I∩((Yi, Yj): …) because X is a relabelling of any pair. The paper does not list the second as an axiom.
  • Consequence. A multivariate decomposition that extends Bertschinger et al.’s bivariate one consistently “cannot be a partial information decomposition”. Which terms it should have is left open.
  • Bivariate monotonicity (Lemmas 3–4, §IV-C). ŨI(X: Y \ (Z, Z′)) ≤ ŨI(X: Y \ Z); enlarging Y or X increases ŨI; S̃I increases when a source is enlarged. S̃I and C̃I violate left monotonicity, which the paper traces to the identity axiom: AND gives S̃I = 0.311 bit about Z but 0 about the copy (X, Y).
  • Conjecture (§V). MI(X; Y1…Yk) − Σ ŨI(X: Yi \ rest) ≥ 0, which would follow from ŨI(X: Y \ Z, W) + ŨI(X: Z \ Y, W) ≤ ŨI(X: (Y, Z) \ W). Unproved here.

Standing in the record

Filed on 2026-10-17 for the gap left open by the PID filing, and read the same day. It changes the gap’s shape. The record asked for a multivariate redundancy function with the identity property for ΦID. This paper proves that, on the lattice ΦID inherits, no such function also gives non-negative atoms. So the gap cannot be closed as stated. One of identity, non-negativity or the lattice has to go. Ince’s I_ccs (LIT-1201) is one way of letting non-negativity go. Filed as THEORY-713. mathematics is for the result’s form, an impossibility theorem on a lattice. It carries no instruction for machine-learning practice.

Backlinks