nucleation

THEORY-713: No redundancy measure on the Williams–Beer partial-information lattice can satisfy symmetry, self-redundancy, monotonicity, the identity axiom and non-negative atoms once there are three sources: for two independent bits, their XOR and a target that copies all three, identity forces three one-bit atoms under a two-bit total, so one atom is at most −1 bit, and a multivariate decomposition that keeps identity must give up non-negativity or the lattice

3 min read

Properties4
StatusActive
Tagsinformation-theory, mathematics
SourceLIT-1202 — Reconsidering unique information: Towards a multivariate information decomposition
SupportsCLAIM-343 — For three or more parts, no ΦID can use a redundancy function that both satisfies the identity axiom and yields nonnegative atoms, because on the Williams–Beer lattice the two are inconsistent from three sources up; a ΦID verdict on three or more parts therefore rests on which axiom the analyst drops, and should name it

Graph View

Source

Rauh, Bertschinger, Olbrich & Jost (2014), “Reconsidering unique information: Towards a multivariate information decomposition”, ISIT 2014, arXiv 1404.3146v1, LIT-1202, read in full in NOTE-965.

What was actually shown

The axioms. Williams and Beer’s symmetry, self-redundancy and monotonicity with subset equality (THEORY-706); local positivity, non-negative atoms after Möbius inversion; and Harder, Salge and Polani’s identity, I∩((Y, Z): Y; Z) = MI(Y: Z) (THEORY-708).

The counterexample (Theorem 2). Y1, Y2 independent uniform bits, Y3 = Y1 ⊕ Y2, X = (Y1, Y2, Y3). Any pair determines the third, so X is a relabelling of each pair.

  • Identity and relabelling give I∩(X: Yi; Yj) = MI(Yi; Yj) = 0, so by monotonicity the lattice’s lower two levels are zero.
  • Identity with a pair as one source gives I∩(X: Yi; YjYk) = MI(Yi; YjYk) = 1, so each of the three atoms {i}{jk} is 1 bit.
  • Monotonicity bounds I∩(X: {12}{13}{23}) by MI(X; Y1Y2Y3) = 2. That node sits above all three {i}{jk}, so its atom is at most 2 − 3 = −1.

Checked: every information value above, and the lattice order, by script (NOTE-965).

Bivariate side results. Bertschinger et al.’s unique information is monotone in each argument (Lemma 3). Their shared information increases with a source (Lemma 4). Shared and synergistic information violate left monotonicity (AND, XOR), so neither has a chain rule in the target.

What could have come out otherwise: XORCOPY could have admitted a non-negative assignment consistent with identity. It does not, and the deficit is exactly one bit.

What this does not say

  • That no multivariate decomposition with identity exists. Only that it cannot be a non-negative decomposition on the Williams–Beer lattice. The authors call for a different structure and do not give one.
  • That the theorem binds a two-source analysis. On the bivariate lattice Bertschinger et al.’s measures satisfy every axiom (THEORY-705). The record’s transfer-entropy examples are bivariate (CLAIM-335).
  • That it is free of further premises. It uses invariance under relabelling the target, and identity applied to a pair as one source. A weaker identity on single variables only is not refuted by it.
  • Which axiom to drop. I_min drops identity and keeps non-negative atoms; on XORCOPY it gives {1}{2}{3} = {12}{13}{23} = 1. I_ccs drops monotonicity and local positivity; it gives {i}{jk} = 1 and {12}{13}{23} = −1 (THEORY-714). The theorem does not choose.

Connections

  • THEORY-497: ΦID’s lattice is a product of Williams–Beer lattices, so for three or more parts an identity-respecting redundancy function must produce negative atoms. That is the record’s inference (NOTE-965).
  • THEORY-568 is the redundancy-measure-free route: its Shannon invariants are fixed by mutual informations once the atoms satisfy the consistency equations, so they do not depend on which axiom is dropped. That they survive negative atoms is the record’s reading of that account, not checked against its source.

Backlinks