nucleation

LIT-1201: Measuring multivariate redundant information with pointwise common change in surprisal

2 min read

Properties7
StatusActive — read 2026-10-17 (NOTE-968): definitions, properties, the operational argument and every worked example in full, the Gaussian section and the discussion at lower resolution. Worth reading as the multivariate redundancy measure that gives up non-negativity rather than identity: I_ccs counts only the pointwise co-information terms where every source, the joint source and the overlap change the surprisal of the target in the same direction, under the maximum-entropy joint that preserves source–target and source–source marginals. It satisfies symmetry, self-redundancy, subset equality and an independent-sources identity, and not Harder et al.'s identity, monotonicity or non-negative atoms. Read it knowing that its game-theoretic case against Bertschinger et al. is an argument about what unique information should mean, not a proof that their measure is wrong; that continuity is conjectured; that the "correct" PIDs it reports are judged against intuition; and that it accepts the −1-bit atom Rauh et al. prove some measure must produce. Filed as THEORY-714.
Tagsinformation-theory, complex-systems
ExtendsLIT-1172 — Nonnegative Decomposition of Multivariate Information
Compared againstLIT-1172 — Nonnegative Decomposition of Multivariate Information, LIT-1162 — Quantifying unique information
RivalsLIT-1162 — Quantifying unique information, LIT-1168 — A Bivariate Measure of Redundant Information
Cited as “paper” byNOTE-968 — Ince — Measuring multivariate redundant information with pointwise common change in surprisal
Cited as “source” byTHEORY-714 — Ince's I_ccs, the expected local co-information over only those events where every source, the joint source and the overlap change the target's surprisal in the same direction, under the maximum-entropy joint that preserves the source–target and joint-source marginals, is symmetric and self-redundant and satisfies the independent identity property; it fails Harder et al.'s identity and monotonicity, so its atoms can be negative, and, though its paper says otherwise, it fails subset equality, THEORY-804 — Ince's common change in surprisal is a correct measure of the redundancy among any number of sources, and the negative partial-information atoms it gives are local misinformation and mechanistic redundancy, not defects of the measure

Graph View

Robin A. A. Ince (2017), “Measuring multivariate redundant information with pointwise common change in surprisal”, Entropy 19(7): 318 — ARXIV-1602.05063

Key takeaways

From the reading, NOTE-968.

  • The measure (Defs. 4.1–4.2). I_ccs(S; A1…An) = Σ p̂ · c_p̂(a1…an, s) over the events where sgn Δ_s h(ai) for every i, sgn Δ_s h(a1…an) and sgn c agree; other events count zero. P̂ maximises entropy subject to Q(Ai, S) = P(Ai, S) for each i and Q(A1…An) = P(A1…An).
  • Axioms kept and dropped (§§3, 4.4). Kept: symmetry, self-redundancy, subset equality (claimed; it fails, see NOTE-968’s note of 2026-10-20), and the independent identity property, I(A1; A2) = 0 ⇒ I∩((A1, A2); A1, A2) = 0. Dropped: Harder et al.’s identity, deliberately; monotonicity, by the Table 7 counterexample (redundancy 0.77 bit against I(S; X2) = 0.61); and so non-negative atoms.
  • Against Bertschinger et al. (§4.3). In REDUCEDOR a stake-setting, simultaneous one-shot game gives each player an advantage, though no decision problem does. Ince concludes unique information is not invariant to P(X1, X2). I_broja reports 0.31 bit redundancy, I_ccs 0.31 bit unique to each source.
  • Three sources (§6). XORCOPY gives {i}{jk} = 1 and {12}{13}{23} = −1; DBLXOR gives {12} = {13} = {23} = 1 and {123} = −1. Giant bit, parity and XORUNQ come out as built.
  • Gaussians (§7). Unlike the minimum-MI redundancy, I_ccs moves with the correlation between the predictors.

Standing in the record

Filed on 2026-10-17 for the gap left open by the PID filing, and read the same day. It is the held multivariate measure. It does not close the gap as the record stated it, a multivariate redundancy with the identity property and a non-negative decomposition, because Rauh et al. prove there can be none on this lattice (LIT-1202). I_ccs keeps an independent-sources form of identity and gives up non-negativity. It is a rival to LIT-1162 and LIT-1168 on what unique information and identity should mean. ΦID work held here uses the minimum-mutual-information redundancy (NOTE-799), not I_ccs. Filed as THEORY-714. It carries no instruction for machine-learning practice.

Backlinks