nucleation

LIT-1204: Quantifying synergistic mutual information

2 min read

Properties6
StatusActive — read in full 2026-10-17 (NOTE-966), arXiv v6, with v1 and v2 checked for the passage that changed; worth reading as the "whole minus union" account of synergy, S_VK = I(X;Y) − I_VK, where the union information I_VK is the least I(X;Y) over joints with every source–target marginal fixed, and as the source of the early position, quoted by Harder, Salge and Polani, that independent sources carry no redundant information. Read it knowing that v6 withdraws that position (Fig. 11's note), that for two sources I_VK is the same optimisation as Bertschinger et al.'s, so S_VK is their synergy CI and the paper's open AND value is 0.5 bit, that several properties of the derived intersection measure are not proved, and that the bound S_VK ≤ S_max rests on a step the reading could not reconstruct. Filed as THEORY-716.
Tagsinformation-theory, complex-systems
ExtendsLIT-1172 — Nonnegative Decomposition of Multivariate Information
Compared againstLIT-1172 — Nonnegative Decomposition of Multivariate Information
Cited as “paper” byNOTE-966 — Griffith and Koch — Quantifying synergistic mutual information
Cited as “source” byTHEORY-716 — Synergy can be measured as the whole minus the union of its parts, the union information being the least information the sources could jointly carry about the target with every source–target marginal held fixed; this synergy lies between WholeMinusSum and Williams and Beer's S_max, which respectively subtract redundancy and count unique information as synergy, and for two sources it is Bertschinger et al.'s synergy, so independent sources can share information about a target, as the authors' last version concedes against their first

Graph View

Virgil Griffith and Christof Koch (2014), “Quantifying synergistic mutual information”, in M. Prokopenko (ed.), Guided Self-Organization: Inception, Springer: 159–190 — ARXIV-1205.4265

Key takeaways

From the reading, NOTE-966.

  • Whole minus union (Eqs. 13–14). I_VK(X : Y) = min I*(X_1…n : Y) over Pr* with Pr*(X_i, Y) = Pr(X_i, Y) for every i, and S_VK = I(X_1…n : Y) − I_VK. A minimum exists, since Pr(y) Π Pr(x_i | y) satisfies the constraints, and that distribution gives an analytic upper bound on I_VK (Appendix D).
  • Bounds (Eq. 15). max(0, WMS) ≤ S_VK ≤ S_max ≤ I(X_1…n : Y).
  • Three prior measures criticised (§4, Table 1). S_max (from I_max) reports 1 bit of synergy in Unq, where two predictors each carry one unique bit. WholeMinusSum is synergy minus redundancy, 0 in RdnXor. ΔI exceeds the mutual information in one published case, is 0.104 bit in AND, below the 0.189 lower bound, and falls to 0.038 when a predictor is duplicated.
  • Properties of I_VK (§6). Global positivity, self-redundancy, weak symmetry, weak local positivity for two sources, and strong identity, I_VK(X : X_1…n) = H(X_1…n), are proved. Monotonicity and target monotonicity are claimed, and strong symmetry is disproved.
  • The reversal. v1 and v2 assert that because X_1 and X_2 are independent in AND there must be zero redundant information, and so give 0.311 bit unique each and 0.189 synergy. v6 says this was wrong: the analytic bound gives redundancy at least 0.082 bit.

Standing in the record

Filed on 2026-10-17 as the Griffith view Harder, Salge and Polani oppose (LIT-1168), and read the same day. Filed as THEORY-716. The reading finds the dispute resolved by the authors themselves. Harder, Salge and Polani quote v1’s “independent … thus necessitating there is zero redundant information” accurately, and call what they defend against it “mechanistic redundancy”. Version 6 concedes the point for its own measure. For two sources its union information is Bertschinger et al.’s optimisation (LIT-1162), so S_VK is their CI, and AND’s synergy is 0.5 bit with 0.311 bit redundant, the value Harder, Salge and Polani expected. That identification is the reading’s, not the paper’s. It carries no instruction for machine-learning practice.

Backlinks