nucleation

LIT-1176: Generalized Measures of Information Transfer

2 min read

Properties6
StatusActive — read in full 2026-10-17 (NOTE-942); worth reading as the decomposition that the critique of transfer entropy (LIT-1165) stands on: under the Williams–Beer partial information decomposition, transfer entropy is the source's unique information about the target's next state plus its synergy with the target's past, which the authors call state-independent and state-dependent transfer, and a multivariate transfer entropy splits into unique, redundant and synergistic transfer. Read it knowing that every number depends on the I_min redundancy measure, that the PID it applies is Williams and Beer 2010 (LIT-1172), whose I_min is contested (LIT-1168, LIT-1162), that it is an unrefereed arXiv preprint, and that its text swaps two parameter settings in the three-process example. Filed as THEORY-712.
Tagsinformation-theory, complex-systems
ExtendsLIT-1177 — Measuring Information Transfer, LIT-1172 — Nonnegative Decomposition of Multivariate Information
Compared againstLIT-1168 — A Bivariate Measure of Redundant Information
Cited as “paper” byNOTE-942 — Williams and Beer — Generalized measures of information transfer
Cited as “source” byTHEORY-712 — Under any partial information decomposition satisfying the consistency equations, a transfer entropy T_{Y→X} = I(X_{t+1}; Y_t | X_t) is the unique information of the source's past about the target's next state plus the synergy of source and target pasts, which Williams and Beer call state-independent and state-dependent transfer; time-delayed mutual information is the unique part plus redundancy; with the I_min redundancy the state-independent part alone suffices for perfect open-loop control, and a two-source transfer entropy splits into unique, redundant and synergistic transfer

Graph View

Paul L. Williams and Randall D. Beer (2011), “Generalized Measures of Information Transfer”, arXiv preprint — ARXIV-1102.1507

Key takeaways

From the reading, NOTE-942.

  • The decomposition (Eq. 6, Eq. 8). I(X; Y | Z) = Π(X; {Y}) + Π(X; {Y, Z}): conditioning removes redundancy and adds synergy. So T_{Y→X} = SITE + SDTE, the unique information of Y_t and the synergy of (X_t, Y_t) about X_{t+1}.
  • Control reading (Theorem 1). A system is perfectly controllable by open-loop control iff it is perfectly controllable with SITE alone. SDTE is what closed-loop control adds.
  • One-parameter family (Fig. 2). x_{t+1} = y_t if x_t = 0; if x_t = 1, flipped with probability d. d = 0 is pure SITE (1 bit), d = 1 is a controlled-NOT and pure SDTE (1 bit). Checked: T = 1 − ½h(d), smallest at d = ½ (0.5 bit).
  • Against TDMI. I(X_{t+1}; Y_t) = SITE + redundancy, so it misses SDTE and includes shared history.
  • Multivariate transfer. Unique, redundant and synergistic transfer from two sources, checked at three corners. Redundant transfer means observation cannot tell which source drives the target.

Standing in the record

Filed on 2026-10-17 and read the same day, as the decomposition behind LIT-1165’s critique. Filed as THEORY-712. The PID it applies, Williams and Beer 2010, is held (LIT-1172), with two critiques of I_min (LIT-1168, LIT-1162). Under both alternative measures the coupled pair’s split is unchanged; Harder, Salge and Polani’s bystander example shows it can reverse elsewhere (NOTE-944). THEORY-497 already records that a ΦID verdict can turn on the redundancy function, and the same holds for SITE and SDTE away from the XOR and copy corners. It carries no instruction for machine-learning practice.

Backlinks