Ryan G. James, Nix Barnett and James P. Crutchfield (2016), “Information Flows? A Critique of Transfer Entropies”, Physical Review Letters 116: 238701 — ARXIV-1512.06479
Key takeaways
From the reading, NOTE-945.
- Flow, defined. Information flow from X to Y is information currently in Y whose “cause” can be solely attributed to X’s past: localized information (p. 1).
- Overestimating flow. X_t, Y_0 fair coins, Y_t = X_{t−1} ⊕ Y_{t−1}. T_{X→Y} = 1 bit, but I[Y_t : Y_{0:t}] = I[Y_t : X_{0:t}] = 0 and I[Y_t : Y_{0:t} | X_{0:t}] = 1 bit. The bit belongs to both pasts together; transfer entropy assigns it to X. Checked.
- Underestimating influence. X, Y fair coins, Z_t = X_{t−1} ⊕ Y_{t−1}. T_{X→Z} = T_{Y→Z} = 0, all pairwise informations vanish, and the causation entropy C_{X→Z|(Y,Z)} = 1 bit, which in turn localizes a joint bit. Checked.
- Why. I[X : Y | Z] is not “X and Y with Z removed”; it can exceed I[X : Y]. In Williams and Beer’s PID, I[X₁ : Y | X₂] = U₁ + S, so transfer entropy is unique plus synergistic information.
- Consequence for networks. Graphs with transfer-entropy edges encode only dyadic relations. In the second example the three series are pairwise independent, so the graph is empty while Z is fixed by X and Y.
Standing in the record
Filed on 2026-10-17 to fill the record’s gap on whether transfer entropy
measures flow, and read the same day. Filed as THEORY-707. It is
the main source for CLAIM-335. Its examples are the d = 1 corner of
Williams and Beer’s coupled-process family (LIT-1176), which those
authors call state-dependent transfer: the two papers agree on the
decomposition and disagree on the word. causality is for its question,
what kind of influence a transfer entropy reports. It carries no
instruction for machine-learning practice.