Source
James, Barnett & Crutchfield (2016), “Information Flows? A Critique of Transfer Entropies”, arXiv 1512.06479v2 (Phys. Rev. Lett. 116, 238701), LIT-1165, read in full in NOTE-945.
What was actually shown
Flow, defined. Information in Y_t whose cause “can be solely attributed to X’s past” (p. 1).
Overestimate. X_t, Y_0 fair bits, Y_t = X_{t−1} ⊕ Y_{t−1}: T_{X→Y} = 1 bit, while I[Y_t : X_{0:t}] = I[Y_t : Y_{0:t}] = 0 and I[Y_t : Y_{0:t} | X_{0:t}] = 1. Read one way X’s past predicts Y_t, read the other way Y’s past does. The co-information is −1. Recomputed: the I_min PID gives 1 bit of synergy and nothing else.
Underestimate. X, Y fair bits, Z_t = X_{t−1} ⊕ Y_{t−1}: T_{X→Z} = T_{Y→Z} = 0 and every pairwise information is 0, while C_{X→Z|(Y,Z)} = C_{Y→Z|(X,Z)} = 1. Recomputed.
Why. I[X₁ : Y | X₂] = U₁ + S. Transfer entropy, with the source’s past as X₁ and the target’s past as X₂, is unique plus synergistic information. Conditioning removes redundancy and adds synergy; it is not subtraction.
What could have come out otherwise: Example 1’s bit could have been attributable to one past. It is not, under any PID, because XOR with independent uniform inputs is pure synergy for every proposed redundancy measure.
What this does not say
- That transfer entropy is wrong as a measure of predictive improvement. The authors say they “do not object” to it as the reduction in uncertainty about one series given another. THEORY-641 and THEORY-700 stand.
- That X has no causal influence on Y in Example 1. Intervening on X_{t−1} changes Y_t. The failure is in localizing the bit to X, not in detecting X’s influence. That is the record’s observation (NOTE-945), and it is separate from CLAIM-333’s point about intervention.
- How much of a transfer entropy is synergy outside the XOR case. That depends on the redundancy measure, and none is agreed.
- That directed information or Ay and Polani’s interventional flow fail the same way. The first is asserted from its being “essentially a sum of transfer entropies”; the second is a conjecture.
- That unique information measures flow. Proposed, not shown.
Connections
- Williams and Beer (2011, THEORY-712) give the same decomposition and call the synergistic part state-dependent transfer. The two accounts agree on the mathematics and disagree on whether that part is “transfer”.
- THEORY-497’s ΦID decomposes past→future information into source and target atoms, which is the kind of tool that would separate the parts this account says transfer entropy conflates.
- THEORY-257’s dynamical independence uses transfer entropy from the whole micro history at once. Example 2 shows why the pairwise version would not do. The record’s inference.
- TERM-057 states transfer entropy in the record’s words; CLAIM-335 is the record’s position.