nucleation

CLAIM-335: Transfer entropy measures directed predictive improvement, not information flow: as a conditional mutual information it is the source's unique information about the target's next value plus its synergy with the target's past, so it can credit a source with a bit the source carries only jointly and read zero from sources that jointly fix the target, and conditioning on more series does not remove the synergy

4 min read

Properties9
StatusActive
Version3
Rolethesis
Warrantdemonstrated
Tagsinformation-theory, causality
Lineformal-apparatus
GroundsTHEORY-707 — Transfer entropy is not a measure of information flow, if flow means information in the target attributable solely to the source's past: as a conditional mutual information it equals the source's unique information plus its synergy with the target's past, so it can credit a full bit to a source that carries none alone (Y_t = X_{t−1} ⊕ Y_{t−1}) and report zero from each of two sources that jointly fix the target (Z_t = X_{t−1} ⊕ Y_{t−1}); conditioning on further series, as causation entropy does, has the same fault, THEORY-712 — Under any partial information decomposition satisfying the consistency equations, a transfer entropy T_{Y→X} = I(X_{t+1}; Y_t | X_t) is the unique information of the source's past about the target's next state plus the synergy of source and target pasts, which Williams and Beer call state-independent and state-dependent transfer; time-delayed mutual information is the unique part plus redundancy; with the I_min redundancy the state-independent part alone suffices for perfect open-loop control, and a two-source transfer entropy splits into unique, redundant and synergistic transfer, THEORY-702 — Transfer entropy is the average of a local transfer entropy, the log ratio at each observation of the destination's next-state probability with and without the source given the destination's past, which can be negative where the source misleads; the destination history must be long for it to exclude self-influence routed through the source; and conditioning also on the destination's other causal parents (complete) can differ sharply from conditioning on its past alone (apparent), as in elementary cellular automata, where long-history local transfer entropy picks out gliders and domain walls, THEORY-705 — For a target and two sources with finite alphabets, if unique information is to depend only on the two source–target pair marginals, then the minimum of the conditional mutual information over all joints with those marginals is an upper bound on every non-negative decomposition's unique information, and the shared and synergistic information it implies are lower bounds; this unique information vanishes exactly when the other source is Blackwell-sufficient, and the measures are non-negative, satisfy the bivariate partial-information axioms and the identity property, and do not extend to three sources on the partial-information lattice
ComplementsCLAIM-333 — Measuring every relevant variable is necessary but not sufficient for a Granger-causal analysis to say anything about interventions: the intervention must also be an isolated exogenous change and no influence may act within one sampling step, and then Granger noncausality implies no intervention effect and the effect is identified from the observational predictor, but a Granger-causal relation is still only potentially causal, CLAIM-343 — For three or more parts, no ΦID can use a redundancy function that both satisfies the identity axiom and yields nonnegative atoms, because on the Williams–Beer lattice the two are inconsistent from three sources up; a ΦID verdict on three or more parts therefore rests on which axiom the analyst drops, and should name it
UsesTERM-057 — transfer entropy: how much the last l values of a source reduce uncertainty about a target's next value beyond the target's own last k values, I(X_{t+1}; Y_t^{(l)} | X_t^{(k)}), a directed predictive quantity that depends on k, l and any conditioning series, not a measure of information flow and not an intervention effect, TERM-058 — Granger causality: the past of one series improving the prediction of another's next value beyond the past of everything else in a stated information set, a predictive relation relative to that set and its lags, and not by itself the effect of an intervention

Graph View

The claim

Transfer entropy is I(X_{t+1}; Y_t^{(l)} | X_t^{(k)}) (TERM-057), and for Gaussian data it is half Granger causality (TERM-058), so what follows holds of the Granger measure there too. By the consistency equations of any partial information decomposition with the two sources Y’s past and X’s past, a conditional mutual information is the source’s unique information plus the synergy of the two. The redundancy drops out, and the synergy comes in (Williams and Beer, THEORY-712). Only the decomposition’s two-term split is needed here, not their I_min redundancy measure.

If “flow from Y to X” means information in X’s next value attributable to Y’s past alone, as James, Barnett and Crutchfield define it (THEORY-707), then flow is at most the unique part, and transfer entropy overstates it by the synergy. Where several sources fix the target jointly, each pairwise transfer entropy can be zero. Conditioning on the other sources (causation entropy, complete transfer entropy) brings their joint bit back, but credits it to each source in full, so it is counted once per source.

The two examples

Both recomputed exactly on 2026-10-17 (a scratchpad script with an I_min PID; NOTE-945), and again under Harder, Salge and Polani’s I_red and Bertschinger et al.’s unique information, with the same values (NOTE-944, NOTE-937), and under Ince’s I_ccs (NOTE-968).

  1. Overstating flow. X_t i.i.d. uniform bits, Y_t = X_{t−1} ⊕ Y_{t−1}. T_{X→Y} = I(Y_t; X_{t−1} | Y_{t−1}) = 1 bit. But I(Y_t; X_{t−1}) = 0 and I(Y_t; Y_{t−1}) = 0: neither past alone says anything about Y_t. The co-information is −1, and the PID gives 0 redundant, 0 unique to each, 1 synergistic. The whole bit is joint.
  2. Understating influence. X, Y i.i.d. uniform, Z_t = X_{t−1} ⊕ Y_{t−1}. Both T_{X→Z} and T_{Y→Z} are 0, and every pairwise information is 0, yet Z is fixed by the two. The causation entropies, each conditioning on the other source, are 1 bit each, so the one bit is counted twice.

Lizier et al.’s rule 18 is the same structure in a cellular automaton (THEORY-702): where the centre cell is 0 the update is L ⊕ R, apparent transfer entropy from each neighbour is 0 and complete transfer entropy is 1 bit from each. That reading of their case is the record’s; they read the contrast as complex against chaotic dynamics.

The first process is Williams and Beer’s coupled pair at d = 1, where they call the bit “state-dependent transfer”. The two papers agree on the decomposition. They disagree only on whether that part is to be called transfer.

What it does not say

  • That transfer entropy is wrong as a predictive measure. As predictive improvement it is exactly what THEORY-641 and THEORY-700 say: a likelihood-ratio limit, and twice Granger causality for Gaussian data.
  • That a nonzero transfer entropy shows no influence. In example 1, intervening on X_{t−1} does change Y_t, so Ay and Polani’s interventional flow credits X (THEORY-710). What fails is localizing the bit to X alone, not the existence of an effect. Prediction against intervention is CLAIM-333’s separate question.
  • What the unique part is. It has no agreed measure. In general every number beyond the two-term split depends on a choice of redundancy function, though Bertschinger et al. prove the split itself for any bivariate decomposition (THEORY-705, Remark 14). In the record’s examples the numbers are the same under I_min and under their measure, because the target’s own past says nothing about its next value there (NOTE-937).
  • That directed information escapes this. James, Barnett and Crutchfield say it inherits the flaw. Its lagged part is full-history transfer entropy (THEORY-711), so it does at least that far.

Note of 2026-10-21: synergy in the target

A6 and A7 of the exchange of 2026-10-20 called the three-bit chain of THEORY-820 “synergistic” without citing this claim. Its arrangement differs from the one here. This claim’s synergy is between a source and the target’s past. In the chain, the micro past’s bit is carried by the target’s two components jointly (A′ ⊕ B′ = C_t), and by the chain rule T(A, B) = I(X⃗; A′ | A⃗, B⃗) + I(X⃗; B′ | A⃗, B⃗, A′) = 0 + 1 bit (checked). So transfer entropy onto a joint target can be positive where it is zero onto each component, the dual of this claim’s “read zero from sources that jointly fix the target”. A7 rightly did not commit to a PID measure.

Backlinks