nucleation

LIT-1177: Measuring Information Transfer

2 min read

Properties6
StatusActive — read in full 2026-10-17 (NOTE-940); worth reading as the four-page source of transfer entropy, which the record uses in seven documents without holding it: T_{J→I} is the Kullback entropy of the transition probabilities of I with and without the past of J, at history lengths k and l that the definition leaves open. Read it knowing that "information transfer" and "exchange" are the paper's words for a conditional mutual information, that common inputs are excluded only when they are known and conditioned on, that the continuum-limit claim is asserted, and that the evidence is two coupled-map lattices and one physiological pair, the last of which Schreiber himself says a common trigger could explain. Filed as THEORY-701.
Tagsinformation-theory, complex-systems, causality
Extended byLIT-1167 — Information transfer in continuous processes, LIT-1175 — Local information transfer as a spatiotemporal filter for complex systems, LIT-1176 — Generalized Measures of Information Transfer
Corrected byLIT-1165 — Information Flows? A Critique of Transfer Entropies
Cited as “paper” byNOTE-940 — Schreiber — Measuring information transfer
Cited as “source” byTHEORY-701 — Transfer entropy from J to I is the Kullback entropy of assuming that I's next state, given its last k states, does not depend on J's last l states, so it is zero exactly when J's past leaves I's transition law unchanged; unlike time-delayed mutual information it is directional and zero for static correlation, and it removes a common input only when that input is conditioned on

Graph View

Thomas Schreiber (2000), “Measuring Information Transfer”, Physical Review Letters 85: 461 — ARXIV-nlin/0001042

Key takeaways

From the reading, NOTE-940.

  • The definition (Eq. 4). T_{J→I} = Σ p(i_{n+1}, i_n^{(k)}, j_n^{(l)}) log[p(i_{n+1} | i_n^{(k)}, j_n^{(l)}) / p(i_{n+1} | i_n^{(k)})], with i_n^{(k)} = (i_n, …, i_{n−k+1}). It measures the departure from the “generalized Markov property” p(i_{n+1} | i_n^{(k)}) = p(i_{n+1} | i_n^{(k)}, j_n^{(l)}). The paper leaves k and l to the user: l = k or l = 1 are “most natural”, l = 1 cheaper.
  • Why not delayed MI. Mutual information is symmetric. Introducing a lag gives it a direction “in a somewhat ad-hoc way”, and still counts correlation from common history or input as exchange.
  • Continuous systems. For coarse-grained states the ε → 0 limit is asserted finite and partition-independent, except under deterministic coupling. A known common driver Z can be excluded by conditioning on it.
  • Estimation. A step-kernel (correlation-sum) estimator of the joint probabilities at radius r, with temporally close neighbours excluded.
  • Examples. A tent-map lattice: T = α²ε²/ln 2 + O(ε⁴) in the coupling direction, zero against it, and zero delayed MI. An Ulam-map lattice: T stays consistent with zero against the coupling for every ε, where delayed MI does not. Heart and breath rates of a sleeper: T(heart → breath) > T(breath → heart), with a common trigger left open.

Standing in the record

Filed on 2026-10-17 as the source of the term, and read the same day. The record’s documents choose k and l differently, and all of them choose within Schreiber’s definition. NOTE-053 uses l = k; NOTE-788 uses k = l = 1 with an added delay τ that the paper does not define. Filed as THEORY-701. TERM-057 is the record’s own statement of the term, and CLAIM-335 its position on whether the quantity is a flow.

Backlinks