nucleation

LIT-1240: Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction

1 min read

Properties5
StatusActive — read 2026-10-20 (NOTE-976) in full, from arXiv:nlin/0006025v1; worth reading for the identification, in the finite memoryless case, of three constructions as one partition: the inputs grouped by Pr(Y|X = x), which are the information bottleneck's β → ∞ end, computational mechanics' causal states and Salmon's statistical relevance basis. Read it knowing that it is a three-page note that proves none of the identifications. It calls the IB limit "evident" and defers the Salmon proofs. It treats the IB optima as partitions, which at finite β they are not. It misstates the IB's marginal trade-off of 1/β as a global exchange rate. And it says nothing about the IB solutions at finite β or whether they nest. The record proves the limit and the factoring of every finite-β optimum through the causal states (THEORY-743).
Version2
Tagsinformation-theory, complex-systems
Cited as “paper” byNOTE-976 — Shalizi and Crutchfield — Information bottlenecks, causal states and statistical relevance bases
Cited as “source” byTHEORY-743 — For finite X and Y, every information-bottleneck optimum at every β gives one encoder to all inputs sharing p(y|x), so it is a stochastic coarsening of the minimal sufficient partition (the causal states of memoryless transduction); as β → ∞ the optima converge to that partition, but in general no finite β attains it, except when Y is a function of X

Graph View

Shalizi, C. R. & Crutchfield, J. P. (2002). Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction. Advances in Complex Systems 5(1): 91–95. doi:10.1142/S0219525902000481. Preprint: arXiv:nlin/0006025 (16 June 2000).

Read 2026-10-20 in NOTE-976. The summary was corrected against the reading.

Why this is here

A2 named it as the link between information bottlenecks, minimal predictive statistics and causal states, asking when progressively refined predictive states form a hierarchy. Bears on QUESTION-220. The authors’ “Computational Mechanics” is held, unread, as LIT-1126.

Backlinks