nucleation

LIT-1258: Exact and Soft Successive Refinement of the Information Bottleneck

1 min read

Properties6
StatusActive — read 2026-10-20 (NOTE-981) in full, from the published version in the University of Hertfordshire Research Archive; worth reading for the single-letter definition of successive refinement for the information bottleneck and its equivalent forms (Proposition 1: some pair of optimal encoders forms the Markov chain X → T₂ → T₁; Proposition 8: the coarse encoder is a Blackwell garbling of the fine one); for the convex-hull test (Proposition 4: with an injective fine decoder, refinement holds iff every decoder point q(X|t₁) lies in the convex hull of the q(X|t₂)), checkable by a linear programme; for proofs of refinability when X and Y are both binary, when Y is a function of X, and for jointly Gaussian variables (the last resting on Kline and Palmer's semigroup result); and for soft refinement measured by unique information. Read it knowing that the linear programme tests the particular optima a solver returns, so that a failed test certifies nothing unless the optima are globally optimal and unique, which the paper leaves as Conjecture 1 and calls its own figures "mere numerical proxies" (Appendix D); that its evidence that refinement fails is therefore numerical and uncertified, with no count of how many sampled distributions failed; that the unique information computed is an upper bound on the soft measure for the same reason; that the deep-learning reading of §4.2 is speculative; and that the commentary on Proposition 8 swaps its points (ii) and (iii).
Version2
Tagsinformation-theory, learning-theory, anthology-candidate
SupportsCLAIM-392 — Stochastic information-bottleneck optima at two budgets need not be nested: for X uniform on four states and binary Y with P(Y=1|x) = (0, 0, 1/8, 3/8), the unique optima at β = 8 and β = 12 admit no Markov chain X → T₂ → T₁, so raising the budget reorganizes the optimum even with noisy encoders
Cited as “paper” byNOTE-981 — Charvin, Catenacci Volpi and Polani — Exact and soft successive refinement of the IB
Cited as “source” byTHEORY-753 — For finite X and Y, the information bottleneck is successively refinable from budget λ₁ to λ₂ > λ₁ iff some pair of optimal encoders forms a Markov chain X → T₂ → T₁, that is, iff the coarse optimal encoder is a garbling of a fine one; when the fine optimum's decoder points q(X|t₂) are affinely independent, this holds for that pair iff every decoder point q(X|t₁) of the coarse optimum lies in their convex hull, so a failed hull test on optima that are unique up to merging and relabelling proves that refinement fails, THEORY-757 — The information bottleneck is successively refinable between every pair of budgets when Y is a function of X, by nested erasures of Y, and when X and Y are both binary, where the optimum at each nonzero budget is unique up to merging and relabelling and its two decoder points are the ends of an interval of the source simplex that only widens as the budget grows

Graph View

Charvin, H., Catenacci Volpi, N. & Polani, D. (2023). Exact and Soft Successive Refinement of the Information Bottleneck. Entropy 25(9): 1355. doi:10.3390/e25091355. Open access (CC BY), also PMC10528077.

Read in full in NOTE-981, from the copy in the University of Hertfordshire Research Archive.

Why this is here

Named by NOTE-980 as the direct reading for the stochastic half of QUESTION-220: when are the information-bottleneck optima at two budgets nested, X → Z₂ → Z₁? The paper supplies the exact criterion and a test for it, and proves nesting for three special cases. Its own evidence that nesting fails in general is numerical and uncertified. The reading certified one failure on the record’s four-state example (NOTE-981), and THEORY-753 and THEORY-757 hold what the paper proves.

Backlinks