nucleation

LIT-810: Efficient compression in color naming and its evolution

2 min read

Properties4
StatusActive — read 2026-10-09 (NOTE-631); worth reading as the paper that puts the information bottleneck to work on word meanings: a lexicon is an encoder q(w|m) from meanings (Gaussians over CIELAB colour) to words, scored by complexity I(M;W) against accuracy I(W;U), and the colour-naming systems of the World Color Survey and English lie close to the IB curve at β only slightly above 1, with 93% beating all 39 hue-rotated variants of themselves. Read it knowing that the need distribution (the LI source) is estimated from the naming data, cross-validated over languages, and that the "evolution" is a reading of the annealing path along the curve, not a model of historical change. Source of THEORY-163.
Tagslinguistics, information-theory, cognition
Cited as “paper” byNOTE-631 — Efficient compression in color naming
Cited as “source” byTHEORY-163 — In Zaslavsky, Kemp, Regier and Tishby's analysis, the colour-naming systems of the World Color Survey languages and English lie near the information-bottleneck bound for compressing perceptual meanings into words, at a trade-off parameter just above 1 and nearer than hue-rotated variants of themselves, with the need distribution over meanings estimated from the naming data, THEORY-791 — The colour-naming systems of the world's languages lie near the information-bottleneck bound for compressing perceptual meanings into words, and differences in one trade-off parameter place each language along it, as Zaslavsky, Kemp, Regier and Tishby propose

Graph View

Noga Zaslavsky, Charles Kemp, Terry Regier and Naftali Tishby (2018), Proceedings of the National Academy of Sciences 115(31):7937–7942 — DOI-10.1073/pnas.1800521115

Key takeaways

  • The model. Meanings are distributions m(u) over colours (isotropic Gaussians in CIELAB, σ² = 64, one per World Color Survey chip); a language is a naming policy q(w|m); the listener is the Bayesian decoder m̂_w = Σ_m q(m|w) m. Complexity is I(M;W), and the expected KL distortion between m and m̂ equals I(M;U) − I(W;U), so accuracy is I(W;U). Languages are scored against the IB objective I(M;W) − β I(W;U) of Tishby, Pereira and Bialek (LIT-338).
  • Near the bound. Every language’s fitted β_l is about 1.03 (LI source; 1.06 under a uniform source), on the steep part of the curve. Efficiency loss ε_l = 0.18 against 0.70 for the deterministic RKK+ baseline, and the soft-partition distance gNID 0.18 against 0.47; on hard mode maps (NID) the two models tie. 93% of languages beat all 39 hue-rotated variants of their own system.
  • Soft categories are efficient. At finite β the IB optimum is stochastic, q(w|m) ∝ q(w) exp(−β D[m‖m̂_w]), so graded membership and low-consensus regions are predicted rather than counted as noise.
  • Phase transitions as an evolutionary path. Annealing β adds categories at critical points, which roughly follows Berlin and Kay’s order, while categories also drift between transitions. The one clear miss: IB puts yellow first, where low-complexity WCS languages have black, white and red.
  • What is fitted. The need distribution p(m) is a “least informative” prior estimated from the naming data, averaged over languages and five-fold cross-validated. β_l is fitted per language.

Standing in the record

Filed on 2026-10-09 at the owner’s request, from the manuscript bibliography of 2026-10-09 (work what-survives-translation): one of the works the manuscript considered and dropped from its final reference list.

Read on 2026-10-09 (NOTE-631); the reading is the source of THEORY-163. It is the record’s first worked application of the bottleneck in LIT-338 to natural-language semantics, and it shows numerically the structural phase transitions that LIT-338’s reading found asserted there but not shown. Its “evolution” sits beside the iterated learning result of Griffiths and Kalish (LIT-769, THEORY-155): one says where efficient systems lie, the other where transmission takes them.

Backlinks