Thomas M. Cover and Joy A. Thomas (2006), Elements of Information Theory, 2nd edition, Wiley-Interscience — DOI-10.1002/047174882X
Key takeaways
Filed, not read. What follows is from the publisher’s excerpts only (see the history note): the table of contents and Chapter 1, the preview.
- Scope, from the contents. Seventeen chapters: entropy, relative entropy and mutual information (2); the AEP (3); entropy rates (4); data compression (5); gambling (6); channel capacity (7); differential entropy (8); the Gaussian channel (9); rate distortion theory (10); information theory and statistics, from the method of types to Fisher information (11); maximum entropy (12); universal source coding, including Lempel–Ziv (13); Kolmogorov complexity, including MDL (14); network information theory (15); portfolio theory (16); inequalities (17).
- Where the side-information results are. §15.4 (Slepian–Wolf, p. 549), §15.8 (source coding with side information, p. 575) and §15.9 (rate distortion with side information, p. 580), the textbook treatment of the Wyner–Ziv paper, LIT-767; side information in gambling is §6.2 and in investment §16.4.
- The authors’ framing (Chapter 1). Information theory gives the two extreme points of communication: the compression limit, a minimum of I(X; X̂), and the transmission limit, capacity, a maximum of I(X; Y). They also say they “consider Kolmogorov complexity to be more fundamental than Shannon entropy”, and that the increase in a gambler’s doubling rate from side information equals the mutual information between the race and the side information.
Standing in the record
Filed on 2026-10-09 at the owner’s request, as one of the works in the reference list of the owner’s working manuscript (October 2026) that the record did not yet hold. See the curation entry of that day.