NOTE-088: The Homological Nature of Entropy

Contribution

The paper sets out a cohomology theory, information cohomology, whose coefficients are measurable functions of probability laws and whose monoid action is Shannon’s averaged conditioning. It proves three things:

• Shannon entropy is its unique degree-one class on sufficiently rich classical structures, up to scale.
• In the quantum, unitary-invariant setting that class becomes trivial, and von Neumann entropy appears as the 0-cochain it bounds.
• The same characterisation holds for a dynamical, tree-indexed (“observation strategy”) version built from an operad of ordered partitions.

This gives an axiom-light characterisation of entropy: no symmetry and no stability under refinement are assumed, and both come out of the cohomology (p. 3273). It also proposes a programme in which mutual informations of all orders, KL divergence and “arborescent” mutual information are (twisted) cocycles.

Key insight

Shannon’s chain rule H(X,Y) = H(X) + X.H(Y) is the 1-cocycle equation of a bar complex, f((X,Y);P) = f(X;P) + X.f(Y;P), in which a partition X acts on functions of P by averaged conditioning. Once entropy is read this way, its axiomatic uniqueness becomes the computation of a cohomology group. Changing what “conditioning” and “partition” mean changes the answer in informative ways:

• quantum projective measurement makes the class trivial;
• tree-grafting makes it survive.

Assumptions

• Finite setting. Ω is finite with a Boolean algebra B. Quantum: E = ℂᴺ with a fixed positive-definite form h₀. Infinite extensions are deferred to [7].
• Information structure S.
  • S is a set of partitions (quantum: orthogonal decompositions) containing the trivial partition 1.
  • It is closed under the join (Y,Z) whenever Y, Z are both refined by some X ∈ S.
  • Quantum structures additionally require that such Y, Z commute. Joint observables are always commuting families, by convention (§4.2).
• Probability functor Q.
  • It is adapted to S, i.e. stable under conditioning by elements of S.
  • For Theorem 1 it must be complete: Q_X contains all laws on Ω/X.
  • Simplicial sub-complexes of the simplex are allowed and are characterised by “if A, B, … possible then D impossible” rules (Theorem 2, §2.4).
• Cochains. Cochains are measurable in P and satisfy joint locality: F(X₁;…;X_m;P) depends only on the image law (X₁,…,X_m)∗P (Eq. 18).
• Richness and connectedness (Theorems 1, 3, 4).
  • Every element of S lies on a rich edge: X, Y ∈ S with XY ∈ S, each with ≥ 2 values, and XY cutting them into ≥ 4 pieces.
  • S is connected by edges.
• Theorem 3 (quantum). S and Q must be unitary-invariant. Q is the canonical functor Q_can, the minimal functor that is complete with respect to S. Cochains are invariant (Def. 6).
• Theorem 4 (trees). The structure S(A) generated by allowed observation trees is connected and sufficiently rich. Q is canonical. Cochains are “regular” (Def. 17).

Key results

• Cocycle equation (Eq. 30). f((Y,Y′);P) = f(Y;P) + Y.f(Y′;P), with Y.f(P) = Σ_j P(Y=y_j)·f(P|Y=y_j) (Eq. 16).
  • Every 1-cocycle vanishes on a certainty: f(X; P|X=x_i) = 0 (Eq. 33).
  • Every 1-cocycle satisfies the symmetric relation f(Y) − Z.f(Y) = f(Z) − Y.f(Z) (Eq. 36).
• H⁰ = ℝ for connected S, and ℝ^m for m components (Eq. 28).
• Lemma 2. On Ω = {00, 01, 10, 11} with the two projections, any measurable 1-cocycle reduces to Tverberg’s functional equation h(u) − h(v) = (1−v)·h(u/(1−v)) − (1−u)·h(v/(1−u)) (Eq. 48). Hence h(x) = C(x log x + (1−x) log(1−x)).
• Theorem 1. For connected, sufficiently rich S and complete Q, H¹(S;Q) is one-dimensional and generated by Shannon entropy.
  • The symmetric cohomology variant gives the same, H¹_S = ℝH.
• Theorem 2 (§2.4). A subset of the simplex Δ_N is a simplicial sub-complex iff it is defined by finitely many constraints of the form “A, B, C, … possible ⇒ D impossible”.
• §3 (Theorem B, sketched). I₂ = δ_t H.
  • I_{2m} = δ_t δ δ_t … δ δ_t H (2m − 1 factors) and I_{2m+1} = −δ δ_t … δ δ_t H (2m factors).
  • Lemma 3 (unproved here): I_N((S₀,S₁);S₂;…) = I_N(S₀;S₂;…) + S₀.I_N(S₁;S₂;…).
  • Also (δ − δ_t)F = (S₀ − 1).F (Eq. 58), and I_{N+1}(S₀;…;S_N) = I_N(S₁;…;S_N) − S₀.I_N(S₁;…;S_N) (Eq. 62).
  • The intro adds, without proof: if the pairwise informations of X, Y, Z vanish, then I₃ is a Milnor/Massey triple-linking invariant, and the absolute minima of I₃ are “Borromean links, interpreted as synergy”.
• Lemma 5 / Lemma 6 (quantum).
  • S_{(X,Y)}(ρ) = H(Y;ρ) + Y.S_X(ρ).
  • Hence δ̂(S_X)(Y;ρ) = −H_X(Y;ρ): Shannon entropy is the coboundary of minus the von Neumann entropy.
• Theorem 3. For unitary-invariant, connected, sufficiently rich S with the canonical Q:
  • (i) H¹_U(S;Q) = 0;
  • (ii) H⁰_U is one-dimensional (the constants), so von Neumann entropy is, up to an additive constant, the only invariant 0-cochain with δS = −H.
  • The proof uses Lemma 7 (a unitary-invariant function unchanged by the farthest off-diagonal entry is a function of the trace), whose proof is two sentences.
• Proposition 4 / Lemma 8.
  • Classical KL of image laws, H_X(Y;P;Q) = H(Y∗P;Y∗Q), is a 1-cocycle.
  • The quantum cross-entropy S(ρ;σ) = −Tr(ρ log σ) satisfies S_{(X,Y)}(ρ;σ) = H_Y(X;ρ;σ) + X.S_Y(ρ;σ).
  • The authors call −Tr(ρ log σ) “the generating function of all the co-cycles we have considered so far” (p. 3297).
• Proposition 8. Shannon entropy satisfies the arborescent cocycle equation (Eq. 158): H(µ_m(S;(S₁,…,S_m));P) = H(S;P) + Σᵢ P(S=i)·H(Sᵢ;P|S=i). This is Faddeev’s / Baez–Fritz–Leinster’s identity, which is finer than Shannon’s chain rule because µ_m is not the join unless all Sᵢ coincide.
• Theorem 4. For connected, sufficiently rich S(A) with canonical Q:
  • classical H¹_τ is one-dimensional and generated by entropy;
  • quantum invariant H¹_U = 0, with von Neumann entropy as the unique invariant 0-cochain bounding Shannon entropy.
  • The proof is stated as a corollary of Lemma 10 and Theorems 1 and 3, with no separate argument.
• Definitions 18–19. Arborescent mutual information I_α(S;T;P) = δ_t H = Σᵢ [H(Tᵢ;P) − P(S=i)·H(Tᵢ;P|S=i)]. When every Tᵢ = T this equals I(S;T) + (m−1)·H(T) (Eq. 171). Higher orders are defined by analogy with Eqs. 55–56.
• Stated with the proof elsewhere.
  • Theorem D (odd I_N super-harmonic, even I_N sub-harmonic; the I_{N,J} for binary generators give algebraically independent coordinates on Δ(Ω)/finite group).
  • Theorem F (max-entropy weighing).

Claims

id	claim	strength	support
C1	Shannon entropy is, up to a constant per component, the only measurable 1-cocycle on connected, sufficiently rich classical structures with complete Q	strong	proof, Thm 1 (Eqs. 41–50 + Tverberg/Kendall/Lee). The proof’s cross-reference to Eq. 100 should be Eq. 44, and the hypotheses are stated inconsistently across Thm A, p. 3271 and Thm 1.
C2	No symmetry or refinement-stability axiom is needed for this characterisation	moderate	argued from the proof (p. 3273). Measurability and richness do the work.
C3	In the unitary-invariant quantum cohomology, H¹_U = 0 and von Neumann entropy is the unique invariant 0-cochain whose coboundary is −(Shannon entropy)	moderate	proof, Thm 3; step IV rests on the two-sentence Lemma 7
C4	Shannon entropy is a 1-cocycle of the arborescent (operadic) complex, and H¹_τ is generated by it	moderate	Prop. 8 is proved (strong). Thm 4 is asserted as a corollary without its own argument.
C5	Higher mutual informations are δ- or δ_t-images of H (Theorem B)	weak	sketch; Lemma 3 unproved here; deferred to [7] (in preparation)
C6	I₃ relates to Milnor/Massey linking invariants, and its minima are Borromean configurations read as synergy	weak	assertion with citations (§1.2)
C7	Odd/even I_N are super/sub-harmonic on the simplex (Theorem D)	weak (here)	stated; proof in [15] (in preparation)
C8	The max-entropy algorithm is optimal for the one-false-coin weighing problem (Theorem F)	weak (here)	stated; proof in [22] (in preparation)
C9	KL divergence of image laws is a 1-cocycle; −Tr(ρ log σ) generates the cocycles considered	strong / weak	Prop. 4 proof (strong); the “generating function” statement is informal
C10	Simplicial probability sub-complexes are exactly those cut out by “possible ⇒ impossible” rules	strong	proof, Thm 2
C11	Information topology “already proved its efficiency” for synergy in neural coding, gene expression, cancer and signalling	weak	assertion citing others’ applications of three-way mutual information (§1.7), not of the cohomology

Method

This is pure mathematics.

• Classical. A ringed site/topos over the category of partitions, with cohomology Ext*(ℝ_S, F(Q)), computed concretely by the non-homogeneous bar resolution (Eilenberg–MacLane/Hochschild coboundary, Eq. 22).
• Twisted complex δ_t. The same formula with the trivial action, used for the mutual informations.
• Quantum. Von Neumann conditioning ρ ↦ E_A ρ E_A / Tr(E_A ρ E_A), with invariant cochains.
• Dynamics. Ordered partitions with empty parts form an operad µ(m; n₁,…,n_m) of grafted observation trees. Its Schur monad acts on “divided probabilities”, and Beck’s bar construction for monads defines the arborescent complex.

Concepts

• Information structure — a set of partitions containing 1 and closed under joins of elements refined by a common element (§2.1). W(Σ) is the monoid generated by a family Σ and is identified with the barycentric subdivision of a simplex.
• Averaged conditioning — Y.F(P) = Σ_j P(Y=y_j)·F(P|Y=y_j). It is an action of the monoid (Lemma 1).
• Joint locality — a cochain depends on P only through the law of the joint variable of its arguments (Eq. 18).
• Complete Q — Q_X contains all laws on Ω/X.
• Rich edge — see Assumptions.
• Twisted (topological) coboundary δ_t — the bar coboundary with Y.F replaced by F.
• Q_min, Q_max, Q_can — the extreme choices of quantum density functor. Q_can is the minimal complete one (Def. 4).
• Trace — the natural transformation ρ ↦ P_ρ from quantum to classical laws (Def. 5).
• Divided probability — a sequence (pᵢ, Pᵢ, Uᵢ) with Σpᵢ = 1 and disjoint or orthogonal supports (Def. 16).
• Arborescent cohomology — the cohomology of regular cochains on forests of observation trees (Def. 17).
• Galois group of an edge — the permutations of X(α) that respect the allowed observables, the conditioned probabilities and the equations along the branch (Def. 12). Successive quotients are proposed as “Galoisian information”.

Connections

• Builds on:
  • Hu Kuo Ting (1962), whose set-theoretic reading of the multivariate mutual informations is claimed equivalent to Theorem B;
  • Tverberg (1958), Kendall (1964) and Lee (1964) on the fundamental equation of information;
  • Faddeev and Baez–Fritz–Leinster (2011) on the operadic characterisation of entropy;
  • Grothendieck/SGA 4 topos cohomology;
  • Beck (1967) and Fresse on monadic bar constructions;
  • Cathelineau (1988) on entropy in the homology of SL₂;
  • Döring–Isham’s topos formulation of physics, from which it explicitly distinguishes itself (§4.2).
• In this record:
  • It shares with LIT-016 (Abramsky–Brandenburger, sheaf-theoretic contextuality) the move of attaching probability data to a site of commuting (jointly measurable) observables. Both papers separate themselves from the Döring–Isham topos programme. The difference is that LIT-016 asks whether local data glue into a global section, while this paper computes the derived-functor cohomology of the conditioning module.
  • Its I₃ < 0 “synergy” reading is the same interaction information that LIT-042 (Shannon invariants; reads/c35.md) shows equals redundancy minus synergy for n = 2 under any PID (its Eq. 3). That is a concrete bridge between the two information-decomposition threads in this record.
• In the anthology. ANTH-LIT-698 (Belfiore & Bennequin, “Topos and Stacks of Deep Neural Networks”) defines semantic information “by analogy with the Baudot–Bennequin homological reading of Shannon entropy”. This paper is the reading it refers to.

Bearing on the record

• For this record.
  • It is a mathematics/information-theory foundation. No THEORY document here depends on it.
  • If one is written on the axiomatic characterisation of entropy, this paper supplies a characterisation that needs no symmetry or continuity axiom, only measurability. It also supplies a clean, cohomological statement of how classical entropy “dies” when measurement becomes quantum.
  • Its higher-order and synergy claims should not be cited from this paper as established. They are sketched or deferred.
• For the Anthology of the SOTA. It carries nothing for ML practice. The only anthology document that touches it, ANTH-LIT-698, uses it by analogy, and the anthology needs nothing further from this reading.

Limitations

• Only degree one (and zero) is computed. Higher-degree cohomology is Problem 1, conjectured zero for binary variables and possibly non-trivial via polylogarithms otherwise.
• So the “topology of information” is, in this paper, a statement about H¹. The higher mutual informations appear as coboundaries, not as non-trivial classes.
• Theorems B, D and F, the “Borromean synergy” reading and the Hu Kuo Ting equivalence rely on companion papers listed as “in preparation” [7, 15, 22]. They are not established here.
• The hypotheses of the main theorem are stated three different ways across the paper. The proof of Theorem 1 cross-references the quantum recurrence (Eq. 100) where the classical one (Eq. 44) is meant. Theorem 4’s proof is a one-line appeal to earlier results.
• Everything is on finite Ω, or finite-dimensional E. The infinite case is deferred.
• The quantum theory restricts joint observables to commuting families by convention (“information requires a form of commutativity”, §4.2), and depends on a chosen h₀, which the authors discuss at length (§4.1).
• The application claims (§1.7) cite uses of three-way mutual information by others, not uses of the cohomology.

Open questions

• Problem 1: compute Hⁿ(S;Q) for n ≥ 2. Is it zero for binary variables, and are there polylogarithmic classes in general?
• Problem 2: the critical points and Morse theory of I_N on level sets of the other information functions.
• Problem 3: generalise Theorem F; use Galoisian or higher tree informations for optimal discrimination.
• Is there a uniqueness theorem for KL divergence as a 1-cocycle of pairs of laws, analogous to Theorem 1? The paper shows only that it is a cocycle.
• Did the companion papers “Information Topology I–III” appear, and do they prove Theorems B, D and F as stated? This was not checked here.

Corrections to the seeded skim

• Access. The seed could not reach the full text. It is reachable at MDPI’s CDN (mdpi-res.com), though not at www.mdpi.com.

• The quantum case is not “the same”. The dossier’s summary says “the same holds for quantum probabilities”. The body says the opposite for degree one.

  • In the unitary-invariant quantum cohomology, H¹_U(S;Q) = 0 (Theorem 3(i), pp. 3290–3293).
  • Shannon entropy of a decomposition is still a 1-cocycle, but it is a coboundary: δ̂(S_X)(Y;ρ) = −H_X(Y;ρ) (Lemma 6).
  • The theorem’s positive content is Theorem 3(ii). The only invariant 0-cochain whose coboundary is (minus) Shannon entropy is the von Neumann entropy, up to an additive constant.
  • The authors themselves compare this with Souriau’s observation that mass is a non-trivial class for the Galilei group and a trivial one for the Poincaré group (Remark 8).
  • The abstract’s “entropy is the first homological invariant … in four contexts” is therefore true classically, while in the invariant quantum context the invariant is the von Neumann 0-cochain.

• What “universal” means in Theorem A. This answers the dossier’s question.

  • Degree-0 cochains are constants and all are cocycles (Eq. 28), so there are no non-trivial 1-coboundaries and H¹ = Z¹.
  • “Generated by entropy” therefore means that every measurable 1-cocycle equals C·H for one constant C per connected component of S.
  • This is a uniqueness-up-to-scale characterisation of Shannon entropy. It needs no symmetry or continuity axiom, only measurability. The proof rests on Tverberg–Kendall–Lee’s result on measurable solutions of the fundamental equation of information (Eq. 48).

• The hypotheses are stated three different ways.

  • Theorem A (§1.2): the monoid is “generated by a set of at least two variables, such that each pair takes at least four values”, on the full simplex.
  • §2.3, p. 3271: “Σ has more than two elements and Ω more than four elements”.
  • Theorem 1: S connected and “sufficiently rich” (every vertex on a rich edge, i.e. X, Y with ≥ 2 values each and XY cutting them into ≥ 4 pieces), and Q complete.

  Theorem 1 is the operative statement. The proof of Theorem 1 cites “the recurrence Equation (100)”, which is the quantum recurrence in §4.3. The classical recurrence is Eq. 44. Theorem E (§1.5) says “if Ω has more than four points”, while Theorem 4 assumes S(A) connected, sufficiently rich and Q canonical.

• Higher mutual informations are coboundaries, and Theorem B is only sketched.

  • The dossier says higher mutual informations “arise as cocycles” and are “most likely to matter”. The paper’s formulas are I_{2m} = δ_t δ δ_t … δ δ_t H and I_{2m+1} = −δ δ_t … δ δ_t H (Eqs. 55–56). They present the higher mutual informations as images of δ or δ_t, i.e. coboundaries (trivial classes) in the respective complex. This is said explicitly on p. 3276.
  • Theorem B rests on Lemma 3, which is “proved by comparing the completely developed forms” (not shown), and on a reference to “Information Topology I, in preparation” [7].
  • §3 is titled “A Sketch”.

• Theorems D and F are not proved in this paper.

  • Theorem D (odd I_N super-harmonic and even I_N sub-harmonic on the simplex) is cited to “Information Topology II, in preparation” [15].
  • Theorem F (the max-entropy algorithm solves the one-false-coin weighing problem for N ≥ 3) is cited to “Information Topology III, in preparation” [22].
  • The abstract says only that A, C and E are proved, and that is accurate.

• KL divergence is shown to be a cocycle, and nothing more. Proposition 4 shows that H(Y∗P; Y∗Q), the KL divergence of the images, is a non-homogeneous 1-cocycle for pairs of laws. The paper proves no uniqueness theorem for it (§5 is “short”). The dossier’s “accounts for … KL divergence” should be read as “KL is a cocycle”.

• Minor points.

  • The published version is an extended version of the MaxEnt 2014 proceedings paper [14].
  • Theorem C explicitly “corrects a similar statement which was made in the announcement [14]” (p. 3258).