Aaron J. Gutknecht, Fernando E. Rosas, David A. Ehrlich, Abdullah Makkeh, Pedro A. M. Mediano, Michael Wibral (2025), arXiv preprint (cs.IT) — ARXIV-2504.15779
Key takeaways
- Two averages over the partial information decomposition can be computed from Shannon entropies alone, with the number of entropies growing linearly in the number of sources. These are the average degree of redundancy and the average degree of vulnerability. They sidestep PID’s ambiguous choice of redundancy measure and its super-exponential cost, explain what the old redundancy–synergy index actually measures, and show redundancy increasing and vulnerability decreasing with depth and training in small quantised networks.
Seeded from the abstract and a skim, before the reading in NOTE-045. What follows is what the work says about itself.
Biological and artificial neural networks process information through high-order interactions among many parts, but multivariate information measures are hard to define and do not scale. The authors introduce “Shannon invariants”: quantities that capture properties of high-order information processing, depend only on the definition of entropy, and can be computed for large systems. In theory, they settle long-standing ambiguities about how to read common multivariate measures. In practice, they reveal information-processing signatures that differ across layers of deep learning architectures and change over training.
Standing in the record
Filed from the survey of 2026-09-25 of work the anthology set aside as out of scope (tier C): 350 seconds of active reading over 3 sessions in the papers-feed tracker. It was filed Deferred, unread. NOTE-045 is the close reading of 2026-09-25, and it placed the work: Active — the theory (§II–III, App. B) is short, correct and redundancy-measure-free. It gives the right reading of the old redundancy–synergy index and is worth reading for that. The “scalable” deep-learning half is a demonstration on tiny, quantised networks with information computed on the training set as the population. Read it as a proof of concept, not as evidence about realistic models.
When seeded, before the reading in NOTE-045, the entry gave a priority for a deeper reading: medium — 350 s over 3 sessions, and technically load-bearing for any information-decomposition thread. The skim gives the definitions and results but not the proofs, which are what a filing would lean on.
What the seeding asked a deeper reading to check (NOTE-045 is that reading):
- A scalable, measure-agnostic handle on redundancy and synergy is directly usable for interpretability and representation analysis. This is the “boundary” the triage flags, since the anthology’s ML side could also claim it.
- The DL results use small quantised networks. Check the estimation procedure (binning or quantisation, sample sizes) and whether anything scales to realistic models.
- Check the exact theorem linking RSI to average redundancy (§III.A) and the conditions under which the averages are well defined.
Access when seeded: arXiv abs page (v1, submitted 22 Apr 2025; comments “16 pages, 4 figures”) and the full v1 PDF, which I read: introduction, section heads, the deep-learning section (MNIST classifier and face autoencoder) and discussion. The theorem statements and appendices were only glanced at. A Crossref title search found no journal version, but that is unverified.