nucleation

THEORY-811: With a kernel normalised to unit diagonal, the kernel quantum relative entropy of two distributions is within O(σ²) of their Kullback–Leibler divergence as the bandwidth σ shrinks, for translation-invariant kernels on the torus generally and for kernels on other compact groups

2 min read

Properties5
StatusProposed
Tagsinformation-theory, mathematics, probabilistic-modeling
SourceLIT-255 — Information Theory with Kernel Methods
ExtendsTHEORY-313 — With a kernel normalised to k(x,x) = 1, the covariance operator of a distribution is a density operator, and its quantum relative entropy is a lower bound on the Kullback–Leibler divergence that is tight for every pair of distributions only for an orthonormal, hence discrete, embedding, lies above the KL of the kernel-smoothed distributions, and on the torus, with Fourier coefficients proportional to e^(−σ‖ω‖₁) and strictly positive Lipschitz densities, is within O(σ²) of KL as the bandwidth σ shrinks
AnswersQUESTION-198 — Does the O(σ²) gap between the kernel and Shannon relative entropies hold for translation-invariant kernels on the torus beyond the exponential Fourier family, and on other compact spaces?

Graph View

Source

Bach (2022), LIT-255, read in NOTE-837: Proposition 5 and the torus computation (claim C3, strong for the exponential Fourier family), and the assertion, with a leverage-score sketch, that the rate extends.

What is asserted, and by whom

The thesis is that the O(σ²) gap between the kernel and Shannon relative entropies is not special to kernels with Fourier coefficients proportional to e^(−σ‖ω‖₁): it holds for translation-invariant kernels on the torus generally, and on other compact groups, for strictly positive Lipschitz densities. Bach asserts it; the argument is a sketch.

What the record holds toward it

  • THEORY-313 (Active): the bounds D(p̃‖q̃) ≤ D(Σ_p‖Σ_q) ≤ D(p‖q) and, for the exponential Fourier family on the torus, a gap of at most dσ²/16.
  • The lower bound by the smoothed KL holds for any unit-diagonal kernel, so the thesis reduces to how fast the smoothed KL approaches the KL.

What would confirm or reject it

A completed proof for another translation-invariant family on the torus, or for a kernel on another compact group, would confirm it; a family on the torus with a strictly slower gap for Lipschitz densities would reject it. QUESTION-198 asks this as a question; this entry is the answer Bach proposes.

Backlinks