nucleation

LIT-260: Kernel Mean Embedding of Distributions: A Review and Beyond

3 min read

Properties5
StatusActive — read 2026-10-14 (NOTE-838) in full; worth reading as the one held text that runs the chain bounded evaluation → Riesz → k_x → Moore–Aronszajn → μ_P → MMD/HSIC in numbered statements, with Lemma 3.1 (μ_P exists and represents expectation when E_P√k(X,X) < ∞) proved in the text, and as a map of conditional mean embeddings and their limits. Read it knowing that it is a survey whose proofs are mostly sketched or cited, that every result assumes a fixed kernel, that it defines μ_P as a Bochner integral as well as by Riesz, so Riesz is one route to existence and not shown to be the only one, and that the unregularized conditional embedding C_YX C_XX⁻¹ often fails to exist.
Version2
Tagsprobabilistic-modeling, mathematics, learning-theory, anthology-candidate
Cited as “paper” byNOTE-226 — Kernel mean embedding of distributions (review), NOTE-838 — Muandet et al. — Kernel Mean Embedding of Distributions
Cited as “source” byTHEORY-003 — Maximizing HSIC between representations and image identity maximizes the average squared MMD between the images' view distributions, whose kernel mean embeddings exist by the Riesz representation theorem

Graph View

Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, Bernhard Schölkopf (2016), Foundations and Trends in Machine Learning 10(1–2):1–141 (2017); first appeared as arXiv preprint — ARXIV-1605.09522

Key takeaways

From the reading, NOTE-838.

  • Riesz makes the kernel. An RKHS is defined by bounded evaluation (Def. 2.4), Riesz is stated (Thm 2.4), and the representer k_x with f(x) = ⟨k_x, f⟩ follows (Prop. 2.1). Aronszajn’s theorem (Thm 2.5) runs the other way, from a positive-definite k to its unique RKHS.
  • Riesz makes the mean embedding. If E_P√k(X,X) < ∞, then |E_P f| ≤ E√k(X,X)·‖f‖, so Riesz gives μ_P ∈ H with E_P f = ⟨f, μ_P⟩, and μ_P = ∫k(·,x)dP(x) (Lemma 3.1, proved, after Smola et al. 2007). The review also defines μ_P as a Bochner integral, a second route to the same object under the same condition.
  • MMD and HSIC are geometry of embeddings. MMD over the RKHS unit ball is ‖μ_P − μ_Q‖ (eq. 3.29), a metric on distributions iff k is characteristic (Def. 3.2). HSIC is the squared Hilbert–Schmidt norm of the centred cross-covariance, zero iff independence when the product kernel is characteristic (§3.6).
  • Operators. C_YX is the embedding of the joint law, C_XX is positive and self-adjoint, and C_XX on L²(P_X) has the Mercer operator’s eigenvalues (§3.2). The conditional mean embedding C_YX C_XX⁻¹ needs E[g(Y)|X = ·] ∈ H, which fails for Gaussian kernels in simple cases, so a regularized form is used (§4.1, Remark 4.1).
  • What it does not cover. No result for learned kernels or feature maps. Kernel choice is “an ultimate problem” (§6.1).

Standing in the record

Filed on 2026-09-26 while pursuing, at the owner’s request, the connection Riesz, reproducing kernels and spectral representation learning (see the curation entry of that date). Read on 2026-10-14 in NOTE-838.

It is the second source of THEORY-003, whose statements from the review (Thm 2.4, Prop. 2.1, Lemma 3.1 with its condition, Def. 3.2, §3.5) the reading confirms. THEORY-009 cites Lemma 3.1 for Riesz making the mean embedding exist. The review supports “Riesz suffices”, not “Riesz is needed”, since it also defines μ_P as a Bochner integral. No theory was filed from this reading: the review is a survey, and its load-bearing statements are already held in THEORY-003.

Tagged anthology-candidate (ADR-005): the seed judged it chiefly about machine-learning practice, and its applications chapters are. It is kept here by the owner’s decision of 2026-09-26 that new work stays in nucleation until a transfer is judged appropriate (ADR-010).

Backlinks