nucleation

LIT-1801: Word Embeddings as Metric Recovery in Semantic Spaces

1 min read

Properties5
StatusActive — read in full 2026-10-29 (NOTE-1329) from the ACL Anthology PDF; worth reading for the proof that, under a metric random walk, log co-occurrence over long windows converges to squared latent distance whatever the one-step link, so embeddings identify a metric and coordinates only up to isometry; its evidence that real corpora have a Euclidean semantic space is weaker than its theorem.
Version2
Tagsrepresentation-learning, mathematics, anthology-candidate
Cited as “paper” byNOTE-1329 — Hashimoto et al. — Word embeddings as metric recovery
Cited as “source” byTHEORY-1124 — Normalized log co-occurrences of words in large corpora behave as distances in a Euclidean semantic space: they correlate with human similarity ratings (r = 0.75 against MEN) and their nearest-neighbour centrality and reciprocity fall in the range expected of points sampled in a Euclidean space, apart from superordinate and function words, THEORY-1128 — If a sequence is generated by a random walk over points of a compact Riemannian manifold whose step probabilities depend only on (locally scaled) distance, then negative log co-occurrence over a window that grows suitably with the number of points, less a term for each point, approaches the squared geodesic distance, whatever the link from distance to one-step probability; so log co-occurrence statistics identify the metric, and coordinates only up to isometry

Graph View

Hashimoto, T. B., Alvarez-Melis, D., & Jaakkola, T. S. (2016). Word embeddings as metric recovery in semantic spaces. TACL, 4, 273–286.

Read in full in NOTE-1329.

Why this is here

A6’s precursor to LIT-1818: a coordinate system may be arbitrary while the relations it represents are not. A6’s caution is kept: a co-occurrence geometry is not a geometry of pragmatic meaning, and the record’s version would recover distances from receivers’ responses (CLAIM-046).

Backlinks