nucleation

LIT-1818: Gromov-Wasserstein Alignment of Word Embedding Spaces

1 min read

Properties5
StatusActive — read in full 2026-10-29 (NOTE-1336); worth reading as the cleanest statement of correspondence recovered from within-space relations alone, and for what that costs: the objective cannot tell a correspondence from one composed with a symmetry of either space (THEORY-1133), which the paper does not say.
Version2
Tagstranslation, mathematics, representation-learning, anthology-candidate
Cited as “paper” byNOTE-1336 — Alvarez-Melis & Jaakkola — Gromov-Wasserstein word alignment
Cited as “source” byTHEORY-1133 — The Gromov–Wasserstein objective between two spaces depends on each only through its within-space similarity matrix and weights, so every transformation of a space that preserves them, a single orthogonal map or per-vector rescaling under cosine similarity, leaves every coupling's cost unchanged, and any coupling composed with a symmetry of either structure costs exactly what it did; GW can therefore identify a correspondence only up to the automorphisms of the two similarity structures, and on symmetric data it cannot choose between branches or categories that the symmetry exchanges, THEORY-1145 — Unsupervised alignment of two monolingual word-embedding spaces recovers translations only where the spaces are close to isometric, and the distance from isometry depends on the corpora and the embedding algorithm as well as on the languages: adversarial alignment fails for English with Estonian or Finnish, across corpus domains, and between skip-gram and CBOW spaces of the same language pair, GW alignment is weakest on the same Finnish pair, and a spectral dissimilarity of small nearest-neighbour graphs tracks success

Graph View

Alvarez-Melis, D., & Jaakkola, T. S. (2018). Gromov-Wasserstein alignment of word embedding spaces. Proceedings of EMNLP 2018, 1881–1890.

Read in full in NOTE-1336.

Why this is here

The centre of A6: correspondence recovered from the relations elements bear to one another, without a shared coordinate system. That is the structuralist reading of CLAIM-046 and CLAIM-113 in computational form. A6 also gave its limit, kept here: GW finds relational resemblance under a chosen metric, which is not communicative identity (CLAIM-135). Read with LIT-1819. The reading bears A6’s limit out and sharpens it: the relational resemblance is resemblance up to the symmetries of the similarity structure, so on symmetric data even a perfect fit leaves the correspondence open (THEORY-1133). Against the paper’s own claim (p. 9), GW does not dispense with an invariance assumption; under cosine similarity it builds in invariance to every rotation and rescaling of either space.

Backlinks