Source
Søgaard, Ruder and Vulić (2018), LIT-1819, §§2 and 4, Tables 2–3 and Fig. 2, pp. 2–9 of arXiv 1805.03620v1; read in NOTE-1337. Alvarez-Melis and Jaakkola (2018), LIT-1818, Tables 1–2, pp. 7–8 of arXiv 1809.00013v1; read in NOTE-1336.
What was actually shown
Language pairs. Conneau et al.’s adversarial method with Procrustes refinement, on fastText Wikipedia embeddings, reaches P@1 81.89 for English–Spanish and 0.00 and 0.09 for English–Estonian and English–Finnish (Table 2). A 1.7-billion-word Finnish corpus does not help (P@1 0.0, p. 5). Estonian–Finnish aligns at 29.62.
Domains. With every corpus capped at 1.1M sentences, English–Spanish aligns when both corpora are from the same domain. Across EuroParl, Wikipedia and EMEA, “performance is close to zero” (p. 6).
Algorithms. English skip-gram against Spanish skip-gram aligns at 80.15–81.89 under changes of window and n-gram range. English skip-gram against Spanish CBOW aligns at 0.00–0.13 (Table 3), though the CBOW space is of normal monolingual quality (footnote 9).
A second method. GW alignment, which has no adversarial step, is level with the adversarial method on Romance and Germanic pairs. It reaches only 18.3 on English–Finnish in Dinu et al.’s data, against 33.5 for self-learning, and 45.1 on English–Russian in Conneau et al.’s data (LIT-1818, Tables 1–2).
The diagnostic. Δ, the summed squared difference of the top Laplacian eigenvalues of 10-node nearest-neighbour graphs, correlates with P@1 across the eight pairs of Table 2. Recomputed from the table: Pearson −0.889, Spearman −0.905. The paper reports ρ ≈ 0.89; the sign is negative because Δ is a dissimilarity.
What could have failed. Finnish could have aligned once its corpus was large. Spanish could have aligned across domains or across algorithms. Δ could have been unrelated to success. None of these happened.
What this does not say
- That the failures are impossible alignments. About 1 run in 10 on Estonian, Finnish and Greek is “on par with Turkish” (LIT-1819, footnote 5). Part of what is reported is the adversarial optimiser’s instability. The GW evidence is a single table, not a test on the paper’s settings.
- A typological law. The claim about languages that are neither dependent-marking nor isolating rests on three failing languages, which differ from English in Wikipedia size and, for Greek, script. Hungarian, which is agglutinative, aligns.
- That Δ is a usable predictor. It is computed on gold-translated subgraphs (Table 2 caption), so it presupposes the dictionary that unsupervised induction was meant to produce, and it rests on eight points.
- That a good fit establishes that matched words mean the same. The account is about when a fit exists. Where the fit is good it is relational resemblance under the chosen similarity (CLAIM-135). Within that fit, symmetric structure is left undetermined (THEORY-1133).
- That isomorphism fails in the exact sense the paper tested. Exact VF2 isomorphism of 10-node graphs fails almost always (p. 2). That is a different and much stronger property than the near-isometry the methods need.