nucleation

LIT-941: A Theory of Unsupervised Translation Motivated by Understanding Animal Communication

2 min read

Properties5
StatusActive — read 2026-10-14 (NOTE-804) in full; worth reading as the first theory of translation without parallel data: maximum likelihood against a plausibility prior learns the translator with an Occam-type bound divided by the common ground, err = O(log|Θ| / (α·min(m,|T|))) in its common-nonsense model, tight up to a constant. Read it knowing that its models are uniform, random and lossless, that the bounds are information-theoretic, and that the prior substitutes for parallel data in learning a decoder, not for information in a message. Filed THEORY-270; bears on QUESTION-020 without answering it.
Version2
Tagstranslation, learning-theory, information-theory, anthology-candidate
Cited as “paper” byNOTE-804 — Goldwasser et al. — A Theory of Unsupervised Translation Motivated by Animal Communication
Cited as “source” byTHEORY-270 — In Goldwasser, Gruber, Kalai and Paradise's common-nonsense model, where the source language and the plausibility prior are uniform over sets from which shared random nonsense is removed and translators are one-to-one, maximum likelihood with the prior alone translates without parallel data with error O(log|Θ| / (α·min(m, |T|))) with high probability, falling in the shared nonsense α, in the size of the source language and in the number of samples, THEORY-806 — Translation without parallel data, as of an unknown animal communication system into a human language, is learnable from a plausibility prior over the target language alone, with error falling as the common ground between the languages grows and as the source language grows richer

Graph View

Shafi Goldwasser, David F. Gruber, Adam Tauman Kalai and Orr Paradise (2022), arXiv:2211.11081 (NeurIPS 2023).

Key takeaways

From the reading, NOTE-804.

  • A prior instead of parallel data. The learner sees unlabelled source texts and a prior ρ over target texts, and picks the translator that maximises Σ log ρ(f_θ(x_i)). A wrong translator is caught when some of its translations are impossible under the prior.
  • An Occam bound, unsupervised. If every bad translator makes more than a γ fraction of translations impossible, (1/γ) ln(|Θ|/δ) samples suffice (Theorem A.2).
  • Common nonsense. When prior and translations share a random α-fraction of impossible texts, err = O(log|Θ| / (α·min(m,|T|))), and no algorithm does better than a constant factor (Theorems 3.4, 3.5).
  • Knowledge graphs. With agreement α and per-node knowledge pr, err = O(log n/(α²pr) + (1/α)√(r log n/m)): richer languages need less common ground (Theorem 3.2).
  • About as much data as supervised translation, up to the factor 1/α, barring computation.
  • The limit. Uniform, random, lossless models; one-to-one translators into a target that can say everything; no efficient algorithm.

Standing in the record

Filed 2026-10-12 for QUESTION-020 and read 2026-10-14 in NOTE-804. Its results are held as THEORY-270. It was registered as “a formal case of the interpreter substituting for transmitted information”. That is too strong: the receiver’s prior substitutes for parallel data in learning the translator, at a rate of exchange 1/α against source data, and does not shorten any message. It is the record’s nearest formal result to QUESTION-020, not an answer to it. Its lossless assumption is the case THEORY-170 says is not the general one. The anthology-candidate flag, for its learning-theory content, is left as filed.

Backlinks