Contribution
Collects a scattered set of observations — that networks trained on different data, with different architectures, on different objectives, and now in different modalities, end up representing data similarly — and proposes a single endpoint that would explain them. The endpoint is a representation of the joint distribution over events in the world that produced the observations, which the authors call the platonic representation. The paper is careful in an unusual way: it states a mathematical argument that holds only in an idealised case, says which case, and then runs the experiment that probes the boundary rather than one that confirms the headline.
Key insight
Convergence is not a fact about models; it is a claim about what they are estimating. Two networks agreeing is unremarkable if they saw the same data and share an inductive bias. The interesting version is two networks agreeing when they share neither — a vision model and a language model, with no common architecture, objective or input — and the only thing available to explain that is that both are estimating something about the world rather than about their inputs. Which reframes scale: a bigger model is not a better fit to a dataset, it is a less obstructed view of whatever the dataset is a projection of.
Assumptions
- The mathematical argument assumes bijective observation functions. The information in each projection equals the information in the world. Lossy or stochastic observations break it and the paper says so directly.
- Discrete random variables in the idealised world, so that bijections preserve probabilities.
- The contrastive-learner argument assumes sufficient data and optimisation to reach the minimiser.
- Vision and language are the two modalities studied; other modalities are expected to follow and are not shown to.
- Alignment is measured by agreement on pairwise distances between datapoints, not by any stronger notion of representational identity.
Key results
- Cross-modal alignment rises with scale. As vision models and language models get larger, they measure distance between datapoints in increasingly similar ways. Holds when: over the model families evaluated.
- Convergence spans architectures, objectives and datasets — surveyed from prior work, including that early layers are more interchangeable than later ones, and that a model trained on ImageNet can be aligned with one trained on Places-365 with performance retained.
- Weight-space convergence for models of the same architecture: the same basin up to permutation, which is what makes model merging possible at all.
- In the idealised world, convergence is to the same kernel — certain pairwise statistics of the underlying variable — for any modality.
- Caption density improves alignment. Denser captions of the same images align better with the visual representation. This is the hypothesis’s own predicted failure direction: denser captions are closer to bijective.
- Three proposed pressures: task diversity, data diversity, and the simplicity bias of larger models.
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | Representations are converging across models and modalities, more so with scale | moderate | direct measurement on vision and language families, plus a survey of prior convergence results |
| C2 | The endpoint is a representation of the data-generating world | weak | a conjecture, proved only for an idealised bijective world |
| C3 | Task, data and capacity pressures drive convergence | weak | three arguments, none isolated by intervention |
| C4 | Training on a second modality improves the first | moderate | cited rather than run here; the paper notes images improved text performance in a prior report |
| C5 | Scaling should reduce hallucination and bias amplification | weak | an implication of the conjecture, explicitly flagged as conditional on training data being lossless and diverse |
Method
Survey the existing convergence literature — across datasets, architectures, layers and weight space — to establish that the phenomenon is not one result.
Then measure it across modalities directly: take a set of vision models and a set of language models, and compare how each measures distance between the same datapoints, as a function of model size.
Then construct an idealised world of discrete events with bijective observation functions, and show that contrastive learners in that world converge to the same kernel regardless of which projection they see.
Then probe the assumption: vary caption density on fixed images, and check whether alignment moves in the direction the bijectivity assumption predicts.
Concepts
- Platonic representation — the hypothesised converged endpoint: a model of the joint distribution over world events, of which the observed modalities are projections.
- Representational alignment — agreement between two models on pairwise distances among datapoints. The operational measure throughout.
- Bijective observation function — the idealisation under which the proof runs; a projection losing no information about the world.
- Convergent realism — the philosophy-of-science position the authors name as the analogue: that science converges on truth.
Connections
Gathers a decade of alignment results — Lenc and Vedaldi on cross-dataset transferability, the Gabor-filter universality literature, the linear-mode- connectivity and model-merging line — and proposes one explanation where there were several local ones.
The weight-space half connects to model merging, which the record touches through weight averaging; the claim that same-architecture models land in the same basin up to permutation is what makes merging coherent rather than lucky.
Against the multimodal training literature: what is common practice for vision (finetune from a pretrained LLM) is argued to hold in the other direction too, and that direction is much less common.
Recommendations
- R1 — When training a single-modality model, include data from another modality. Topic: data. Status: experimental. Strength: moderate. Applies when: the target is a general representation rather than a narrow task; the paper suggests a conversion ratio exists without giving one.
- R2 — Expect representations to be more portable across models than intuition suggests, and adapt across modalities by mapping representations rather than retraining. Topic: transfer. Status: experimental. Strength: moderate. Applies when: both models are large enough to be in the converged regime.
- R3 — Do not expect convergence where the modalities carry genuinely different information. Topic: modelling. Status: standard. Strength: strong. Applies when: always; this is the hypothesis’s own boundary.
Bearing on the record
- Should produce a theory for C1 and C2, kept separate: the convergence is measured and the endpoint is conjectured, and a document that blurs them would overclaim.
- Should produce a practice for R1, which is concrete, has a cited instance, and contradicts common practice in one direction while matching it in the other.
- Fills the other half of
representation-and-encoding. That topic’s fourteen notes are entirely tokenisation and positional encoding; nothing in the record says what a representation converges to. - Names an absent trunk. The Linear Representation Hypothesis is the foundation this and LIT-460 both build on, and the record holds no document for it.
- R2 should not be filed. “Representations are portable” is not an instruction anybody can act on without the mapping, and the paper does not supply one.
Limitations
- C2 is a conjecture with a proof for a world nobody trains in. The bijectivity assumption is exactly what real observation violates.
- Alignment is pairwise-distance agreement, which is a weak notion of sameness — two representations can agree on all pairwise distances and differ in ways that matter downstream.
- Vision and language only; robotics is named as a case where convergence has not happened, attributed to a data bottleneck rather than tested.
- The three pressures are arguments, not ablations.
- C5 is an implication rather than a finding, and the authors flag its condition; it is the claim most likely to be quoted without it.
Open questions
- What is the conversion ratio? The paper says a pixel should be worth some number of words for training language models and does not estimate it, which is what would turn R1 from a direction into a recipe.
- Does convergence continue, or is there a capacity at which models stop getting more alike? Nothing here distinguishes an asymptote from a trend.
- Is pairwise-distance alignment the right measure? A stronger notion might show convergence is shallower than it looks.