LIT-450: How do language models learn facts? Dynamics, curricula and hallucinations

Key takeaways

• Three phases, not a curve. A short first phase learning generic attribute-value statistics; then a plateau at exactly the loss an ideal model with no individual-specific knowledge would reach; then the acquisition of subject-attribute associations. The plateau’s length grows with the number of individuals — “almost linearly” in the text, fitted as 0.43·N^0.81 (R² = 0.998, 4k–256k individuals, five seeds), which is sublinear.
• The plateau is the recall circuit being built, and the evidence is an intervention rather than a correlation. Patching in the attention patterns of a reference checkpoint makes the plateau disappear; patterns taken progressively later in the plateau are progressively better. Before the circuit exists, prediction errors at the attribute token do not backpropagate to the name tokens, so the rest of the model cannot learn.
• Attention patterns from very early training are worse than the untrained ones — the model first attends to attribute-type tokens to predict the generic distribution, which is exactly the wrong place for recall.
• A trade-off in the data distribution. The paper argues (§3.1, “we can already develop an intuition”) that plateau length is set by the frequency of the most common individuals and acquisition speed after it by the frequency of the least common; neither dependence is isolated experimentally. What was measured, sampling individuals ∝ i^(−α) with α ∈ {0, 0.2, …, 1} (1 is Zipf): the plateau shortens as α rises to an optimum of 0.6–0.8 irrespective of population, and excessive imbalance hurts, “likely due to overfitting”; the α minimising final attribute loss grows with the population and as the step budget shrinks (Fig. 4). Single seed.
• A data schedule beats the fixed distributions tried. What was tested is a warm-up: train uniformly on a subset of individuals for a few epochs, then uniformly on all of them (§3.2, App. B.2). Gains are largest when the population is large. A control argues against mere shrinking: uniform training on a population of 83k scores 97.72% on it, which is 63.26% counted over 128k individuals, against 94.92% for the warm-up on 128k (App. E.2). No skewed distribution annealed toward flat was run, the grid optimum for 128k lies at the grid’s edge, and it is single-seed. The authors call it a promising, “preliminary and task-specific” case of a curriculum helping self-supervised learning. Corrected 2026-09-30. This read “start imbalanced to escape the plateau, then flatten … Optimal fixed imbalance is an inverse power law with exponent between 1 and 2”; no exponent above 1 was tried, and the frequency dependences were presented as measured.
• Hallucinations arrive with knowledge, not after it. Overconfident predictions on unseen individuals begin “simultaneously” with individual-specific knowledge (§4.1) — “shortly after the plateau phase” in App. F.2 — though the model is less confident on unseen individuals than on seen ones, so they “can be detected to some extent” (Fig. M).
• Fine-tuning is a poor way to add knowledge about new individuals: their facts are absorbed slowly and existing parametric memories are corrupted within a few hundred steps; replay only partially mitigates it. Fine-tuning on rare individuals seen in pre-training is the counter-result: their accuracy rises from about 50% to close to 70% without the collapse (App. F.3, Fig. P). Single seed.

Standing in the anthology

It supplies a mechanism for something the record has only ever recorded as a phenomenon. Loss plateaus appear throughout the record’s training material as things to wait out or schedule around. This says that at least one kind of plateau is a circuit under construction, and that the stall is causal: without the extraction circuit, gradient signal does not reach the tokens that need it. THEORY-028 is that account, and the patching experiment is why it is more than a story.

It gives the record its first data-schedule practice with a stated mechanism. SOTA-267 is the recommendation. The record holds several curriculum practices (SOTA-129, SOTA-130, SOTA-123) and all of them are about stages of task — pretrain, then SFT, then RL. This is a curriculum over the distribution within a stage, justified by two quantities moving in opposite directions, which is a different and more falsifiable kind of claim. The two dependences are the paper’s argument; what it measured is their consequences — plateau end and final loss against α, population and budget — and a single-seed warm-up that beat the fixed distributions tried.

The fine-tuning result belongs beside SOTA-199, which regularises a narrow fine-tune against the base model’s own samples to stop it drifting. That practice treats drift as a side effect to be contained; this says that for knowledge about new entities specifically the corruption is fast and the absorption is slow, so the trade is bad before any regulariser is applied. Fine-tuning on rare entities the model already saw in pre-training did not show the collapse.

What it is, and is not. A synthetic biography task, chosen so that knowledge can be counted rather than probed. Everything here is a claim about that setting, and the value is that the setting admits interventions — attention patching, exact population control — that a natural corpus does not.