nucleation

LIT-797: An Explanation of In-context Learning as Implicit Bayesian Inference

3 min read

Properties4
StatusActive — read 2026-10-09 (NOTE-633); worth reading as the cleanest formal statement of in-context learning as posterior inference over a latent concept: if pretraining documents are a mixture of HMMs and the model fits that mixture exactly, then conditioning on a prompt of concatenated examples selects the prompt's concept by posterior concentration, despite the prompt being low-probability under pretraining, provided each example carries more KL signal about the concept than the error from its unnatural boundaries (Theorem 1). The guarantees are asymptotic in the number of examples and say nothing about how a trained network computes the posterior; the experiments are on a synthetic dataset (GINC) built to satisfy the assumptions.
Tagsprobabilistic-modeling, learning-theory, anthology-candidate
Cited as “paper” byNOTE-633 — Implicit Bayesian inference account of in-context learning
Cited as “source” byTHEORY-614 — If pretraining text is a mixture of hidden Markov models indexed by a latent concept and a model fits that distribution exactly, then a prompt of concatenated examples moves the posterior onto the prompt's concept, and the in-context predictor becomes Bayes-optimal as examples accumulate, whenever each example's evidence for the concept outweighs the cost of the improbable transitions between examples; whether trained networks learn in context this way is not shown

Graph View

Sang Michael Xie, Aditi Raghunathan, Percy Liang and Tengyu Ma (2021), ICLR 2022 — ARXIV-2111.02080

Key takeaways

  • The model of the data, not of the network. A document is generated by drawing a concept θ from a prior and then a sequence from the HMM whose transition matrix θ fixes. The LM is assumed to fit this distribution exactly, so in-context learning becomes a question about the pretraining distribution’s own conditional p(output | prompt), which marginalises over θ (equation 1). In-context learning is “implicit Bayesian inference” in that sense and no other: nothing is claimed about how a network computes it.
  • The mismatch is the hard part. A prompt concatenates independent examples, so its transitions at example boundaries are improbable under pretraining, and Bernstein–von Mises does not apply. With delimiter states that every concept emits alike (Assumptions 1–2) and a prompt start distribution close to the pretraining one (Assumption 3), the likelihood factorises across examples with O(1) error each.
  • Theorem 1 (distinguishability). If, for every θ ≠ θ*, the summed per-token KL divergence over one example exceeds the boundary error terms (Condition 1), the posterior concentrates on the prompt concept θ* and the in-context predictor’s 0-1 risk tends to the optimum as n → ∞. Longer examples carry more signal, so information in the inputs, not only in the input-output mapping, drives it.
  • Theorems 2–3 (no distinguishability). With a second-order Taylor expansion of the KL (Fisher information) or test inputs of random length, the excess risk is bounded by g⁻¹(O(γ·sup(error)/(k − 1))), roughly O(1/k) in the example length.
  • GINC experiments. Accuracy rises with the number and length of examples; pretraining on one concept, or on random transitions, gives no in-context learning; prompts from concepts outside the pretraining family fail. Larger Transformers do better at the same pretraining loss, LSTMs beat Transformers, the 24 orders of the same four examples vary by 10–40 points, and zero-shot can beat few-shot when transitions are low entropy.

Standing in the record

Filed on 2026-10-09 at the owner’s request, from the bibliography of the owner’s manuscript What Survives Translation? (work what-survives-translation), which considered it and dropped it from the final reference list. Read on its own terms (NOTE-633).

It carries anthology-candidate: its subject, how in-context learning arises from pretraining, is one an anthology topic holds, and the anthology has deferred it rather than declined it. Here it is read for its formal content as a statement about inference under misspecification: when a context of concatenated examples, improbable as a whole, still identifies the latent variable that generated each example.

Backlinks