Topics — The twenty-eight kinds of claim this anthology files against — the primary axis of both the practice registry and the reading list, and the same twenty-eight for each (ADR-026). The filing rule is the point: take a domain topic when the claim is ABOUT the domain; a claim merely discovered in one still takes its kind. The axis organizes the record, it does not bound it: content the list cannot place is evidence the list is short (ADR-059).
In-context learning — getting behaviour out of a fixed model by what you put in the context — few-shot exemplars, chain of thought, prompting strategy, and what the context can and cannot buy.
4 of 129 THEORY documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| THEORY-038 | A decoder cannot compose over a long context in few layers, because each position forgets what it forwarded | Chen et al. (2024), LIT-464 — causal masking makes a decoder a line of forgetful communicating players, one epoch per layer, and sequential composition needs more epochs than a constant-depth model has. An unconditional bound on what can be expressed, not on what can be learned. | Active |
| THEORY-067 | A transformer can run a learning algorithm on a model held in its activations, and the architecture admits several, so a construction identifies none | Three groups have constructed transformer weights that run a named learning algorithm in the forward pass on a model carried in the activations: one gradient step per linear-self-attention layer (von Oswald et al., LIT-533), a Sherman–Morrison ridge update in constant depth and O(d²) width (Akyürek et al., LIT-532), and k Iterative Newton steps in k + 8 layers (Fu et al., LIT-535). The frame is what survives the dispute about which algorithm; the corollary is that no construction can settle that dispute, because the architecture admits all three. | Active |
| THEORY-068 | In-context learning is gradient descent on an implicit model | The identification the field read off LIT-533’s title: that what a transformer does with its context is gradient descent on a model held in its activations. Filed Rejected and kept, on three separate grounds. Beyond one layer the source paper’s own models match GD++ — gradient descent on data transformed by I − γXXᵀ — not gradient descent. LIT-535 matches layers to steps and finds a linear correspondence with Iterative Newton and an exponential one with GD, and a conditioning regime GD cannot reach. LIT-536 proves an equivalent algorithm must share ICL’s order sensitivity, which GD has none of. What survives is THEORY-067, the frame without the name. | Rejected |
| THEORY-073 | Training passes through stages marked by changes in loss-landscape degeneracy, and the loss curve does not show them | Hoogland et al. (LIT-543) estimate the local learning coefficient through training and take critical points of that curve as stage boundaries. In a 2-layer language transformer the five stages land on bigrams, n-grams, previous-token heads and the induction circuit; in an in-context regression transformer they land on the acquisition of in-context learning and then its deterioration. The claimed advantage over a progress measure is that no mechanistic hypothesis is needed in advance. The evidence is coincidence across two models, and the authors call it suggestive. | Proposed |