nucleation

THEORY-982: Claude 3.5 Haiku routinely computes intermediate steps within a single forward pass and plans its outputs ahead, representing a target before the text that leads to it and writing towards it, as Lindsey et al. propose from their case studies

2 min read

Properties4
StatusProposed
Tagsrepresentation-learning, cognition
SourceLIT-1541 — On the Biology of a Large Language Model
ExtendsTHEORY-984 — In Claude 3.5 Haiku, on the prompt "Fact: the capital of the state containing Dallas is", which it completes with "Austin", an attribution graph shows Dallas features activating Texas features that, with "say a capital" features, promote Austin, beside a direct edge from Dallas to "say Austin"; inhibiting the Dallas features lowers the Texas features and leaves "say a capital" largely intact, and replacing the Texas features with those of California, Georgia, British Columbia, China or the Byzantine Empire makes the model name that place's capital, THEORY-1017 — In Claude 3.5 Haiku, writing the second line of a rhyming couplet whose first line ends "grab it", features for the candidate rhymes "rabbit" and "habit" are active on the newline token before the line, steering them changes the line's final word only when applied at that token, and suppressing "rabbit" turns the line towards "habit"; injecting a "rabbit" or "green" word feature at that point in 25 other poems ended the line with the injected word in 70% of cases, and such planned-word features were found in about half of the poems the authors examined

Graph View

Source

Lindsey et al. (2025), LIT-1541, read in full in NOTE-1181: the Introduction, the case studies, and the Discussion.

What is asserted

That the mechanisms shown on single prompts are how the model generally works: “Claude 3.5 Haiku routinely uses multiple intermediate reasoning steps ‘in its head’ to decide its outputs. It displays signs of forward planning, considering multiple possibilities for what it will say well in advance of saying it. It performs backward planning, working backwards from goal states to formulate earlier parts of its response.”

What the record holds toward it

  • Two-hop. THEORY-984: one prompt and five swaps, with a direct shortcut beside the two-step path.
  • Planning. THEORY-1017: one couplet in depth, a 25-poem injection test, and planned-word features in “about half” of poems examined.
  • An unnamed intermediate. In the medical case, “preeclampsia” features are active though the word is not in the prompt, and inhibiting them changes the suggested follow-up question to one suited to the alternative diagnosis (NOTE-1181). One prompt; three further examples without interventions.
  • Backward reasoning in a chain of thought. One hinted cos(23423) prompt in which the stated intermediate is computed from the hint (NOTE-1181, C6).

What the record holds against it

  • The jailbreak. THEORY-1018: on one prompt the output word is assembled letter by letter with no internal representation before it is written, and the request is recognized only from the model’s own text. The authors: “the model doesn’t know what it plans to say until it actually says it”.
  • Shortcuts beside steps. The two-hop case has a direct Dallas → say Austin edge, with its share of the output unmeasured.

Why it stays Proposed

The authors’ own Limitations: “Our results are only claims about specific examples. We don’t make claims about mechanisms more broadly. For example, when we discuss planning in poems, we show a few specific examples in which planning appears to occur. It seems likely that the phenomenon is more widespread, but it’s not our intent to make that claim.” And the selection: a satisfying graph for “about a quarter of the prompts we’ve tried”, with key nodes hand-tagged to survive the extra pruning applied to the graphs shown (appendix “Graph Pruning and Visualization”). The same article contains a case without a represented plan.

What would confirm or reject it

The promote_when above.

Backlinks