nucleation

LIT-946: The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models

2 min read

Properties5
StatusActive — read 2026-10-14 (NOTE-789) in full; worth reading for its split of accuracy into representational separability and readout: demonstrations reshape how the input itself is represented, driven by the example sentences rather than their labels, an instruction acts mainly on the last token, and few-shot failures from example choice or relabelled categories leave the classes separable. Read it knowing that it uses base models, one instruction wording and mostly synthetic data, and infers the readout failure rather than measuring it. Filed THEORY-251, and supports THEORY-264 with LIT-942.
Version2
Tagsrepresentation-learning, anthology-candidate
Cited as “paper” byNOTE-789 — Kirsanov et al. — The Geometry of Prompting
Cited as “source” byTHEORY-251 — In few-shot classification by a base language model, failures from the choice of examples or from relabelled categories leave the class representations separable and fail at the vocabulary readout, THEORY-264 — A task given by demonstrations and the same task given by an instruction are carried in a language model by mostly different attention heads, so function vectors find no common task representation across the two presentations

Graph View

Artem Kirsanov, Chi-Ning Chou, Kyunghyun Cho and SueYeon Chung (2025), Findings of the Association for Computational Linguistics: NAACL 2025.

Key takeaways

From the reading, NOTE-789.

  • Same accuracy, different mechanisms. In base Llama-3.1-8B (and Gemma-2-2B), an instruction beats up to five demonstrations and more examples overtake it; a trained soft prompt beats both.
  • Demonstrations reshape the input. They raise the separability of the input sentence’s representation at early-to-middle layers; an instruction does not. Gold, letter and shuffled labels make little difference: the examples’ sentences do the work.
  • An instruction acts on the last token, raising separability from about layer 8 on, less than demonstrations do.
  • Failures sit at the readout. At five demonstrations accuracy varies from below 0.1 to about 0.6 across draws of examples while separability barely moves; consistently swapped labels fail with separability intact.
  • Cross-task effects. Demonstrations for one labelling of the same sentences help other labellings early and hurt them late; soft prompts hurt them more.
  • The limit. Base models only, one instruction wording, mostly synthetic data, and a readout explanation inferred, not measured.

Standing in the record

Filed 2026-10-12 for QUESTION-024 and read 2026-10-14 in NOTE-789. Its readout finding is held as THEORY-251, and its demonstrations-against- instruction finding supports THEORY-264 with LIT-942. It was registered as counting “against a single frame induced from examples”; it does not test that, since it holds the instruction’s wording fixed and varies the prompt form. It points the same way as LIT-942, not the opposite way as the triage of 2026-10-14 had it. Its suggestions for prompt design belong to the anthology; the anthology-candidate flag stands.

Backlinks