Source
Grosse et al. (2023), LIT-401 — ARXIV-2308.03296.
What was measured
Synthetic training sequences were constructed for two queries about fictional entities — the first President of the Republic of Astrobia, and the composition of a substance called Gleem — and for an English→Mandarin translation query, and their influence on the model’s completion was measured directly.
A sequence counts only when the prompt-related phrase precedes the completion-related one. Reversing the order while holding the content identical collapses the influence: for the translation query, a Mandarin-then-English sequence scores 0.030, against 0.020 for a sequence containing no English at all. The signal from the reversed sequence is barely distinguishable from the signal from an unrelated one. Consistent across model sizes.
The measurement is LIT-401’s, and the practice is first drawn from it. Grosse et al. scaled influence functions to models of up to 52B parameters with an EK-FAC approximation, and ran this synthetic construction with the content held fixed so that only the order varied. They report the asymmetry as a finding about generalization; the corpus instruction is the record’s reading of it, and nothing in the paper trains on a reordered corpus.
Why the corpus is where this has to be fixed
The effect is not a quirk of prompting, and it is not something a retrieval step patches. If a sequence in the training data teaches only the direction it was written in, then the ordering of statements inside documents is a corpus property with consequences, and the only place to act on it is when the corpus is built.
Concretely: a knowledge base rendered to text as X is the Y of Z throughout
teaches X → Y of Z and does not teach Y of Z → X. Generating the reverse
rendering is cheap and mechanical wherever the relation is structured, which
is exactly where this kind of systematic one-directionality arises.
The gap between the evidence and the instruction
Influence is not accuracy. What was measured is which training sequences
the model’s completion can be attributed to, not whether a model trained on a
one-directional corpus fails to answer the reverse question. The inference
from the first to the second is natural and is an inference, and the
promote_when above is what would close it.
And the attribution estimate has its own caveat. Per THEORY-018, influence functions on neural networks estimate the proximal Bregman response function rather than the effect of removing the sequence and retraining. The asymmetry is a real property of that estimate. Whether it is a property of what the model learned is the same question as the paragraph above, arrived at from the method’s side.
Conditions
Twenty-eight data-pipeline practices and none of them is about what is
inside a document.
They govern what to include, in what proportion
(SOTA-238), and in what order the documents arrive (SOTA-239). This one is
about the order of clauses within a sequence, which is a fourth thing and
the reason it is filed separately rather than folded into the ordering
practice.
The cost is not symmetric with the benefit. Duplicating every factual
statement in both directions inflates a corpus and interacts with everything
the record says about deduplication and repetition — SOTA-164, SOTA-171.
The instruction is worth acting on where a relation is structured and
generated, and is not an argument for rewriting prose.
Evidence is one model series, up to 52B, pretrained only. No fine-tuning, and MLP parameters only.