The claim
Model stitching carries one network’s representation r into another network at layer ℓ through a learned map s, and judges the result by what the second network can then do. It comes in two forms, and only one of them tests an order of information:
- Reconstruction form. s is fitted so that s ∘ r reproduces the target layers, as in Lenc and Vedaldi (LIT-363). If it succeeds on the input distribution, then given each label the target representation is a function of r. The target is then a garbling of r, and r is at least as informative for every decision problem on the label (THEORY-176). Success certifies one direction of sufficiency.
- Task-loss form. s is fitted to minimise the second network’s task loss, the protocol of Bansal, Nakkiran and Barak (LIT-850). The penalty compares one decision problem through a fixed head and a restricted map class. Both losses are upper bounds on the best attainable risk, so the penalty orders neither, and it is not in general invariant under invertible reparametrisation.
In both forms the test runs one way. A failure to stitch says only that no map in the class works.
Where the record applies it
CLAIM-126 applies the result to the question of when a transport between scenarios preserves decision-relevant information, resting on CLAIM-050’s account of fidelity. It now also rests on this claim for the result about stitching.
What it does not say
- It does not characterise which transports preserve information, or measure how much a non-sufficient one loses. That is Le Cam’s deficiency.
- It does not say task-loss stitching is useless. It is the right test when the decision problem really is fixed in advance.
Note of 2026-10-19: the three steps, written out
Reconstruction form: a composition argument. Let Y be the label, r the source representation and t the target layers, and let s ∘ r = t almost surely on the input distribution. Then P(t | Y) = s#P(r | Y), so t is a deterministic garbling of r about Y. Any decision rule δ that reads t gives the rule δ ∘ s on r, with the same action under every input and so the same risk under every label. So r is at least as informative as t for every decision problem on Y. Nothing here needs a finite outcome space, and none of it needs THEORY-176, which holds the measured size of the effect.
Task-loss form: a difference of upper bounds. The stitched network’s loss is the risk of one rule on r, the fixed head composed with s, so it bounds from above the best risk attainable from r. The target network’s loss bounds the best risk from t in the same way. The penalty is a difference of two upper bounds, so it orders neither minimum.
Not in general invariant: a construction. Use the 1×1 convolutional stitcher of LIT-850, which applies one channel map at each spatial position and mixes no positions. Take inputs whose layer-ℓ activations at position (0, 0) are independent of those at every other position, a label that is a function of the activations at (0, 0), and a top network that reads only position (0, 0). Let φ be a spatial permutation that moves (0, 0) to another position. Then r and φ(r) are each a function of the other, so they are equivalent in Blackwell’s order. Stitched into the top network, r with the identity map recovers the label exactly. From φ(r), every 1×1 stitcher gives position (0, 0) a channel map of the activations from φ⁻¹(0, 0), which are independent of the label, so no map in the class does better than chance. Equivalent representations get different penalties.