Ekdeep Singh Lubana, Eric J. Bigelow, Robert P. Dick, David Krueger and Hidenori Tanaka (2022), Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202 — ARXIV-2211.08422
Key takeaways
- Mechanistic similarity is defined by shared invariances. Model a dataset as generated from independent latents; a model is invariant to an intervention on one latent if setting it to random values does not raise its loss. Two models are mechanistically similar when they are invariant to the same set of such interventions (Definitions 2–4).
- Mechanistically dissimilar minimizers are mode connected, but not linearly. A model trained with a label-correlated box cue and one trained without it are joined by a quadratic Bezier path of low loss on the training data, but that path does not hold on counterfactual data, and the linear path has a barrier even after activation-matching permutation (Fig. 4).
- Conjecture 1: no linear connectivity, up to symmetries, implies mechanistic dissimilarity. It is not proved. Appendix F.3 proves the converse for a one-hidden-layer ReLU network with interpolating minimizers: Lemma 2 (linear connectivity forces shared activation patterns) with Theorem 1 (mechanisms of different complexity force different patterns) gives dissimilar ⇒ no linear connectivity. The paper presents this as verifying the conjecture; the stated direction rests on the fine-tuning experiments.
- Naive fine-tuning keeps the pretrained mechanism. After fine-tuning on cue-free data with small or medium learning rates, the model stays linearly connected to the pretrained one and still relies on the cue; only a large learning rate or a perfectly correlated cue gives both a barrier and a changed mechanism (Fig. 5).
- Connectivity-based fine-tuning (CBFT) adds a loss that pushes the linear path to the pretrained model up to a barrier and a loss that pulls class-mean representations with and without the cue together. On CIFAR-10 with 60% cue data it drops accuracy with a randomized image from 83.4% (medium-LR fine-tuning) to 8.75% while clean accuracy stays at 74.1% (Table 1).
Standing in the record
Filed on 2026-10-03 with the owner’s batch of mode-connectivity readings (ADR-027). It is the batch’s clearest statement that a barrier can mean the two models compute different things, not only that their units are labelled differently.
It builds on two foundations held here. From Garipov et al. (LIT-673) it takes the quadratic Bezier path, trained to low loss on a dataset, as its non-linear connection, and its Appendix C.1 makes the point that such a path is fitted to the data it was trained on and need not hold on counterfactual data. From Frankle et al. (LIT-654) it takes linear mode connectivity as the test, and it follows them in plotting accuracy rather than loss along paths (Appendix C.3). It also takes the activation matching of Git Re-Basin (LIT-661) to permute before interpolating, and its Remark 1 reads Lemma 2 as a precise condition under which the permutation conjecture of Entezari et al. (LIT-652) holds: if two networks’ activation patterns are a permutation of each other, unpermuting them makes them linearly connected.
Across the boundary it complicates the anthology’s ANTH-THEORY-010 (most of the barrier between independently trained networks is permutation, not disagreement). This paper’s barriers are disagreement by construction: the two endpoints rely on different input attributes. That is a different setting from two seeds on one dataset, so it does not refute the theory, but it names the case the theory’s account leaves out. Its fine-tuning results bear on the anthology’s weight-averaging practice, which assumes a fine-tuned model is still in its parent’s basin (WiSE-FT, ANTH-LIT-674; model soups, ANTH-LIT-675). The paper says so in its conclusion, and adds that averaging can only move between mechanisms of similar complexity.
The paper carries an instruction for practice, which is that naive fine-tuning on clean data may not remove a spurious mechanism, so it is flagged for the anthology.