Source
Wang, Hu, Du, Cheng, Wang and Zou (2025), ICML 2025, arXiv:2502.11812, LIT-1539, read in full in NOTE-1183: §4 and Appendices D, E, F, G, H and K. The conclusion’s words: “fine-tuning primarily modifying edges rather than merely introducing new components to form new circuits”.
What is asserted, and by whom
The authors assert that fine-tuning on a task the model fails “creates new circuits through significant reconfiguration of connections among components”, against earlier findings that fine-tuning on tasks already done well adds components to an existing circuit.
The measurement (Table 5). Average change between consecutive checkpoints, relative to the initial circuit’s size:
| task | nodes | edges | ratio |
|---|---|---|---|
| Add/Sub (100) | 17.4% | 69.5% | 4.0 |
| Mul/Div | 23.2% | 76.9% | 3.3 |
| Sequence | 24.2% | 67.9% | 2.8 |
| LCM | 23.9% | 80.4% | 3.4 |
| Function | 15.4% | 65.8% | 4.3 |
The contrast holds under AdaLoRA, IA3 and full fine-tuning, and for GPT-Neo-2.7B and OPT-6.7B on Add/Sub.
What the record holds toward it
Why the counts alone do not settle it (the record’s). A circuit here is the top 5% of edges of a dense layered graph. The record built a graph the size of Pythia-1.4B’s (24 layers of 16 heads and an MLP, plus input and logits: 410 nodes, 80,581 forward edges), gave each edge a score equal to a fixed node importance at each end times independent log-normal noise, and compared the top-5% circuits from two independent draws of the noise only. The node sets differed by 5.1% and the edge sets by 129.7% (counted as the symmetric difference over the circuit’s size). With uniform random scores the node sets did not differ at all. So, with node importances held fixed, resampling edge-level noise yields “nodes kept, edges changed” by itself. The paper’s robustness analysis (edge Jaccard under data perturbation, higher for trained than for random models) bounds the noise in its circuits, but it is not a comparison at the same budget, and the paper reports none.
What does support it. The circuits are faithful at every checkpoint, the change concentrates in the middle layers rather than spreading uniformly, and the edges that change most predict where extra LoRA rank helps (CircuitLoRA beats random critical layers in every column of Tables 1 and 4). That last is evidence that the changed edges are not noise.
Two further cautions from the method’s lineage. EAP-IG descends from attribution patching (AtP*, LIT-1531) and from ACDC’s edge-level circuits (LIT-1547), both read on 2026-10-26. Attribution’s misses concentrate in queries and keys behind a saturated softmax and in nodes whose direct and indirect effects cancel (THEORY-996), so an edge that “appears” between checkpoints may be one that stopped being missed. And a greedy circuit is one of several sufficient subgraphs, chosen by the order and threshold of the search (THEORY-966, THEORY-970), so an edge set that changes between checkpoints may be a change in which sufficient subgraph was returned. Both would need holding fixed, or measuring, before a change in edges is read as a change in the model.
What would confirm or reject it
The promote_when above.
Bearing
- QUESTION-275 cites this work for relations reorganizing while components stay. As filed, it is the hypothesis, not the finding, and the null above is the test that question’s own design would also need.
- CLAIM-329: “the same components” is judged here by resemblance (the node appears again), not by what the component does or where its function came from.