Source
Koh and Liang (2017), LIT-403. This is not an aside in that paper — the counterfactual is how the quantity is defined, and matching it against retraining is the experiment that validates the method.
The account, as it was stated
Removing a training point and refitting gives a new minimizer. That is the
quantity of interest and it is unaffordable, so approximate it: upweight the
point by ε instead of deleting it, differentiate the minimizer with respect
to ε via the implicit function theorem, and evaluate at the removal
direction. What comes out is a gradient alignment warped by the inverse
Hessian, and it is an approximation to retraining.
The paper checks this. On 10-class MNIST, for the 500 most influential training points, the predicted change in test loss tracks the change actually measured by removing the point and refitting.
It is a good explanation, and the validation is the right one to have run. It makes the method mean something — a large influence is a claim about a counterfactual world — and that meaning is why the four use cases sounded like use cases rather than like coincidences.
Where it holds, and where it stopped
The derivation assumes a twice-differentiable, strictly convex empirical risk minimized exactly. MNIST logistic regression satisfies this. A trained neural network satisfies none of it: the objective is non-convex, training stops before any minimum, and it stops warm — at parameters reached from a particular trajectory rather than at the global optimum of anything.
The paper knew the assumptions were violated and argued, reasonably, that a damped quadratic approximation around the obtained parameters would still be informative. The part that was not anticipated is that the residual would not behave like error. LIT-402 decomposes the discrepancy into five terms and finds the three largest are structural differences in what is being asked, not failures to ask it accurately — see THEORY-018.
The reading it invites and does not support
That a large influence score means this example made the model what it is, and without it the model would be different. On a neural network that reading is unsupported, and it is the one every intuitive presentation of the method produces, including this one’s.
There is a second reading that is safe and is easy to lose along with the first: the ranking is still informative. The practice built on this account — finding mislabelled data by self-influence, SOTA-246 — survives, because what it needs is an ordering over training points and not a counterfactual about any of them. Rejecting an explanation is not rejecting what it explained, which is the whole reason this record keeps the two apart.