anthology-of-the-sota

LIT-403: Understanding Black-box Predictions via Influence Functions

2 min read

Properties5
StatusActive
Tagsanalysis-and-evaluation, data-pipeline
Cited as “introduced by” bySOTA-246 — Find mislabelled training data by self-influence, not by training loss
Cited as “paper” byNOTE-181 — Understanding Black-box Predictions via Influence Functions
Cited as “source” bySOTA-246 — Find mislabelled training data by self-influence, not by training loss, THEORY-017 — An influence-function estimate predicts the effect of removing a training point and retraining, THEORY-018 — Influence functions on neural networks estimate the proximal Bregman response function, not leave-one-out retraining

Graph View

Koh and Liang (2017) — ARXIV-1703.04730

Key takeaways

  • Asks a counterfactual and answers it without running it. How would this prediction change if this training point were not in the training set? Answering by retraining is prohibitive; the influence function answers it analytically by upweighting the point infinitesimally and differentiating
  • Made practical by never forming the Hessian. Both routes — conjugate gradients, and the stochastic estimator of Agarwal et al. — need only Hessian-vector products, which cost about a gradient each. The stochastic version estimated H⁻¹v on MNIST without looking at every training point, and even a single repeat still identified the most influential examples
  • The theory assumes a twice-differentiable, strictly convex risk and the paper then breaks both assumptions on purpose. Under non-convergence and non-convexity a damped quadratic approximation around the actual parameters still gives meaningful results. Under non-differentiability it does not: setting the derivative at an SVM hinge to zero gave inaccurate influences, and a smooth surrogate loss was needed
  • Four uses, and they are still the four. Understanding what a model relies on; quantifying vulnerability to training-set attacks; debugging domain mismatch; and finding mislabelled examples
  • Two models, same predictions, different mechanism. An Inception v3 with frozen lower layers and an RBF-SVM agree on a dog-versus-fish task. The SVM’s influences vary inversely with raw pixel distance — a soft nearest neighbour — while Inception’s barely correlate with it, and its fifth most helpful image for classifying a fish was a dog
  • The attack is the same computation as the explanation. Constructing visually indistinguishable training-set perturbations is mathematically the same gradient step, which is why the method also measures how exposed a model is to data poisoning
  • Dataset repair, quantified. On Enron1 spam (4,147 train, logistic regression on bag-of-words) with 10% of labels flipped, prioritising inspection by influence repaired the dataset after checking fewer points than either the highest-training-loss baseline or random, over 40 repeats, with no access to test data

Standing in the anthology

Read — NOTE-181. Sources SOTA-246, and states THEORY-017 — the account of what an influence estimate measures that LIT-402 later corrected for neural networks.

Backlinks