about
Understanding Black-Box Machine Learning Predictions via Influence Functions (arxiv.org)
1 point by capocannoniere on Mar 20, 2017 | hide | past | pdf | discuss on HN

In plain words: To explain a prediction, this traces it back through training to the examples that most pushed the model toward that answer, using simple gradient math. Even on complex models where the math shouldn't hold, it still pinpointed the culprits, spotting bad training data.

Abstract · Understanding Black-box Predictions via Influence Functions

How can we explain the predictions of a black-box model? In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks.

Pang Wei Koh, Percy Liang
arXiv:1703.04730 · stat.ML, cs.AI, cs.LG · submitted Mar 14, 2017 · updated Dec 29, 2020
abstract · pdf · html · International Conference on Machine Learning, 2017. (This version adds more historical references and fixes typos.)

add comment on HN