Understanding Black-box Predictions via Influence Functions

Understanding Black-box Predictions via Influence Functions：用影响函数理解黑盒预测

Pang Wei Koh 1 Percy Liang 1

1 Stanford University, Stanford, CA. Correspondence to: Pang Wei Koh pangwei@cs.stanford.edu, Percy Liang pliang@cs.stanford.edu.

Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, PMLR 70, 2017. Copyright 2017 by the author(s).

Abstract

摘要

How can we explain the predictions of a black-box model? In this paper, we use influence functions — a classic technique from robust statistics — to trace a model’s prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks.

怎样解释黑盒模型的预测？本文将稳健统计里的经典工具——影响函数——用来沿学习算法把一次预测追回到训练数据，从而找出对这次预测最该负责的训练点。为了把影响函数做到现代机器学习的规模，我们给出一种只需梯度与 Hessian-向量积预言机的简单、高效实现。我们还表明，即便在理论假定不再成立的非凸、不可微模型上，影响函数的近似仍然能提供有用信息。在线性模型和卷积神经网络上，影响函数可以同时服务于多种用途：理解模型行为、调试模型、发现数据错误，甚至构造肉眼几乎分不出的训练集攻击。

1. Introduction

1. 引言

A key question often asked of machine learning systems is “Why did the system make this prediction?” We want models that are not just high-performing but also explainable. By understanding why a model does what it does, we can hope to improve the model (Amershi et al., 2015), discover new science (Shrikumar et al., 2016), and provide end-users with explanations of actions that impact them (Goodman & Flaxman, 2016).

机器学习系统常被问到的一个关键问题是：“系统为什么会做出这个预测？”我们希望模型不仅性能好，也要能解释。弄清模型为何如此行事，才有望改进模型（Amershi et al., 2015）、发现新的科学（Shrikumar et al., 2016），并向受其决策影响的终端用户给出说明（Goodman & Flaxman, 2016）。

However, the best-performing models in many domains — e.g., deep neural networks for image and speech recognition (Krizhevsky et al., 2012) — are complicated, black-box models whose predictions seem hard to explain. Work on interpreting these black-box models has focused on understanding how a fixed model leads to particular predictions, e.g., by locally fitting a simpler model around the test point (Ribeiro et al., 2016) or by perturbing the test point to see how the prediction changes (Simonyan et al., 2013; Li et al., 2016b; Datta et al., 2016; Adler et al., 2016). These works explain the predictions in terms of the model, but how can we explain where the model came from?

不过，许多领域里表现最好的模型——例如用于图像和语音识别的深度神经网络（Krizhevsky et al., 2012）——都是复杂的黑盒，预测看起来很难解释。已有工作解释这类黑盒，重点是理解一个固定模型如何导向某次预测，例如在测试点附近局部拟合一个更简单的模型（Ribeiro et al., 2016），或扰动测试点看预测如何变化（Simonyan et al., 2013; Li et al., 2016b; Datta et al., 2016; Adler et al., 2016）。这些工作是就模型来解释预测，但我们如何解释模型本身从何而来？

In this paper, we tackle this question by tracing a model’s predictions through its learning algorithm and back to the training data, where the model parameters ultimately derive from. To formalize the impact of a training point on a prediction, we ask the counterfactual: what would happen if we did not have this training point, or if the values of this training point were changed slightly?

本文沿学习算法把模型的预测追回到训练数据——参数最终都来自那里。为了把训练点对一次预测的影响形式化，我们问一个反事实问题：如果没有这个训练点，或把这个训练点的取值稍作改动，会发生什么？

Answering this question by perturbing the data and retraining the model can be prohibitively expensive. To overcome this problem, we use influence functions, a classic technique from robust statistics (Cook & Weisberg, 1980) that tells us how the model parameters change as we upweight a training point by an infinitesimal amount. This allows us to “differentiate through the training” to estimate in closed-form the effect of a variety of training perturbations.

若靠扰动数据再重训模型来回答这个问题，代价往往高得无法接受。为此，我们使用影响函数：这是稳健统计里的经典方法（Cook & Weisberg, 1980），它告诉我们，把某个训练点的权重上调一个无穷小量时，模型参数会怎样变化。这样就可以“对训练过程求导”，用闭式估计各种训练扰动的效果。

Despite their rich history in statistics, influence functions have not seen widespread use in machine learning; to the best of our knowledge, the work closest to ours is Wojnowicz et al. (2016), which introduced a method for approximating a quantity related to influence in generalized linear models. One obstacle to adoption is that influence functions require expensive second derivative calculations and assume model differentiability and convexity, which limits their applicability in modern contexts where models are often non-differentiable, non-convex, and high-dimensional. We address these challenges by showing that we can efficiently approximate influence functions using second-order optimization techniques (Pearlmutter, 1994; Martens, 2010; Agarwal et al., 2016), and that they remain accurate even as the underlying assumptions of differentiability and convexity degrade.

尽管影响函数在统计学里历史很长，在机器学习中却还没有被广泛使用。据我们所知，与本文最接近的是 Wojnowicz et al. (2016)，他们提出了一种在广义线性模型中近似与影响相关的量的方法。阻碍采用的一个原因是：影响函数需要昂贵的二阶导数计算，并假定模型可微且凸；而现代模型常常不可微、非凸、维数又高，适用范围因此受限。我们说明，可以用二阶优化技术高效地近似影响函数（Pearlmutter, 1994; Martens, 2010; Agarwal et al., 2016），而且即使可微性和凸性这些底层假定逐渐放宽，近似仍然准确。

Influence functions capture the core idea of studying models through the lens of their training data. We show that they are a versatile tool that can be applied to a wide variety of seemingly disparate tasks: understanding model behavior, debugging models, detecting dataset errors, and creating visually-indistinguishable adversarial training examples that can flip neural network test predictions, the training set analogue of Goodfellow et al. (2015).

影响函数抓住的核心想法是：透过训练数据来研究模型。我们表明，它是一种用途很广的工具，可以用于许多看起来互不相干的任务：理解模型行为、调试模型、发现数据错误，以及构造肉眼几乎分不出、却能翻转神经网络测试预测的对抗训练样本——这是 Goodfellow et al. (2015) 在训练集上的对应物。

2. Approach

2. 方法

Consider a prediction problem from some input space $\mathcal{X}$ (e.g., images) to an output space $\mathcal{Y}$ (e.g., labels). We are given training points $z_1, \ldots, z_n$, where $z_i = (x_i, y_i) \in \mathcal{X} \times \mathcal{Y}$. For a point $z$ and parameters $\theta \in \Theta$, let $L(z, \theta)$ be the loss, and let $\frac{1}{n}\sum_{i=1}^{n} L(z_i, \theta)$ be the empirical risk. The empirical risk minimizer is given by $\hat{\theta} \stackrel{\mathrm{def}}{=} \arg\min_{\theta \in \Theta} \frac{1}{n}\sum_{i=1}^{n} L(z_i, \theta)$. Assume that the empirical risk is twice-differentiable and strictly convex in $\theta$; in Section 4 we explore relaxing these assumptions.

考虑从输入空间 $\mathcal{X}$（例如图像）到输出空间 $\mathcal{Y}$（例如标签）的预测问题。给定训练点 $z_1, \ldots, z_n$，其中 $z_i = (x_i, y_i) \in \mathcal{X} \times \mathcal{Y}$。对点 $z$ 和参数 $\theta \in \Theta$，记 $L(z, \theta)$ 为损失，记 $\frac{1}{n}\sum_{i=1}^{n} L(z_i, \theta)$ 为经验风险。经验风险最小点为 $\hat{\theta} \stackrel{\mathrm{def}}{=} \arg\min_{\theta \in \Theta} \frac{1}{n}\sum_{i=1}^{n} L(z_i, \theta)$。假定经验风险对 $\theta$ 二阶可微且严格凸；第 4 节再讨论如何放宽这些假定。

1 We fold in any regularization terms into $L$.

1 我们把正则项一并写进 $L$。

2.1. Upweighting a training point

2.1. 上调一个训练点的权重

Our goal is to understand the effect of training points on a model’s predictions. We formalize this goal by asking the counterfactual: how would the model’s predictions change if we did not have this training point?

我们的目标是理解训练点对模型预测的影响。把这一目标形式化，就是问一个反事实问题：如果没有这个训练点，模型的预测会怎样变化？

Let us begin by studying the change in model parameters due to removing a point $z$ from the training set. Formally, this change is $\hat{\theta}_{-z} - \hat{\theta}$, where $\hat{\theta}_{-z} \stackrel{\mathrm{def}}{=} \arg\min_{\theta \in \Theta} \sum_{z_i \neq z} L(z_i, \theta)$. However, retraining the model for each removed $z$ is prohibitively slow.

先看从训练集中去掉一个点 $z$ 时，模型参数会怎样变。形式上，这一变化是 $\hat{\theta}_{-z} - \hat{\theta}$，其中 $\hat{\theta}_{-z} \stackrel{\mathrm{def}}{=} \arg\min_{\theta \in \Theta} \sum_{z_i \neq z} L(z_i, \theta)$。可是，每去掉一个 $z$ 都重训一遍，慢得难以接受。

Fortunately, influence functions give us an efficient approximation. The idea is to compute the parameter change if $z$ were upweighted by some small $\epsilon$, giving us new parameters $\hat{\theta}_{\epsilon,z} \stackrel{\mathrm{def}}{=} \arg\min_{\theta \in \Theta} \frac{1}{n}\sum_{i=1}^{n} L(z_i, \theta) + \epsilon L(z, \theta)$. A classic result (Cook & Weisberg, 1982) tells us that the influence of upweighting $z$ on the parameters $\hat{\theta}$ is given by

好在影响函数能给出高效的近似。想法是：如果把 $z$ 的权重上调一个很小的 $\epsilon$，参数会变成 $\hat{\theta}_{\epsilon,z} \stackrel{\mathrm{def}}{=} \arg\min_{\theta \in \Theta} \frac{1}{n}\sum_{i=1}^{n} L(z_i, \theta) + \epsilon L(z, \theta)$。经典结果（Cook & Weisberg, 1982）告诉我们，上调 $z$ 的权重对参数 $\hat{\theta}$ 的影响是

$$I_{\mathrm{up,params}}(z) \stackrel{\mathrm{def}}{=} \left.\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}\right|_{\epsilon=0} = -H_{\hat{\theta}}^{-1}\nabla_{\theta}L(z,\hat{\theta}), \tag{1}$$

where $H_{\hat{\theta}} \stackrel{\mathrm{def}}{=} \frac{1}{n}\sum_{i=1}^{n} \nabla_{\theta}^{2} L(z_i, \hat{\theta})$ is the Hessian and is positive definite (PD) by assumption. In essence, we form a quadratic approximation to the empirical risk around $\hat{\theta}$ and take a single Newton step; see appendix A for a derivation. Since removing a point $z$ is the same as upweighting it by $\epsilon = -\frac{1}{n}$, we can linearly approximate the parameter change due to removing $z$ by computing $\hat{\theta}_{-z} - \hat{\theta} \approx -\frac{1}{n} I_{\mathrm{up,params}}(z)$, without retraining the model.

其中 $H_{\hat{\theta}} \stackrel{\mathrm{def}}{=} \frac{1}{n}\sum_{i=1}^{n} \nabla_{\theta}^{2} L(z_i, \hat{\theta})$ 是 Hessian，并按假定为正定（PD）。本质上，我们是在 $\hat{\theta}$ 附近对经验风险做二次近似，再走一步牛顿法；推导见附录 A。去掉一个点 $z$ 等价于把它的权重上调 $\epsilon = -\frac{1}{n}$，因此去掉 $z$ 引起的参数变化可以线性近似为 $\hat{\theta}_{-z} - \hat{\theta} \approx -\frac{1}{n} I_{\mathrm{up,params}}(z)$，不必重训模型。

Next, we apply the chain rule to measure how upweighting $z$ changes functions of $\hat{\theta}$. In particular, the influence of upweighting $z$ on the loss at a test point $z_{\mathrm{test}}$ again has a closed-form expression:

接下来用链式法则，衡量上调 $z$ 的权重会怎样改变 $\hat{\theta}$ 的函数。特别地，上调 $z$ 对测试点 $z_{\mathrm{test}}$ 上损失的影响同样有闭式：

$$I_{\mathrm{up,loss}}(z,z_{\mathrm{test}}) \stackrel{\mathrm{def}}{=} \left.\frac{dL(z_{\mathrm{test}},\hat{\theta}_{\epsilon,z})}{d\epsilon}\right|_{\epsilon=0} = \nabla_{\theta}L(z_{\mathrm{test}},\hat{\theta})^{\top}\left.\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}\right|_{\epsilon=0} = -\nabla_{\theta}L(z_{\mathrm{test}},\hat{\theta})^{\top}H_{\hat{\theta}}^{-1}\nabla_{\theta}L(z,\hat{\theta}). \tag{2}$$

2.2. Perturbing a training input

2.2. 扰动一个训练输入

Let us develop a finer-grained notion of influence by studying a different counterfactual: how would the model’s predictions change if a training input were modified?

再看另一种更细的影响：如果改动一个训练输入，模型的预测会怎样变化？

For a training point $z = (x, y)$, define $z_{\delta} = (x + \delta, y)$. Consider the perturbation $z \mapsto z_{\delta}$, and let $\hat{\theta}_{z_{\delta},-z}$ be the empirical risk minimizer on the training points with $z_{\delta}$ in place of $z$. To approximate its effects, define the parameters resulting from moving $\epsilon$ mass from $z$ onto $z_{\delta}$: $\hat{\theta}_{\epsilon,z_{\delta},-z} \stackrel{\mathrm{def}}{=} \arg\min_{\theta \in \Theta} \frac{1}{n}\sum_{i=1}^{n} L(z_i, \theta) + \epsilon L(z_{\delta}, \theta) - \epsilon L(z, \theta)$. An analogous calculation to (1) yields:

对训练点 $z = (x, y)$，定义 $z_{\delta} = (x + \delta, y)$。考虑扰动 $z \mapsto z_{\delta}$，并记 $\hat{\theta}_{z_{\delta},-z}$ 为用 $z_{\delta}$ 替换 $z$ 后的经验风险最小点。为近似其效果，定义把 $\epsilon$ 的质量从 $z$ 移到 $z_{\delta}$ 后得到的参数：$\hat{\theta}_{\epsilon,z_{\delta},-z} \stackrel{\mathrm{def}}{=} \arg\min_{\theta \in \Theta} \frac{1}{n}\sum_{i=1}^{n} L(z_i, \theta) + \epsilon L(z_{\delta}, \theta) - \epsilon L(z, \theta)$。与 (1) 类似的计算给出：

$$\left.\frac{d\hat{\theta}_{\epsilon,z_{\delta},-z}}{d\epsilon}\right|_{\epsilon=0} = I_{\mathrm{up,params}}(z_{\delta}) - I_{\mathrm{up,params}}(z) = -H_{\hat{\theta}}^{-1}\bigl(\nabla_{\theta}L(z_{\delta},\hat{\theta}) - \nabla_{\theta}L(z,\hat{\theta})\bigr). \tag{3}$$

As before, we can make the linear approximation $\hat{\theta}_{z_{\delta},-z} - \hat{\theta} \approx -\frac{1}{n}\bigl(I_{\mathrm{up,params}}(z_{\delta}) - I_{\mathrm{up,params}}(z)\bigr)$, giving us a closed-form estimate of the effect of $z \mapsto z_{\delta}$ on the model. Analogous equations also apply for changes in $y$. While influence functions might appear to only work for infinitesimal (therefore continuous) perturbations, it is important to note that this approximation holds for arbitrary $\delta$: the $\epsilon$-upweighting scheme allows us to smoothly interpolate between $z$ and $z_{\delta}$. This is particularly useful for working with discrete data (e.g., in NLP) or with discrete label changes.

和前面一样，可以做线性近似 $\hat{\theta}_{z_{\delta},-z} - \hat{\theta} \approx -\frac{1}{n}\bigl(I_{\mathrm{up,params}}(z_{\delta}) - I_{\mathrm{up,params}}(z)\bigr)$，从而得到 $z \mapsto z_{\delta}$ 对模型影响的闭式估计。对 $y$ 的改动也有类似公式。影响函数看起来似乎只适用于无穷小（因而连续）的扰动，但需要强调的是，这一近似对任意 $\delta$ 都成立：$\epsilon$-上调权重的做法让我们可以在 $z$ 与 $z_{\delta}$ 之间平滑插值。这对处理离散数据（例如自然语言）或离散的标签改动尤其有用。

If $x$ is continuous and $\delta$ is small, we can further approximate (3). Assume that the input domain $\mathcal{X} \subseteq \mathbb{R}^{d}$, the parameter space $\Theta \subseteq \mathbb{R}^{p}$, and $L$ is differentiable in $\theta$ and $x$. As $\|\delta\| \to 0$, $\nabla_{\theta} L(z_{\delta}, \hat{\theta}) - \nabla_{\theta} L(z, \hat{\theta}) \approx \bigl[\nabla_{x}\nabla_{\theta} L(z, \hat{\theta})\bigr]\delta$, where $\nabla_{x}\nabla_{\theta} L(z, \hat{\theta}) \in \mathbb{R}^{p \times d}$. Substituting into (3),

若 $x$ 连续且 $\delta$ 很小，还可以进一步近似 (3)。假定输入域 $\mathcal{X} \subseteq \mathbb{R}^{d}$，参数空间 $\Theta \subseteq \mathbb{R}^{p}$，且 $L$ 对 $\theta$ 和 $x$ 可微。当 $\|\delta\| \to 0$ 时，$\nabla_{\theta} L(z_{\delta}, \hat{\theta}) - \nabla_{\theta} L(z, \hat{\theta}) \approx \bigl[\nabla_{x}\nabla_{\theta} L(z, \hat{\theta})\bigr]\delta$，其中 $\nabla_{x}\nabla_{\theta} L(z, \hat{\theta}) \in \mathbb{R}^{p \times d}$。代入 (3)，

$$\left.\frac{d\hat{\theta}_{\epsilon,z_{\delta},-z}}{d\epsilon}\right|_{\epsilon=0} \approx -H_{\hat{\theta}}^{-1}\bigl[\nabla_{x}\nabla_{\theta}L(z,\hat{\theta})\bigr]\delta. \tag{4}$$

We thus have $\hat{\theta}_{z_{\delta},-z} - \hat{\theta} \approx -\frac{1}{n} H_{\hat{\theta}}^{-1}\bigl[\nabla_{x}\nabla_{\theta} L(z, \hat{\theta})\bigr]\delta$. Differentiating w.r.t. $\delta$ and applying the chain rule gives us

于是 $\hat{\theta}_{z_{\delta},-z} - \hat{\theta} \approx -\frac{1}{n} H_{\hat{\theta}}^{-1}\bigl[\nabla_{x}\nabla_{\theta} L(z, \hat{\theta})\bigr]\delta$。再对 $\delta$ 求导并使用链式法则，得到

$$I_{\mathrm{pert,loss}}(z,z_{\mathrm{test}})^{\top} \stackrel{\mathrm{def}}{=} \left.\nabla_{\delta}L(z_{\mathrm{test}},\hat{\theta}_{z_{\delta},-z})^{\top}\right|_{\delta=0} = -\nabla_{\theta}L(z_{\mathrm{test}},\hat{\theta})^{\top}H_{\hat{\theta}}^{-1}\nabla_{x}\nabla_{\theta}L(z,\hat{\theta}). \tag{5}$$

$I_{\mathrm{pert,loss}}(z, z_{\mathrm{test}})^{\top}\delta$ tells us the approximate effect that $z \mapsto z + \delta$ has on the loss at $z_{\mathrm{test}}$. By setting $\delta$ in the direction of $I_{\mathrm{pert,loss}}(z, z_{\mathrm{test}})$, we can construct local perturbations of $z$ that maximally increase the loss at $z_{\mathrm{test}}$. In Section 5.2, we will use this to construct training-set attacks. Finally, we note that $I_{\mathrm{pert,loss}}(z, z_{\mathrm{test}})$ can help us identify the features of $z$ that are most responsible for the prediction on $z_{\mathrm{test}}$.

$I_{\mathrm{pert,loss}}(z, z_{\mathrm{test}})^{\top}\delta$ 告诉我们，$z \mapsto z + \delta$ 对 $z_{\mathrm{test}}$ 上损失的近似影响。把 $\delta$ 取在 $I_{\mathrm{pert,loss}}(z, z_{\mathrm{test}})$ 的方向上，就可以构造使 $z_{\mathrm{test}}$ 上损失增加最多的局部扰动。第 5.2 节会用它来构造训练集攻击。另外，$I_{\mathrm{pert,loss}}(z, z_{\mathrm{test}})$ 还可以帮助找出 $z$ 中对 $z_{\mathrm{test}}$ 上预测最该负责的特征。

2.3. Relation to Euclidean distance

2.3. 与欧氏距离的关系

To find the training points most relevant to a test point, it is common to look at its nearest neighbors in Euclidean space (e.g., Ribeiro et al. (2016)); if all points have the same norm, this is equivalent to choosing $x$ with the largest $x \cdot x_{\mathrm{test}}$. For intuition, we compare this to $I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ on a logistic regression model and show that influence is much more accurate at accounting for the effect of training.

要找与某个测试点最相关的训练点，常见做法是在欧氏空间里看它的近邻（例如 Ribeiro et al. (2016)）；若所有点范数相同，这等价于选取使 $x \cdot x_{\mathrm{test}}$ 最大的 $x$。为建立直觉，我们在逻辑回归上把它与 $I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ 比较，并表明影响函数能准确得多地计入训练过程的作用。

Let $p(y \mid x) = \sigma(y\theta^{\top}x)$, with $y \in \{-1, 1\}$ and $\sigma(t) = \frac{1}{1+\exp(-t)}$. We seek to maximize the probability of the training set. For a training point $z = (x, y)$, $L(z, \theta) = \log\bigl(1 + \exp(-y\theta^{\top}x)\bigr)$, $\nabla_{\theta} L(z, \theta) = -\sigma(-y\theta^{\top}x)\, yx$, and $H_{\theta} = \frac{1}{n}\sum_{i=1}^{n} \sigma(\theta^{\top}x_i)\sigma(-\theta^{\top}x_i)\, x_i x_i^{\top}$. From (2), $I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ is:

设 $p(y \mid x) = \sigma(y\theta^{\top}x)$，其中 $y \in \{-1, 1\}$，$\sigma(t) = \frac{1}{1+\exp(-t)}$。目标是最大化训练集的概率。对训练点 $z = (x, y)$，$L(z, \theta) = \log\bigl(1 + \exp(-y\theta^{\top}x)\bigr)$，$\nabla_{\theta} L(z, \theta) = -\sigma(-y\theta^{\top}x)\, yx$，且 $H_{\theta} = \frac{1}{n}\sum_{i=1}^{n} \sigma(\theta^{\top}x_i)\sigma(-\theta^{\top}x_i)\, x_i x_i^{\top}$。由 (2)，$I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ 为：

$$-y_{\mathrm{test}} y \cdot \sigma(-y_{\mathrm{test}}\theta^{\top}x_{\mathrm{test}}) \cdot \sigma(-y\theta^{\top}x) \cdot x_{\mathrm{test}}^{\top} H_{\hat{\theta}}^{-1} x.$$

We highlight two key differences from $x \cdot x_{\mathrm{test}}$. First, $\sigma(-y\theta^{\top}x)$ gives points with high training loss more influence, revealing that outliers can dominate the model parameters. Second, the weighted covariance matrix $H_{\hat{\theta}}^{-1}$ measures the “resistance” of the other training points to the removal of $z$; if $\nabla_{\theta} L(z, \hat{\theta})$ points in a direction of little variation, its influence will be higher since moving in that direction will not significantly increase the loss on other training points. As we show in Fig 1, these differences mean that influence functions capture the effect of model training much more accurately than nearest neighbors.

与 $x \cdot x_{\mathrm{test}}$ 相比，有两点关键差别。第一，$\sigma(-y\theta^{\top}x)$ 让训练损失高的点获得更大影响，说明离群点可以支配模型参数。第二，加权协方差矩阵 $H_{\hat{\theta}}^{-1}$ 衡量的是其他训练点对去掉 $z$ 的“阻力”：若 $\nabla_{\theta} L(z, \hat{\theta})$ 指向变化很小的方向，影响就会更大，因为沿这个方向移动并不会显著增加其他训练点上的损失。如图 1 所示，这些差别意味着影响函数比近邻法更准确地抓住了模型训练的效果。

Figure 1. Components of influence. (a) What is the effect of the training loss and $H_{\hat{\theta}}^{-1}$ terms in $I_{\mathrm{up,loss}}$? Here, we plot $I_{\mathrm{up,loss}}$ against variants that are missing these terms and show that they are necessary for picking up the truly influential training points. For these calculations, we use logistic regression to distinguish 1’s from 7’s in MNIST (LeCun et al., 1998), picking an arbitrary test point $z_{\mathrm{test}}$; similar trends hold across other test points. Green dots are train images of the same label as the test image (7) while red dots are 1’s. Left: Without the train loss term, we overestimate the influence of many training points: the points near the $y=0$ line should have $I_{\mathrm{up,loss}}$ close to 0, but instead have high influence when we remove the train loss term. Mid: Without $H_{\hat{\theta}}^{-1}$, all green training points are helpful (removing each point increases test loss) and all red points are harmful (removing each point decreases test loss). This is because $\forall x,\, x \succeq 0$ (all pixel values are positive), so $x \cdot x_{\mathrm{test}} \ge 0$, but it is incorrect: many harmful training points actually share the same label as $z_{\mathrm{test}}$. See panel (b). Right: Without training loss or $H_{\hat{\theta}}^{-1}$, what is left is the scaled Euclidean inner product $y_{\mathrm{test}} y \cdot \sigma(-y_{\mathrm{test}}\theta^{\top}x_{\mathrm{test}}) \cdot x_{\mathrm{test}}^{\top}x$, which fails to accurately capture influence; the scatter plot deviates quite far from the diagonal. (b) The test image and a harmful training image with the same label. To the model, they look very different, so the presence of the training image makes the model think that the test image is less likely to be a 7. The Euclidean inner product does not pick up on these less intuitive, but important, harmful influences.

图 1. 影响的各个组成部分。(a) $I_{\mathrm{up,loss}}$ 里的训练损失项和 $H_{\hat{\theta}}^{-1}$ 项起什么作用？这里把 $I_{\mathrm{up,loss}}$ 与去掉这些项后的变体对照，说明它们对找出真正有影响的训练点是必要的。计算时，用逻辑回归在 MNIST（LeCun et al., 1998）上区分 1 和 7，并任取一个测试点 $z_{\mathrm{test}}$；其他测试点上也有类似趋势。绿点是与测试图像标签相同（7）的训练图像，红点是 1。左：去掉训练损失项后，许多训练点的影响会被高估：靠近 $y=0$ 线的点，其 $I_{\mathrm{up,loss}}$ 本应接近 0，但去掉训练损失项后影响却很高。中：去掉 $H_{\hat{\theta}}^{-1}$ 后，所有绿色训练点都变成有益的（去掉该点会增加测试损失），所有红点都变成有害的（去掉该点会降低测试损失）。这是因为 $\forall x,\, x \succeq 0$（所有像素值都为正），于是 $x \cdot x_{\mathrm{test}} \ge 0$，但这一判断并不对：许多有害的训练点其实与 $z_{\mathrm{test}}$ 标签相同。见面板 (b)。右：既去掉训练损失又去掉 $H_{\hat{\theta}}^{-1}$ 后，剩下的就是缩放后的欧氏内积 $y_{\mathrm{test}} y \cdot \sigma(-y_{\mathrm{test}}\theta^{\top}x_{\mathrm{test}}) \cdot x_{\mathrm{test}}^{\top}x$，它无法准确刻画影响，散点会明显偏离对角线。(b) 测试图像，以及一张标签相同却有害的训练图像。在模型看来，它们差别很大，因此这张训练图像的存在会让模型觉得测试图像更不像是 7。欧氏内积捕捉不到这类不那么直观、却很重要的有害影响。

3. Efficiently Calculating Influence

3. 高效计算影响

There are two computational challenges to using $I_{\mathrm{up,loss}}(z, z_{\mathrm{test}}) = -\nabla_{\theta} L(z_{\mathrm{test}}, \hat{\theta})^{\top} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta})$. First, it requires forming and inverting $H_{\hat{\theta}} = \frac{1}{n}\sum_{i=1}^{n} \nabla_{\theta}^{2} L(z_i, \hat{\theta})$, the Hessian of the empirical risk. With $n$ training points and $\theta \in \mathbb{R}^{p}$, this requires $O(np^{2} + p^{3})$ operations, which is too expensive for models like deep neural networks with millions of parameters. Second, we often want to calculate $I_{\mathrm{up,loss}}(z_i, z_{\mathrm{test}})$ across all training points $z_i$.

使用 $I_{\mathrm{up,loss}}(z, z_{\mathrm{test}}) = -\nabla_{\theta} L(z_{\mathrm{test}}, \hat{\theta})^{\top} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta})$ 有两个计算上的难点。第一，需要形成并求逆经验风险的 Hessian $H_{\hat{\theta}} = \frac{1}{n}\sum_{i=1}^{n} \nabla_{\theta}^{2} L(z_i, \hat{\theta})$。有 $n$ 个训练点且 $\theta \in \mathbb{R}^{p}$ 时，这需要 $O(np^{2} + p^{3})$ 次运算，对参数以百万计的深度网络来说太贵。第二，我们常常要对所有训练点 $z_i$ 计算 $I_{\mathrm{up,loss}}(z_i, z_{\mathrm{test}})$。

The first problem is well-studied in second-order optimization. The idea is to avoid explicitly computing $H_{\hat{\theta}}^{-1}$; instead, we use implicit Hessian-vector products (HVPs) to efficiently approximate $s_{\mathrm{test}} \stackrel{\mathrm{def}}{=} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z_{\mathrm{test}}, \hat{\theta})$ and then compute $I_{\mathrm{up,loss}}(z, z_{\mathrm{test}}) = -s_{\mathrm{test}} \cdot \nabla_{\theta} L(z, \hat{\theta})$. This also solves the second problem: for each test point of interest, we can precompute $s_{\mathrm{test}}$ and then efficiently compute $-s_{\mathrm{test}} \cdot \nabla_{\theta} L(z_i, \hat{\theta})$ for each training point $z_i$.

第一个问题在二阶优化里已被充分研究。做法是避免显式计算 $H_{\hat{\theta}}^{-1}$；改为用隐式的 Hessian-向量积（HVP）高效近似 $s_{\mathrm{test}} \stackrel{\mathrm{def}}{=} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z_{\mathrm{test}}, \hat{\theta})$，再计算 $I_{\mathrm{up,loss}}(z, z_{\mathrm{test}}) = -s_{\mathrm{test}} \cdot \nabla_{\theta} L(z, \hat{\theta})$。这也解决了第二个问题：对每个感兴趣的测试点，可以预先算出 $s_{\mathrm{test}}$，再对每个训练点 $z_i$ 高效计算 $-s_{\mathrm{test}} \cdot \nabla_{\theta} L(z_i, \hat{\theta})$。

We discuss two techniques for approximating $s_{\mathrm{test}}$, both relying on the fact that the HVP of a single term in $H_{\hat{\theta}}$, $\bigl[\nabla_{\theta}^{2} L(z_i, \hat{\theta})\bigr]v$, can be computed for arbitrary $v$ in the same time that $\nabla_{\theta} L(z_i, \hat{\theta})$ would take, which is typically $O(p)$ (Pearlmutter, 1994).

下面讨论两种近似 $s_{\mathrm{test}}$ 的技术，它们都依赖于这样一个事实：$H_{\hat{\theta}}$ 中单独一项的 HVP，$\bigl[\nabla_{\theta}^{2} L(z_i, \hat{\theta})\bigr]v$，对任意 $v$ 都可以在与计算 $\nabla_{\theta} L(z_i, \hat{\theta})$ 相同的时间内完成，通常是 $O(p)$（Pearlmutter, 1994）。

Conjugate gradients (CG).

共轭梯度（CG）。

The first technique is a standard transformation of matrix inversion into an optimization problem. Since $H_{\hat{\theta}} \succ 0$ by assumption, $H_{\hat{\theta}}^{-1}v \equiv \arg\min_{t} \bigl\{ \frac{1}{2} t^{\top} H_{\hat{\theta}} t - v^{\top} t \bigr\}$. We can solve this with CG approaches that only require the evaluation of $H_{\hat{\theta}} t$, which takes $O(np)$ time, without explicitly forming $H_{\hat{\theta}}$. While an exact solution takes $p$ CG iterations, in practice we can get a good approximation with fewer iterations; see Martens (2010) for more details.

第一种技术是把矩阵求逆标准地改写成优化问题。由于按假定 $H_{\hat{\theta}} \succ 0$，有 $H_{\hat{\theta}}^{-1}v \equiv \arg\min_{t} \bigl\{ \frac{1}{2} t^{\top} H_{\hat{\theta}} t - v^{\top} t \bigr\}$。可以用只需计算 $H_{\hat{\theta}} t$ 的共轭梯度来求解，耗时 $O(np)$，不必显式形成 $H_{\hat{\theta}}$。精确求解需要 $p$ 次 CG 迭代，但实践中用更少的迭代就能得到很好的近似；细节见 Martens (2010)。

Stochastic estimation.

随机估计。

With large datasets, standard CG can be slow; each iteration still goes through all $n$ training points. We use a method developed by Agarwal et al. (2016) to get an estimator that only samples a single point per iteration, which results in significant speedups.

数据量大时，标准 CG 可能仍很慢：每次迭代还是要扫过全部 $n$ 个训练点。我们采用 Agarwal et al. (2016) 提出的方法，得到每步只抽一个点的估计器，从而显著加速。

Dropping the $\hat{\theta}$ subscript for clarity, let $H_{j}^{-1} \stackrel{\mathrm{def}}{=} \sum_{i=0}^{j} (I - H)^{i}$, the first $j$ terms in the Taylor expansion of $H^{-1}$. Rewrite this recursively as $H_{j}^{-1} = I + (I - H) H_{j-1}^{-1}$. From the validity of the Taylor expansion, $H_{j}^{-1} \to H^{-1}$ as $j \to \infty$. The key is that at each iteration, we can substitute the full $H$ with a draw from any unbiased (and faster-to-compute) estimator of $H$ to form $\tilde{H}_{j}$. Since $\mathbb{E}[\tilde{H}_{j}^{-1}] = H_{j}^{-1}$, we still have $\mathbb{E}[\tilde{H}_{j}^{-1}] \to H^{-1}$.

为书写清楚，下面略去 $\hat{\theta}$ 下标。令 $H_{j}^{-1} \stackrel{\mathrm{def}}{=} \sum_{i=0}^{j} (I - H)^{i}$，即 $H^{-1}$ 的 Taylor 展开的前 $j$ 项。递归地写成 $H_{j}^{-1} = I + (I - H) H_{j-1}^{-1}$。由 Taylor 展开的成立性，当 $j \to \infty$ 时 $H_{j}^{-1} \to H^{-1}$。关键在于：每一步都可以把完整的 $H$ 换成任何一个无偏（且算得更快）的估计，从而形成 $\tilde{H}_{j}$。由于 $\mathbb{E}[\tilde{H}_{j}^{-1}] = H_{j}^{-1}$，仍然有 $\mathbb{E}[\tilde{H}_{j}^{-1}] \to H^{-1}$。

2 We assume w.l.o.g. that $\forall i,\, \nabla_{\theta}^{2} L(z_i, \hat{\theta}) \preceq I$; if this is not true, we can scale the loss down without affecting the parameters. In some cases, we can get an upper bound on $\nabla_{\theta}^{2} L(z_i, \hat{\theta})$ (e.g., for linear models and bounded input), which makes this easy. Otherwise, we treat the scaling as a separate hyperparameter and tune it such that the Taylor expansion converges.

2 我们不失一般性地假定 $\forall i,\, \nabla_{\theta}^{2} L(z_i, \hat{\theta}) \preceq I$；若不成立，可以把损失缩小一个尺度而不影响参数。有些情况下可以得到 $\nabla_{\theta}^{2} L(z_i, \hat{\theta})$ 的上界（例如线性模型且输入有界），这件事就很容易。否则，把这个尺度当作单独的超参数，调到 Taylor 展开收敛为止。

In particular, we can uniformly sample $z_i$ and use $\nabla_{\theta}^{2} L(z_i, \hat{\theta})$ as an unbiased estimator of $H$. This gives us the following procedure: uniformly sample $t$ points $z_{s_1}, \ldots, z_{s_t}$ from the training data; define $\tilde{H}_{0}^{-1}v = v$; and recursively compute $\tilde{H}_{j}^{-1}v = v + \bigl(I - \nabla_{\theta}^{2} L(z_{s_j}, \hat{\theta})\bigr)\tilde{H}_{j-1}^{-1}v$, taking $\tilde{H}_{t}^{-1}v$ as our final unbiased estimate of $H^{-1}v$. We pick $t$ to be large enough such that $\tilde{H}_{t}$ stabilizes, and to reduce variance we repeat this procedure $r$ times and average results. Empirically, we found this significantly faster than CG.

具体地，可以均匀抽取 $z_i$，并用 $\nabla_{\theta}^{2} L(z_i, \hat{\theta})$ 作为 $H$ 的无偏估计。于是有如下步骤：从训练数据中均匀抽取 $t$ 个点 $z_{s_1}, \ldots, z_{s_t}$；定义 $\tilde{H}_{0}^{-1}v = v$；再递归计算 $\tilde{H}_{j}^{-1}v = v + \bigl(I - \nabla_{\theta}^{2} L(z_{s_j}, \hat{\theta})\bigr)\tilde{H}_{j-1}^{-1}v$，以 $\tilde{H}_{t}^{-1}v$ 作为 $H^{-1}v$ 的最终无偏估计。把 $t$ 取得足够大，使 $\tilde{H}_{t}$ 稳定；为降低方差，把这一过程重复 $r$ 次再取平均。经验上，这比 CG 快得多。

We note that the original method of Agarwal et al. (2016) dealt only with generalized linear models, for which $\bigl[\nabla_{\theta}^{2} L(z_i, \hat{\theta})\bigr]v$ can be efficiently computed in $O(p)$ time. In our case, we rely on Pearlmutter (1994)’s more general algorithm for fast HVPs, described above, to achieve the same time complexity.

需要说明，Agarwal et al. (2016) 的原方法只处理广义线性模型，在那种情况下 $\bigl[\nabla_{\theta}^{2} L(z_i, \hat{\theta})\bigr]v$ 可以在 $O(p)$ 时间内高效计算。我们这里则依赖上面介绍的 Pearlmutter (1994) 更一般的快速 HVP 算法，达到同样的时间复杂度。

3 To increase stability, especially with non-convex models (see Section 4.2), we can also sample a mini-batch of training points at each iteration, instead of relying on a single training point.

3 为提高稳定性，尤其是对非凸模型（见第 4.2 节），每一步也可以抽取一小批训练点，而不是只依赖单个训练点。

With these techniques, we can compute $I_{\mathrm{up,loss}}(z_i, z_{\mathrm{test}})$ on all training points $z_i$ in $O(np + rtp)$ time; we show in Section 4.1 that empirically, choosing $rt = O(n)$ gives accurate results. Similarly, we compute $I_{\mathrm{pert,loss}}(z_i, z_{\mathrm{test}})^{\top} = -\frac{1}{n} \nabla_{\theta} L(z_{\mathrm{test}}, \hat{\theta})^{\top} H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z_i, \hat{\theta})$ with two matrix-vector products: we first compute $s_{\mathrm{test}}$, then $s_{\mathrm{test}}^{\top} \nabla_{x} \nabla_{\theta} L(z_i, \hat{\theta})$, with the same HVP trick. These computations are easy to implement in auto-grad systems like TensorFlow (Abadi et al., 2015) and Theano (Theano D. Team, 2016), as users need only specify $L$; the rest is automatically handled.

有了这些技术，就可以在 $O(np + rtp)$ 时间内对所有训练点 $z_i$ 计算 $I_{\mathrm{up,loss}}(z_i, z_{\mathrm{test}})$；第 4.1 节将表明，经验上取 $rt = O(n)$ 就能得到准确结果。类似地，计算 $I_{\mathrm{pert,loss}}(z_i, z_{\mathrm{test}})^{\top} = -\frac{1}{n} \nabla_{\theta} L(z_{\mathrm{test}}, \hat{\theta})^{\top} H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z_i, \hat{\theta})$ 只需两次矩阵-向量积：先算 $s_{\mathrm{test}}$，再算 $s_{\mathrm{test}}^{\top} \nabla_{x} \nabla_{\theta} L(z_i, \hat{\theta})$，用的是同一套 HVP 技巧。这些计算很容易在 TensorFlow（Abadi et al., 2015）和 Theano（Theano D. Team, 2016）这类自动求导系统里实现：用户只需指定 $L$，其余都自动完成。

4. Validation and Extensions

4. 验证与推广

Recall that influence functions are asymptotic approximations of leave-one-out retraining under the assumptions that (i) the model parameters $\hat{\theta}$ minimize the empirical risk, and that (ii) the empirical risk is twice-differentiable and strictly convex. Here, we empirically show that influence functions are accurate approximations (Section 4.1) that provide useful information even when these assumptions are violated (Sections 4.2, 4.3).

回顾一下，影响函数是在如下假定下对留一法重训的渐近近似：(i) 模型参数 $\hat{\theta}$ 最小化经验风险；(ii) 经验风险二阶可微且严格凸。下面用实验表明，影响函数是准确的近似（第 4.1 节），即使这些假定被打破，它们仍能提供有用信息（第 4.2、4.3 节）。

4.1. Influence functions vs. leave-one-out retraining

4.1. 影响函数与留一法重训的对比

Influence functions assume that the weight on a training point is changed by an infinitesimally small $\epsilon$. To investigate the accuracy of using influence functions to approximate the effect of removing a training point and retraining, we compared $-\frac{1}{n} I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ with $L(z_{\mathrm{test}}, \hat{\theta}_{-z}) - L(z_{\mathrm{test}}, \hat{\theta})$ (i.e., actually doing leave-one-out retraining). With a logistic regression model on 10-class MNIST, the predicted and actual changes matched closely (Fig 2-Left).

影响函数假定训练点上的权重只被改动一个无穷小的 $\epsilon$。为考察用它来近似“去掉一个训练点再重训”的效果有多准，我们把 $-\frac{1}{n} I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ 与 $L(z_{\mathrm{test}}, \hat{\theta}_{-z}) - L(z_{\mathrm{test}}, \hat{\theta})$（也就是真正做留一法重训）相比较。在十类 MNIST 的逻辑回归上，预测变化与实际变化吻合得很紧密（图 2 左）。

4 We trained with L-BFGS (Liu & Nocedal, 1989), with $L_2$ regularization of 0.01, $n = 55{,}000$, and $p = 7{,}840$ parameters.

4 我们用 L-BFGS（Liu & Nocedal, 1989）训练，$L_2$ 正则系数为 0.01，$n = 55{,}000$，参数个数 $p = 7{,}840$。

The stochastic approximation from Agarwal et al. (2016) was also accurate with $r = 10$ repeats and $t = 5{,}000$ iterations (Fig 2-Mid). Since each iteration only requires one HVP $\bigl[\nabla_{\theta}^{2} L(z_i, \hat{\theta})\bigr]v$, this runs quickly: in fact, we accurately estimated $H^{-1}v$ without even looking at every data point, since $n = 55{,}000 > rt$. Surprisingly, even $r = 1$ worked; while results were noisier, it was still able to identify the most influential points.

Agarwal et al. (2016) 的随机近似在重复 $r = 10$ 次、迭代 $t = 5{,}000$ 步时同样准确（图 2 中）。每步只需一次 HVP $\bigl[\nabla_{\theta}^{2} L(z_i, \hat{\theta})\bigr]v$，因此跑得很快：实际上，由于 $n = 55{,}000 > rt$，我们甚至不必看遍每个数据点，就能准确估计 $H^{-1}v$。出人意料的是，即便 $r = 1$ 也行；结果噪声更大，但仍能找出影响最大的那些点。

Figure 2. Influence matches leave-one-out retraining. We arbitrarily picked a wrongly-classified test point $z_{\mathrm{test}}$, but this trend held more broadly. These results are from 10-class MNIST. Left: For each of the 500 training points $z$ with largest $|I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})|$, we plotted $-\frac{1}{n} \cdot I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ against the actual change in test loss after removing that point and retraining. The inverse HVP was solved exactly with CG. Mid: Same, but with the stochastic approximation. Right: The same plot for a CNN, computed on the 100 most influential points with CG. For the actual difference in loss, we removed each point and retrained from $\tilde{\theta}$ for 30k steps.

图 2. 影响与留一法重训相符。我们任意选取了一个被分错的测试点 $z_{\mathrm{test}}$，但这一趋势更普遍地成立。结果来自十类 MNIST。左：对 $|I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})|$ 最大的 500 个训练点 $z$，把 $-\frac{1}{n} \cdot I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ 与去掉该点并重训后测试损失的实际变化相对照。逆 HVP 用 CG 精确求解。中：同样的图，但改用随机近似。右：卷积神经网络上的同样对照，用 CG 对影响最大的 100 个点计算。实际损失差的做法是：去掉每个点后，从 $\tilde{\theta}$ 再训练 3 万步。

4.2. Non-convexity and non-convergence

4.2. 非凸与未收敛

In Section 2, we took $\hat{\theta}$ as the global minimum. In practice, if we obtain our parameters $\tilde{\theta}$ by running SGD with early stopping or on non-convex objectives, $\tilde{\theta} \neq \hat{\theta}$. As a result, $H_{\tilde{\theta}}$ could have negative eigenvalues. We show that influence functions on $\tilde{\theta}$ still give meaningful results in practice.

第 2 节把 $\hat{\theta}$ 当作全局最小点。实践中，若参数 $\tilde{\theta}$ 来自带提前停止的 SGD，或来自非凸目标，则 $\tilde{\theta} \neq \hat{\theta}$。结果是 $H_{\tilde{\theta}}$ 可能有负特征值。我们表明，在 $\tilde{\theta}$ 上计算的影响函数在实践中仍然有意义。

Our approach is to form a convex quadratic approximation of the loss around $\tilde{\theta}$, i.e., $\tilde{L}(z, \theta) = L(z, \tilde{\theta}) + \nabla L(z, \tilde{\theta})^{\top}(\theta - \tilde{\theta}) + \frac{1}{2}(\theta - \tilde{\theta})^{\top}(H_{\tilde{\theta}} + \lambda I)(\theta - \tilde{\theta})$. Here, $\lambda$ is a damping term that we add if $H_{\tilde{\theta}}$ has negative eigenvalues; this corresponds to adding $L_2$ regularization on $\theta$. We then calculate $I_{\mathrm{up,loss}}$ using $\tilde{L}$. If $\tilde{\theta}$ is close to a local minimum, this is correlated with the result of taking a Newton step from $\tilde{\theta}$ after removing $\epsilon$ weight from $z$ (see appendix B).

我们的做法是在 $\tilde{\theta}$ 附近对损失做凸的二次近似，即 $\tilde{L}(z, \theta) = L(z, \tilde{\theta}) + \nabla L(z, \tilde{\theta})^{\top}(\theta - \tilde{\theta}) + \frac{1}{2}(\theta - \tilde{\theta})^{\top}(H_{\tilde{\theta}} + \lambda I)(\theta - \tilde{\theta})$。这里的 $\lambda$ 是阻尼项：若 $H_{\tilde{\theta}}$ 有负特征值就加上它，相当于对 $\theta$ 加上 $L_2$ 正则。然后用 $\tilde{L}$ 计算 $I_{\mathrm{up,loss}}$。若 $\tilde{\theta}$ 接近某个局部最小点，这与从 $\tilde{\theta}$ 出发、在去掉 $z$ 上 $\epsilon$ 权重后再走一步牛顿法的结果相关（见附录 B）。

We checked the behavior of $I_{\mathrm{up,loss}}$ in a non-convergent, non-convex setting by training a convolutional neural network for 500k iterations. The model had not converged and $H_{\tilde{\theta}}$ was not PD, so we added a damping term with $\lambda = 0.01$. Even in this difficult setting, the predicted and actual changes in loss were highly correlated (Pearson’s $R = 0.86$, Fig 2-Right).

我们在一个未收敛、非凸的设定里检查了 $I_{\mathrm{up,loss}}$ 的表现：把卷积神经网络训练 50 万步。模型尚未收敛，且 $H_{\tilde{\theta}}$ 不正定，于是加上 $\lambda = 0.01$ 的阻尼。即便在这种困难设定下，预测的损失变化与实际变化仍然高度相关（Pearson 相关系数 $R = 0.86$，图 2 右）。

5 The network had 7 sets of convolutional layers with $\tanh(\cdot)$ non-linearities, modeled after the all-convolutional network from (Springenberg et al., 2014). For speed, we used 10% of the MNIST training set and only 2,616 parameters, since repeatedly retraining the network was expensive. Training was done with mini-batches of 500 examples and the Adam optimizer (Kingma & Ba, 2014). The model had not converged after 500k iterations; training it for another 500k iterations, using a full training pass for each iteration, reduced train loss from 0.14 to 0.12.

5 网络有 7 组带 $\tanh(\cdot)$ 非线性的卷积层，结构仿自 (Springenberg et al., 2014) 的全卷积网络。为了速度，我们只用了 MNIST 训练集的 10%，参数也只有 2,616 个，因为反复重训网络代价很高。训练使用大小为 500 的小批量，以及 Adam 优化器（Kingma & Ba, 2014）。50 万步后模型尚未收敛；再训 50 万步，且每步扫过整个训练集，训练损失从 0.14 降到 0.12。

4.3. Non-differentiable losses

4.3. 不可微损失

What happens when the derivatives of the loss, $\nabla_{\theta} L$ and $\nabla_{\theta}^{2} L$, do not exist? In this section, we show that influence functions computed on smooth approximations to non-differentiable losses can predict the behavior of the original, non-differentiable loss under leave-one-out retraining. The robustness of this approximation suggests that we can train non-differentiable models and swap out non-differentiable components for smoothed versions for the purposes of calculating influence.

如果损失的导数 $\nabla_{\theta} L$ 和 $\nabla_{\theta}^{2} L$ 并不存在，会怎样？本节表明，在不可微损失的光滑近似上算出的影响函数，可以预测原始不可微损失在留一法重训下的行为。这一近似相当稳健，意味着我们可以先训练不可微模型，再在计算影响时把不可微的部分换成光滑版本。

To see this, we trained a linear SVM on the same 1s vs. 7s MNIST task in Section 2.3. This involves minimizing $\mathrm{Hinge}(s) = \max(0, 1 - s)$; this simple piecewise linear function is similar to ReLUs, which cause non-differentiability in neural networks. We set the derivatives at the hinge to 0 and calculated $I_{\mathrm{up,loss}}$. As one might expect, this was inaccurate (Fig 3b-Left): the second derivative carries no information about how close a support vector $z$ is to the hinge, so the quadratic approximation of $L(z, \hat{\theta})$ is linear (up to regularization), which leads to $I_{\mathrm{up,loss}}(z, z_{\mathrm{test}})$ overestimating the influence of $z$.