Generative Adversarial Networks

Generative Adversarial Nets

生成对抗网络

Ian J. Goodfellow, Jean Pouget-Abadie∗, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair†, Aaron Courville, Yoshua Bengio‡

Département d’informatique et de recherche opérationnelle, Université de Montréal, Montréal, QC H3C 3J7

Ian J. Goodfellow, Jean Pouget-Abadie∗, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair†, Aaron Courville, Yoshua Bengio‡

蒙特利尔大学信息与运筹学系，加拿大魁北克省蒙特利尔，QC H3C 3J7

Abstract

摘要

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model $G$ that captures the data distribution, and a discriminative model $D$ that estimates the probability that a sample came from the training data rather than $G$. The training procedure for $G$ is to maximize the probability of $D$ making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions $G$ and $D$, a unique solution exists, with $G$ recovering the training data distribution and $D$ equal to $\frac{1}{2}$ everywhere. In the case where $G$ and $D$ are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.

我们提出一个通过对抗过程来估计生成模型的新框架。在这个框架中，我们同时训练两个模型：一个捕捉数据分布的生成模型 $G$，以及一个判别模型 $D$，后者估计某个样本来自训练数据而不是来自 $G$ 的概率。$G$ 的训练目标是让 $D$ 犯错的概率最大。这个框架对应一个双人极小极大博弈。在 $G$ 和 $D$ 可以是任意函数的空间里，存在唯一解：$G$ 复现训练数据的分布，$D$ 处处等于 $\frac{1}{2}$。当 $G$ 和 $D$ 由多层感知机定义时，整个系统可以用反向传播训练。无论训练还是生成样本，都不需要任何马尔可夫链或展开的近似推断网络。实验通过对生成样本的定性和定量评估，展示了这一框架的潜力。

1 Introduction

1 引言

The promise of deep learning is to discover rich, hierarchical models [2] that represent probability distributions over the kinds of data encountered in artificial intelligence applications, such as natural images, audio waveforms containing speech, and symbols in natural language corpora. So far, the most striking successes in deep learning have involved discriminative models, usually those that map a high-dimensional, rich sensory input to a class label [14, 22]. These striking successes have primarily been based on the backpropagation and dropout algorithms, using piecewise linear units [19, 9, 10] which have a particularly well-behaved gradient. Deep generative models have had less of an impact, due to the difficulty of approximating many intractable probabilistic computations that arise in maximum likelihood estimation and related strategies, and due to difficulty of leveraging the benefits of piecewise linear units in the generative context. We propose a new generative model estimation procedure that sidesteps these difficulties.¹

深度学习的目标，是发现丰富的层次化模型 [2]，用来表示人工智能应用中各类数据的概率分布，例如自然图像、包含语音的音频波形、自然语言语料中的符号。到目前为止，深度学习最引人注目的成功都来自判别模型，通常是把高维、丰富的感官输入映射到类别标签的模型 [14, 22]。这些成功主要依靠反向传播和 dropout 算法，并使用梯度性质特别好的分段线性单元 [19, 9, 10]。深度生成模型的影响要小一些，原因有二：最大似然估计及相关方法中会出现许多难以处理的概率计算，很难近似；在生成任务中也难以发挥分段线性单元的优势。我们提出一种新的生成模型估计方法，绕开了这些困难。¹

In the proposed adversarial nets framework, the generative model is pitted against an adversary: a discriminative model that learns to determine whether a sample is from the model distribution or the data distribution. The generative model can be thought of as analogous to a team of counterfeiters, trying to produce fake currency and use it without detection, while the discriminative model is analogous to the police, trying to detect the counterfeit currency. Competition in this game drives both teams to improve their methods until the counterfeits are indistiguishable from the genuine articles.

在我们提出的对抗网络框架中，生成模型要与一个对手较量。这个对手是一个判别模型，它学习判断一个样本来自模型分布还是数据分布。可以把生成模型比作一伙伪钞制造者，他们试图制造假币并在不被发现的情况下使用；判别模型则像警察，试图识别假币。这场博弈中的竞争促使双方不断改进方法，直到伪造品与真品无法区分。

∗ Jean Pouget-Abadie is visiting Université de Montréal from Ecole Polytechnique.
† Sherjil Ozair is visiting Université de Montréal from Indian Institute of Technology Delhi.
‡ Yoshua Bengio is a CIFAR Senior Fellow.
¹ All code and hyperparameters available at http://www.github.com/goodfeli/adversarial

∗ Jean Pouget-Abadie 来自巴黎综合理工学院，在蒙特利尔大学访问。
† Sherjil Ozair 来自印度理工学院德里分校，在蒙特利尔大学访问。
‡ Yoshua Bengio 是 CIFAR 高级研究员。
¹ 全部代码和超参数见 http://www.github.com/goodfeli/adversarial

This framework can yield specific training algorithms for many kinds of model and optimization algorithm. In this article, we explore the special case when the generative model generates samples by passing random noise through a multilayer perceptron, and the discriminative model is also a multilayer perceptron. We refer to this special case as adversarial nets. In this case, we can train both models using only the highly successful backpropagation and dropout algorithms [17] and sample from the generative model using only forward propagation. No approximate inference or Markov chains are necessary.

这个框架可以为许多种模型和优化算法导出具体的训练算法。本文研究一种特殊情形：生成模型把随机噪声输入一个多层感知机来生成样本，判别模型也是一个多层感知机。我们把这种特殊情形称为对抗网络。在这种情形下，两个模型都可以只用非常成功的反向传播和 dropout 算法 [17] 来训练，从生成模型采样也只需要前向传播，不需要近似推断或马尔可夫链。

2 Related work

2 相关工作

An alternative to directed graphical models with latent variables are undirected graphical models with latent variables, such as restricted Boltzmann machines (RBMs) [27, 16], deep Boltzmann machines (DBMs) [26] and their numerous variants. The interactions within such models are represented as the product of unnormalized potential functions, normalized by a global summation/integration over all states of the random variables. This quantity (the partition function) and its gradient are intractable for all but the most trivial instances, although they can be estimated by Markov chain Monte Carlo (MCMC) methods. Mixing poses a significant problem for learning algorithms that rely on MCMC [3, 5].

带隐变量的有向图模型之外，另一类选择是带隐变量的无向图模型，例如受限玻尔兹曼机（RBM）[27, 16]、深度玻尔兹曼机（DBM）[26] 及其众多变体。这类模型内部的相互作用表示为未归一化势函数的乘积，再通过对随机变量所有状态的全局求和或积分来归一化。这个量（配分函数）及其梯度，除了最简单的情形外都难以计算，尽管可以用马尔可夫链蒙特卡洛（MCMC）方法估计。对依赖 MCMC 的学习算法来说，混合是一个重大难题 [3, 5]。

Deep belief networks (DBNs) [16] are hybrid models containing a single undirected layer and several directed layers. While a fast approximate layer-wise training criterion exists, DBNs incur the computational difficulties associated with both undirected and directed models.

深度信念网络（DBN）[16] 是一种混合模型，包含一个无向层和若干有向层。虽然存在一种快速的近似逐层训练准则，DBN 同时承受无向模型和有向模型两方面的计算困难。

Alternative criteria that do not approximate or bound the log-likelihood have also been proposed, such as score matching [18] and noise-contrastive estimation (NCE) [13]. Both of these require the learned probability density to be analytically specified up to a normalization constant. Note that in many interesting generative models with several layers of latent variables (such as DBNs and DBMs), it is not even possible to derive a tractable unnormalized probability density. Some models such as denoising auto-encoders [30] and contractive autoencoders have learning rules very similar to score matching applied to RBMs. In NCE, as in this work, a discriminative training criterion is employed to fit a generative model. However, rather than fitting a separate discriminative model, the generative model itself is used to discriminate generated data from samples a fixed noise distribution. Because NCE uses a fixed noise distribution, learning slows dramatically after the model has learned even an approximately correct distribution over a small subset of the observed variables.

也有人提出了不去近似对数似然、也不给它定界的替代准则，例如得分匹配 [18] 和噪声对比估计（NCE）[13]。这两种方法都要求所学的概率密度能够以解析形式给出（最多相差一个归一化常数）。要注意，在许多有多层隐变量的有趣生成模型中（如 DBN 和 DBM），甚至无法推导出可处理的未归一化概率密度。去噪自编码器 [30]、收缩自编码器等模型的学习规则，与应用于 RBM 的得分匹配非常相似。NCE 和本文一样，用判别式训练准则来拟合生成模型。但 NCE 没有另外拟合一个判别模型，而是用生成模型本身来区分生成数据和来自某个固定噪声分布的样本。由于 NCE 使用固定的噪声分布，一旦模型在一小部分观测变量上学到了哪怕近似正确的分布，学习就会急剧变慢。

Finally, some techniques do not involve defining a probability distribution explicitly, but rather train a generative machine to draw samples from the desired distribution. This approach has the advantage that such machines can be designed to be trained by back-propagation. Prominent recent work in this area includes the generative stochastic network (GSN) framework [5], which extends generalized denoising auto-encoders [4]: both can be seen as defining a parameterized Markov chain, i.e., one learns the parameters of a machine that performs one step of a generative Markov chain. Compared to GSNs, the adversarial nets framework does not require a Markov chain for sampling. Because adversarial nets do not require feedback loops during generation, they are better able to leverage piecewise linear units [19, 9, 10], which improve the performance of backpropagation but have problems with unbounded activation when used ina feedback loop. More recent examples of training a generative machine by back-propagating into it include recent work on auto-encoding variational Bayes [20] and stochastic backpropagation [24].

最后，还有一些技术不显式定义概率分布，而是训练一台生成机器，让它从目标分布中抽取样本。这种做法的好处是，可以把这样的机器设计成能用反向传播训练。这方面近期的代表性工作是生成随机网络（GSN）框架 [5]，它扩展了广义去噪自编码器 [4]：两者都可以看作定义了一条参数化的马尔可夫链，也就是学习一台执行生成式马尔可夫链一步转移的机器的参数。与 GSN 相比，对抗网络框架采样时不需要马尔可夫链。由于对抗网络在生成过程中不需要反馈回路，它能更好地利用分段线性单元 [19, 9, 10]；这类单元能提升反向传播的效果，但用在反馈回路中时会出现激活值无界的问题。通过反向传播来训练生成机器的更新例子，还包括自编码变分贝叶斯 [20] 和随机反向传播 [24] 方面的近期工作。

3 Adversarial nets

3 对抗网络

The adversarial modeling framework is most straightforward to apply when the models are both multilayer perceptrons. To learn the generator’s distribution $p_g$ over data $\boldsymbol{x}$, we define a prior on input noise variables $p_{\boldsymbol{z}}(\boldsymbol{z})$, then represent a mapping to data space as $G(\boldsymbol{z}; \theta_g)$, where $G$ is a differentiable function represented by a multilayer perceptron with parameters $\theta_g$. We also define a second multilayer perceptron $D(\boldsymbol{x}; \theta_d)$ that outputs a single scalar. $D(\boldsymbol{x})$ represents the probability that $\boldsymbol{x}$ came from the data rather than $p_g$. We train $D$ to maximize the probability of assigning the correct label to both training examples and samples from $G$. We simultaneously train $G$ to minimize $\log(1 - D(G(\boldsymbol{z})))$:

当两个模型都是多层感知机时，对抗建模框架用起来最直接。为了学习生成器在数据 $\boldsymbol{x}$ 上的分布 $p_g$，我们给输入噪声变量定义一个先验 $p_{\boldsymbol{z}}(\boldsymbol{z})$，再把到数据空间的映射表示为 $G(\boldsymbol{z}; \theta_g)$，其中 $G$ 是一个可微函数，由参数为 $\theta_g$ 的多层感知机表示。我们还定义第二个多层感知机 $D(\boldsymbol{x}; \theta_d)$，它输出一个标量。$D(\boldsymbol{x})$ 表示 $\boldsymbol{x}$ 来自数据而不是来自 $p_g$ 的概率。我们训练 $D$，使它给训练样本和 $G$ 生成的样本都打上正确标签的概率最大；同时训练 $G$，使 $\log(1 - D(G(\boldsymbol{z})))$ 最小：

In other words, $D$ and $G$ play the following two-player minimax game with value function $V(G, D)$:

$$\min_G \max_D V(D, G) = \mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}(\boldsymbol{x})}[\log D(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log(1 - D(G(\boldsymbol{z})))]. \tag{1}$$

换句话说，$D$ 和 $G$ 在进行如下的双人极小极大博弈，价值函数为 $V(G, D)$：

$$\min_G \max_D V(D, G) = \mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}(\boldsymbol{x})}[\log D(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log(1 - D(G(\boldsymbol{z})))]. \tag{1}$$

In the next section, we present a theoretical analysis of adversarial nets, essentially showing that the training criterion allows one to recover the data generating distribution as $G$ and $D$ are given enough capacity, i.e., in the non-parametric limit. See Figure 1 for a less formal, more pedagogical explanation of the approach. In practice, we must implement the game using an iterative, numerical approach. Optimizing $D$ to completion in the inner loop of training is computationally prohibitive, and on finite datasets would result in overfitting. Instead, we alternate between $k$ steps of optimizing $D$ and one step of optimizing $G$. This results in $D$ being maintained near its optimal solution, so long as $G$ changes slowly enough. This strategy is analogous to the way that SML/PCD [31, 29] training maintains samples from a Markov chain from one learning step to the next in order to avoid burning in a Markov chain as part of the inner loop of learning. The procedure is formally presented in Algorithm 1.

下一节给出对抗网络的理论分析，主要说明：只要 $G$ 和 $D$ 有足够的容量（即在非参数极限下），这个训练准则就能还原出生成数据的分布。图 1 对这一方法给出了不那么形式化、更便于理解的解释。实际中，我们必须用迭代的数值方法来实现这场博弈。在训练的内循环里把 $D$ 优化到底，计算代价高得无法承受，在有限数据集上还会导致过拟合。因此，我们交替进行：优化 $D$ $k$ 步，再优化 $G$ 一步。只要 $G$ 变化得足够慢，$D$ 就能保持在最优解附近。这一策略类似于 SML/PCD [31, 29] 训练的做法：在相邻学习步之间保留马尔可夫链的样本，以免在学习的内循环里每次都要让马尔可夫链重新预热。算法 1 给出了这一过程的正式描述。

In practice, equation 1 may not provide sufficient gradient for $G$ to learn well. Early in learning, when $G$ is poor, $D$ can reject samples with high confidence because they are clearly different from the training data. In this case, $\log(1 - D(G(\boldsymbol{z})))$ saturates. Rather than training $G$ to minimize $\log(1 - D(G(\boldsymbol{z})))$ we can train $G$ to maximize $\log D(G(\boldsymbol{z}))$. This objective function results in the same fixed point of the dynamics of $G$ and $D$ but provides much stronger gradients early in learning.

实际中，式 1 可能无法给 $G$ 提供足够的梯度，使它学好。在学习初期，$G$ 还很差，$D$ 能以很高的置信度拒绝生成样本，因为它们与训练数据明显不同。这时 $\log(1 - D(G(\boldsymbol{z})))$ 会饱和。与其训练 $G$ 去最小化 $\log(1 - D(G(\boldsymbol{z})))$，不如训练 $G$ 去最大化 $\log D(G(\boldsymbol{z}))$。这个目标函数使 $G$ 和 $D$ 的动力学有相同的不动点，但在学习初期能提供强得多的梯度。

Figure 1: Generative adversarial nets are trained by simultaneously updating the discriminative distribution ($D$, blue, dashed line) so that it discriminates between samples from the data generating distribution (black, dotted line) $p_{\boldsymbol{x}}$ from those of the generative distribution $p_g$ ($G$) (green, solid line). The lower horizontal line is the domain from which $\boldsymbol{z}$ is sampled, in this case uniformly. The horizontal line above is part of the domain of $\boldsymbol{x}$. The upward arrows show how the mapping $\boldsymbol{x} = G(\boldsymbol{z})$ imposes the non-uniform distribution $p_g$ on transformed samples. $G$ contracts in regions of high density and expands in regions of low density of $p_g$. (a) Consider an adversarial pair near convergence: $p_g$ is similar to $p_{\text{data}}$ and $D$ is a partially accurate classifier. (b) In the inner loop of the algorithm $D$ is trained to discriminate samples from data, converging to $D^*(\boldsymbol{x}) = \frac{p_{\text{data}}(\boldsymbol{x})}{p_{\text{data}}(\boldsymbol{x}) + p_g(\boldsymbol{x})}$. (c) After an update to $G$, gradient of $D$ has guided $G(\boldsymbol{z})$ to flow to regions that are more likely to be classified as data. (d) After several steps of training, if $G$ and $D$ have enough capacity, they will reach a point at which both cannot improve because $p_g = p_{\text{data}}$. The discriminator is unable to differentiate between the two distributions, i.e. $D(\boldsymbol{x}) = \frac{1}{2}$.

图 1：训练生成对抗网络时，同时更新判别分布（$D$，蓝色虚线），使它能把来自数据生成分布 $p_{\boldsymbol{x}}$（黑色点线）的样本与来自生成分布 $p_g$（$G$，绿色实线）的样本区分开。下方的水平线是采样 $\boldsymbol{z}$ 的定义域，这里是均匀采样；上方的水平线是 $\boldsymbol{x}$ 定义域的一部分。向上的箭头表示映射 $\boldsymbol{x} = G(\boldsymbol{z})$ 如何让变换后的样本服从非均匀分布 $p_g$。$G$ 在 $p_g$ 的高密度区域收缩，在低密度区域扩张。(a) 考虑一对接近收敛的对抗网络：$p_g$ 与 $p_{\text{data}}$ 相近，$D$ 是一个部分准确的分类器。(b) 在算法的内循环中，训练 $D$ 区分样本与数据，它收敛到 $D^*(\boldsymbol{x}) = \frac{p_{\text{data}}(\boldsymbol{x})}{p_{\text{data}}(\boldsymbol{x}) + p_g(\boldsymbol{x})}$。(c) 更新 $G$ 之后，$D$ 的梯度引导 $G(\boldsymbol{z})$ 流向更可能被判为数据的区域。(d) 经过若干步训练后，如果 $G$ 和 $D$ 容量足够，它们会到达一个双方都无法再改进的点，因为此时 $p_g = p_{\text{data}}$。判别器无法区分这两个分布，即 $D(\boldsymbol{x}) = \frac{1}{2}$。

4 Theoretical Results

4 理论结果

The generator $G$ implicitly defines a probability distribution $p_g$ as the distribution of the samples $G(\boldsymbol{z})$ obtained when $\boldsymbol{z} \sim p_{\boldsymbol{z}}$. Therefore, we would like Algorithm 1 to converge to a good estimator of $p_{\text{data}}$, if given enough capacity and training time. The results of this section are done in a non-parametric setting, e.g. we represent a model with infinite capacity by studying convergence in the space of probability density functions.

生成器 $G$ 隐式地定义了一个概率分布 $p_g$，即当 $\boldsymbol{z} \sim p_{\boldsymbol{z}}$ 时所得样本 $G(\boldsymbol{z})$ 的分布。因此，我们希望在容量和训练时间足够的条件下，算法 1 能收敛到 $p_{\text{data}}$ 的一个良好估计。本节的结果都在非参数设定下得出，也就是说，我们通过研究在概率密度函数空间中的收敛，来表示一个容量无限的模型。

We will show in section 4.1 that this minimax game has a global optimum for $p_g = p_{\text{data}}$. We will then show in section 4.2 that Algorithm 1 optimizes Eq 1, thus obtaining the desired result.

我们将在 4.1 节证明，这个极小极大博弈在 $p_g = p_{\text{data}}$ 时取得全局最优；然后在 4.2 节证明，算法 1 能够优化式 1，从而得到想要的结果。

Algorithm 1 Minibatch stochastic gradient descent training of generative adversarial nets. The number of steps to apply to the discriminator, $k$, is a hyperparameter. We used $k = 1$, the least expensive option, in our experiments.

for number of training iterations do

• for $k$ steps do
  • Sample minibatch of $m$ noise samples $\{\boldsymbol{z}^{(1)}, \ldots, \boldsymbol{z}^{(m)}\}$ from noise prior $p_g(\boldsymbol{z})$.
  • Sample minibatch of $m$ examples $\{\boldsymbol{x}^{(1)}, \ldots, \boldsymbol{x}^{(m)}\}$ from data generating distribution $p_{\text{data}}(\boldsymbol{x})$.
  • Update the discriminator by ascending its stochastic gradient: $$\nabla_{\theta_d} \frac{1}{m} \sum_{i=1}^{m} \left[\log D\left(\boldsymbol{x}^{(i)}\right) + \log\left(1 - D\left(G\left(\boldsymbol{z}^{(i)}\right)\right)\right)\right].$$
• end for
• Sample minibatch of $m$ noise samples $\{\boldsymbol{z}^{(1)}, \ldots, \boldsymbol{z}^{(m)}\}$ from noise prior $p_g(\boldsymbol{z})$.
• Update the generator by descending its stochastic gradient: $$\nabla_{\theta_g} \frac{1}{m} \sum_{i=1}^{m} \log\left(1 - D\left(G\left(\boldsymbol{z}^{(i)}\right)\right)\right).$$

end for

The gradient-based updates can use any standard gradient-based learning rule. We used momentum in our experiments.

算法 1 用小批量随机梯度下降训练生成对抗网络。每轮对判别器执行的步数 $k$ 是一个超参数。我们在实验中使用代价最低的 $k = 1$。

for 训练迭代次数 do

• for $k$ 步 do
  • 从噪声先验 $p_g(\boldsymbol{z})$ 中采样 $m$ 个噪声样本组成的小批量 $\{\boldsymbol{z}^{(1)}, \ldots, \boldsymbol{z}^{(m)}\}$。
  • 从数据生成分布 $p_{\text{data}}(\boldsymbol{x})$ 中采样 $m$ 个样本组成的小批量 $\{\boldsymbol{x}^{(1)}, \ldots, \boldsymbol{x}^{(m)}\}$。
  • 沿随机梯度上升方向更新判别器：$$\nabla_{\theta_d} \frac{1}{m} \sum_{i=1}^{m} \left[\log D\left(\boldsymbol{x}^{(i)}\right) + \log\left(1 - D\left(G\left(\boldsymbol{z}^{(i)}\right)\right)\right)\right].$$
• end for
• 从噪声先验 $p_g(\boldsymbol{z})$ 中采样 $m$ 个噪声样本组成的小批量 $\{\boldsymbol{z}^{(1)}, \ldots, \boldsymbol{z}^{(m)}\}$。
• 沿随机梯度下降方向更新生成器：$$\nabla_{\theta_g} \frac{1}{m} \sum_{i=1}^{m} \log\left(1 - D\left(G\left(\boldsymbol{z}^{(i)}\right)\right)\right).$$

end for

基于梯度的更新可以使用任何标准的梯度学习规则。我们在实验中使用了动量法。

4.1 Global Optimality of $p_g = p_{\text{data}}$

4.1 $p_g = p_{\text{data}}$ 的全局最优性

We first consider the optimal discriminator $D$ for any given generator $G$.

Proposition 1. For $G$ fixed, the optimal discriminator $D$ is

$$D^*_G(\boldsymbol{x}) = \frac{p_{\text{data}}(\boldsymbol{x})}{p_{\text{data}}(\boldsymbol{x}) + p_g(\boldsymbol{x})} \tag{2}$$

我们先考虑对任意给定的生成器 $G$，最优判别器 $D$ 是什么。

命题 1. 固定 $G$ 时，最优判别器 $D$ 为

$$D^*_G(\boldsymbol{x}) = \frac{p_{\text{data}}(\boldsymbol{x})}{p_{\text{data}}(\boldsymbol{x}) + p_g(\boldsymbol{x})} \tag{2}$$

Proof. The training criterion for the discriminator $D$, given any generator $G$, is to maximize the quantity $V(G, D)$

$$V(G, D) = \int_{\boldsymbol{x}} p_{\text{data}}(\boldsymbol{x}) \log(D(\boldsymbol{x}))\,dx + \int_{\boldsymbol{z}} p_{\boldsymbol{z}}(\boldsymbol{z}) \log(1 - D(g(\boldsymbol{z})))\,dz = \int_{\boldsymbol{x}} p_{\text{data}}(\boldsymbol{x}) \log(D(\boldsymbol{x})) + p_g(\boldsymbol{x}) \log(1 - D(\boldsymbol{x}))\,dx \tag{3}$$

For any $(a, b) \in \mathbb{R}^2 \setminus \{0, 0\}$, the function $y \to a \log(y) + b \log(1 - y)$ achieves its maximum in $[0, 1]$ at $\frac{a}{a+b}$. The discriminator does not need to be defined outside of $Supp(p_{\text{data}}) \cup Supp(p_g)$, concluding the proof. $\square$

证明. 给定任意生成器 $G$，判别器 $D$ 的训练准则是最大化 $V(G, D)$：

$$V(G, D) = \int_{\boldsymbol{x}} p_{\text{data}}(\boldsymbol{x}) \log(D(\boldsymbol{x}))\,dx + \int_{\boldsymbol{z}} p_{\boldsymbol{z}}(\boldsymbol{z}) \log(1 - D(g(\boldsymbol{z})))\,dz = \int_{\boldsymbol{x}} p_{\text{data}}(\boldsymbol{x}) \log(D(\boldsymbol{x})) + p_g(\boldsymbol{x}) \log(1 - D(\boldsymbol{x}))\,dx \tag{3}$$

对任意 $(a, b) \in \mathbb{R}^2 \setminus \{0, 0\}$，函数 $y \to a \log(y) + b \log(1 - y)$ 在 $[0, 1]$ 上于 $\frac{a}{a+b}$ 处取得最大值。判别器无需在 $Supp(p_{\text{data}}) \cup Supp(p_g)$ 之外定义，证毕。$\square$

Note that the training objective for $D$ can be interpreted as maximizing the log-likelihood for estimating the conditional probability $P(Y = y|\boldsymbol{x})$, where $Y$ indicates whether $\boldsymbol{x}$ comes from $p_{\text{data}}$ (with $y = 1$) or from $p_g$ (with $y = 0$). The minimax game in Eq. 1 can now be reformulated as:

$$\begin{aligned} C(G) &= \max_D V(G, D) \\ &= \mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}[\log D^*_G(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}}[\log(1 - D^*_G(G(\boldsymbol{z})))] \\ &= \mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}[\log D^*_G(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{x} \sim p_g}[\log(1 - D^*_G(\boldsymbol{x}))] \\ &= \mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}\left[\log \frac{p_{\text{data}}(\boldsymbol{x})}{P_{\text{data}}(\boldsymbol{x}) + p_g(\boldsymbol{x})}\right] + \mathbb{E}_{\boldsymbol{x} \sim p_g}\left[\log \frac{p_g(\boldsymbol{x})}{p_{\text{data}}(\boldsymbol{x}) + p_g(\boldsymbol{x})}\right] \end{aligned} \tag{4}$$

注意，$D$ 的训练目标可以理解为：为估计条件概率 $P(Y = y|\boldsymbol{x})$ 而最大化对数似然，其中 $Y$ 表示 $\boldsymbol{x}$ 来自 $p_{\text{data}}$（$y = 1$）还是来自 $p_g$（$y = 0$）。式 1 中的极小极大博弈现在可以改写为：

$$\begin{aligned} C(G) &= \max_D V(G, D) \\ &= \mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}[\log D^*_G(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}}[\log(1 - D^*_G(G(\boldsymbol{z})))] \\ &= \mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}[\log D^*_G(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{x} \sim p_g}[\log(1 - D^*_G(\boldsymbol{x}))] \\ &= \mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}\left[\log \frac{p_{\text{data}}(\boldsymbol{x})}{P_{\text{data}}(\boldsymbol{x}) + p_g(\boldsymbol{x})}\right] + \mathbb{E}_{\boldsymbol{x} \sim p_g}\left[\log \frac{p_g(\boldsymbol{x})}{p_{\text{data}}(\boldsymbol{x}) + p_g(\boldsymbol{x})}\right] \end{aligned} \tag{4}$$

Theorem 1. The global minimum of the virtual training criterion $C(G)$ is achieved if and only if $p_g = p_{\text{data}}$. At that point, $C(G)$ achieves the value $-\log 4$.

定理 1. 虚拟训练准则 $C(G)$ 取得全局最小值，当且仅当 $p_g = p_{\text{data}}$。此时 $C(G)$ 的值为 $-\log 4$。

Proof. For $p_g = p_{\text{data}}$, $D^*_G(\boldsymbol{x}) = \frac{1}{2}$, (consider Eq. 2). Hence, by inspecting Eq. 4 at $D^*_G(\boldsymbol{x}) = \frac{1}{2}$, we find $C(G) = \log \frac{1}{2} + \log \frac{1}{2} = -\log 4$. To see that this is the best possible value of $C(G)$, reached only for $p_g = p_{\text{data}}$, observe that

$$\mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}[-\log 2] + \mathbb{E}_{\boldsymbol{x} \sim p_g}[-\log 2] = -\log 4$$

and that by subtracting this expression from $C(G) = V(D^*_G, G)$, we obtain:

$$C(G) = -\log(4) + KL\left(p_{\text{data}} \,\Big\|\, \frac{p_{\text{data}} + p_g}{2}\right) + KL\left(p_g \,\Big\|\, \frac{p_{\text{data}} + p_g}{2}\right) \tag{5}$$

where KL is the Kullback–Leibler divergence. We recognize in the previous expression the Jensen–Shannon divergence between the model’s distribution and the data generating process:

$$C(G) = -\log(4) + 2 \cdot JSD\left(p_{\text{data}} \,\|\, p_g\right) \tag{6}$$

Since the Jensen–Shannon divergence between two distributions is always non-negative and zero only when they are equal, we have shown that $C^* = -\log(4)$ is the global minimum of $C(G)$ and that the only solution is $p_g = p_{\text{data}}$, i.e., the generative model perfectly replicating the data generating process. $\square$

证明. 当 $p_g = p_{\text{data}}$ 时，$D^*_G(\boldsymbol{x}) = \frac{1}{2}$（见式 2）。因此，在 $D^*_G(\boldsymbol{x}) = \frac{1}{2}$ 处考察式 4，得到 $C(G) = \log \frac{1}{2} + \log \frac{1}{2} = -\log 4$。为了说明这是 $C(G)$ 可能取到的最好值，且只在 $p_g = p_{\text{data}}$ 时取到，注意到

$$\mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}[-\log 2] + \mathbb{E}_{\boldsymbol{x} \sim p_g}[-\log 2] = -\log 4$$

从 $C(G) = V(D^*_G, G)$ 中减去这个表达式，得到：

$$C(G) = -\log(4) + KL\left(p_{\text{data}} \,\Big\|\, \frac{p_{\text{data}} + p_g}{2}\right) + KL\left(p_g \,\Big\|\, \frac{p_{\text{data}} + p_g}{2}\right) \tag{5}$$

其中 KL 是 Kullback–Leibler 散度。上式中可以认出模型分布与数据生成过程之间的 Jensen–Shannon 散度：

$$C(G) = -\log(4) + 2 \cdot JSD\left(p_{\text{data}} \,\|\, p_g\right) \tag{6}$$

由于两个分布之间的 Jensen–Shannon 散度总是非负的，且只在两者相等时为零，我们就证明了 $C^* = -\log(4)$ 是 $C(G)$ 的全局最小值，唯一解是 $p_g = p_{\text{data}}$，也就是生成模型完美复现了数据生成过程。$\square$

4.2 Convergence of Algorithm 1

4.2 算法 1 的收敛性

Proposition 2. If $G$ and $D$ have enough capacity, and at each step of Algorithm 1, the discriminator is allowed to reach its optimum given $G$, and $p_g$ is updated so as to improve the criterion

$$\mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}[\log D^*_G(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{x} \sim p_g}[\log(1 - D^*_G(\boldsymbol{x}))]$$

then $p_g$ converges to $p_{\text{data}}$.

命题 2. 如果 $G$ 和 $D$ 的容量足够，并且在算法 1 的每一步中，判别器都能在给定 $G$ 的条件下达到最优，同时 $p_g$ 的更新使准则

$$\mathbb{E}_{\boldsymbol{x} \sim p_{\text{data}}}[\log D^*_G(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{x} \sim p_g}[\log(1 - D^*_G(\boldsymbol{x}))]$$

得到改进，那么 $p_g$ 收敛到 $p_{\text{data}}$。

Proof. Consider $V(G, D) = U(p_g, D)$ as a function of $p_g$ as done in the above criterion. Note that $U(p_g, D)$ is convex in $p_g$. The subderivatives of a supremum of convex functions include the derivative of the function at the point where the maximum is attained. In other words, if $f(x) = \sup_{\alpha \in \mathcal{A}} f_\alpha(x)$ and $f_\alpha(x)$ is convex in $x$ for every $\alpha$, then $\partial f_\beta(x) \in \partial f$ if $\beta = \arg\sup_{\alpha \in \mathcal{A}} f_\alpha(x)$. This is equivalent to computing a gradient descent update for $p_g$ at the optimal $D$ given the corresponding $G$. $\sup_D U(p_g, D)$ is convex in $p_g$ with a unique global optima as proven in Thm 1, therefore with sufficiently small updates of $p_g$, $p_g$ converges to $p_x$, concluding the proof. $\square$

证明. 像上面的准则那样，把 $V(G, D) = U(p_g, D)$ 看作 $p_g$ 的函数。注意 $U(p_g, D)$ 关于 $p_g$ 是凸的。一族凸函数的上确界的次导数，包含该函数在取得最大值处的导数。换句话说，如果 $f(x) = \sup_{\alpha \in \mathcal{A}} f_\alpha(x)$，且对每个 $\alpha$，$f_\alpha(x)$ 关于 $x$ 都是凸的，那么当 $\beta = \arg\sup_{\alpha \in \mathcal{A}} f_\alpha(x)$ 时，有 $\partial f_\beta(x) \in \partial f$。这等价于在给定相应 $G$ 的最优 $D$ 处，对 $p_g$ 做一次梯度下降更新。如定理 1 所证，$\sup_D U(p_g, D)$ 关于 $p_g$ 是凸的，且有唯一的全局最优解，因此只要 $p_g$ 的更新足够小，$p_g$ 就会收敛到 $p_x$，证毕。$\square$

In practice, adversarial nets represent a limited family of $p_g$ distributions via the function $G(\boldsymbol{z}; \theta_g)$, and we optimize $\theta_g$ rather than $p_g$ itself. Using a multilayer perceptron to define $G$ introduces multiple critical points in parameter space. However, the excellent performance of multilayer perceptrons in practice suggests that they are a reasonable model to use despite their lack of theoretical guarantees.

实际中，对抗网络通过函数 $G(\boldsymbol{z}; \theta_g)$ 表示的只是一个有限的 $p_g$ 分布族，我们优化的是 $\theta_g$，而不是 $p_g$ 本身。用多层感知机来定义 $G$，会在参数空间中引入多个临界点。不过，多层感知机在实践中的出色表现说明，尽管缺乏理论保证，用它作为模型仍然是合理的。

5 Experiments

5 实验

We trained adversarial nets an a range of datasets including MNIST [23], the Toronto Face Database (TFD) [28], and CIFAR-10 [21]. The generator nets used a mixture of rectifier linear activations [19, 9] and sigmoid activations, while the discriminator net used maxout [10] activations. Dropout [17] was applied in training the discriminator net. While our theoretical framework permits the use of dropout and other noise at intermediate layers of the generator, we used noise as the input to only the bottommost layer of the generator network.

我们在多个数据集上训练了对抗网络，包括 MNIST [23]、多伦多人脸数据库（TFD）[28] 和 CIFAR-10 [21]。生成器网络混合使用了修正线性激活 [19, 9] 和 sigmoid 激活，判别器网络使用 maxout [10] 激活。训练判别器网络时使用了 dropout [17]。虽然我们的理论框架允许在生成器的中间层使用 dropout 和其他噪声，但我们只把噪声作为生成器网络最底层的输入。

We estimate probability of the test set data under $p_g$ by fitting a Gaussian Parzen window to the samples generated with $G$ and reporting the log-likelihood under this distribution. The $\sigma$ parameter of the Gaussians was obtained by cross validation on the validation set. This procedure was introduced in Breuleux et al. [8] and used for various generative models for which the exact likelihood is not tractable [25, 3, 5]. Results are reported in Table 1. This method of estimating the likelihood has somewhat high variance and does not perform well in high dimensional spaces but it is the best method available to our knowledge. Advances in generative models that can sample but not estimate likelihood directly motivate further research into how to evaluate such models.

为了估计测试集数据在 $p_g$ 下的概率，我们对 $G$ 生成的样本拟合一个高斯 Parzen 窗口，并报告测试数据在这个分布下的对数似然。高斯核的 $\sigma$ 参数通过在验证集上交叉验证得到。这一做法由 Breuleux 等人 [8] 提出，已用于多种无法精确计算似然的生成模型 [25, 3, 5]。结果见表 1。这种似然估计方法方差偏高，在高维空间中表现不佳，但据我们所知，它是目前可用的最好方法。能够采样、却不能直接估计似然的生成模型取得进展，也推动人们进一步研究如何评估这类模型。

Model	MNIST	TFD
DBN [3]	138 ± 2	1909 ± 66
Stacked CAE [3]	121 ± 1.6	2110 ± 50
Deep GSN [6]	214 ± 1.1	1890 ± 29
Adversarial nets	225 ± 2	2057 ± 26

Table 1: Parzen window-based log-likelihood estimates. The reported numbers on MNIST are the mean log-likelihood of samples on test set, with the standard error of the mean computed across examples. On TFD, we computed the standard error across folds of the dataset, with a different $\sigma$ chosen using the validation set of each fold. On TFD, $\sigma$ was cross validated on each fold and mean log-likelihood on each fold were computed. For MNIST we compare against other models of the real-valued (rather than binary) version of dataset.

模型	MNIST	TFD
DBN [3]	138 ± 2	1909 ± 66
堆叠 CAE [3]	121 ± 1.6	2110 ± 50
深度 GSN [6]	214 ± 1.1	1890 ± 29
对抗网络	225 ± 2	2057 ± 26

表 1：基于 Parzen 窗口的对数似然估计。MNIST 上报告的数字是测试集样本的平均对数似然，均值的标准误按样本计算。在 TFD 上，标准误按数据集的各折计算，每一折都用该折的验证集选取不同的 $\sigma$。在 TFD 上，$\sigma$ 在每一折上交叉验证，并计算每一折的平均对数似然。在 MNIST 上，我们与其他使用该数据集实值版本（而非二值版本）的模型比较。

In Figures 2 and 3 we show samples drawn from the generator net after training. While we make no claim that these samples are better than samples generated by existing methods, we believe that these samples are at least competitive with the better generative models in the literature and highlight the potential of the adversarial framework.

图 2 和图 3 展示了训练后从生成器网络中抽取的样本。我们并不声称这些样本优于现有方法生成的样本，但我们认为，它们至少能与文献中较好的生成模型相比，并显示了对抗框架的潜力。

Figure 2: Visualization of samples from the model. Rightmost column shows the nearest training example of the neighboring sample, in order to demonstrate that the model has not memorized the training set. Samples are fair random draws, not cherry-picked. Unlike most other visualizations of deep generative models, these images show actual samples from the model distributions, not conditional means given samples of hidden units. Moreover, these samples are uncorrelated because the sampling process does not depend on Markov chain mixing. a) MNIST b) TFD c) CIFAR-10 (fully connected model) d) CIFAR-10 (convolutional discriminator and “deconvolutional” generator)

图 2：模型样本的可视化。最右一列是与相邻样本最接近的训练样本，用来说明模型并没有记住训练集。这些样本是公平的随机抽取结果，没有经过挑选。与大多数深度生成模型的可视化不同，这些图像是从模型分布中抽取的真实样本，而不是给定隐藏单元样本时的条件均值。此外，由于采样过程不依赖马尔可夫链的混合，这些样本彼此不相关。a) MNIST b) TFD c) CIFAR-10（全连接模型）d) CIFAR-10（卷积判别器和“反卷积”生成器）

Figure 3: Digits obtained by linearly interpolating between coordinates in $z$ space of the full model.

图 3：在完整模型的 $z$ 空间中对坐标做线性插值得到的数字。

	Deep directed graphical models	Deep undirected graphical models	Generative autoencoders	Adversarial models
Training	Inference needed during training.	Inference needed during training. MCMC needed to approximate partition function gradient.	Enforced tradeoff between mixing and power of reconstruction generation	Synchronizing the discriminator with the generator. Helvetica.
Inference	Learned approximate inference	Variational inference	MCMC-based inference	Learned approximate inference
Sampling	No difficulties	Requires Markov chain	Requires Markov chain	No difficulties
Evaluating p(x)	Intractable, may be approximated with AIS	Intractable, may be approximated with AIS	Not explicitly represented, may be approximated with Parzen density estimation	Not explicitly represented, may be approximated with Parzen density estimation
Model design	Nearly all models incur extreme difficulty	Careful design needed to ensure multiple properties	Any differentiable function is theoretically permitted	Any differentiable function is theoretically permitted

Table 2: Challenges in generative modeling: a summary of the difficulties encountered by different approaches to deep generative modeling for each of the major operations involving a model.

	深度有向图模型	深度无向图模型	生成式自编码器	对抗模型
训练	训练时需要推断。	训练时需要推断。需要 MCMC 来近似配分函数的梯度。	混合能力与重构生成能力之间必须取舍	判别器需要与生成器保持同步。Helvetica 情形。
推断	学习得到的近似推断	变分推断	基于 MCMC 的推断	学习得到的近似推断
采样	没有困难	需要马尔可夫链	需要马尔可夫链	没有困难
计算 p(x)	难以计算，可用 AIS 近似	难以计算，可用 AIS 近似	没有显式表示，可用 Parzen 密度估计近似	没有显式表示，可用 Parzen 密度估计近似
模型设计	几乎所有模型都极其困难	需要精心设计以保证多项性质	理论上允许任何可微函数	理论上允许任何可微函数

表 2：生成建模中的挑战：不同深度生成建模方法在涉及模型的各项主要操作中遇到的困难汇总。

6 Advantages and disadvantages

6 优点与缺点

This new framework comes with advantages and disadvantages relative to previous modeling frameworks. The disadvantages are primarily that there is no explicit representation of $p_g(\boldsymbol{x})$, and that $D$ must be synchronized well with $G$ during training (in particular, $G$ must not be trained too much without updating $D$, in order to avoid “the Helvetica scenario” in which $G$ collapses too many values of $\boldsymbol{z}$ to the same value of $\boldsymbol{x}$ to have enough diversity to model $p_{\text{data}}$), much as the negative chains of a Boltzmann machine must be kept up to date between learning steps. The advantages are that Markov chains are never needed, only backprop is used to obtain gradients, no inference is needed during learning, and a wide variety of functions can be incorporated into the model. Table 2 summarizes the comparison of generative adversarial nets with other generative modeling approaches.

与以往的建模框架相比，这个新框架有优点也有缺点。缺点主要有两个：一是没有 $p_g(\boldsymbol{x})$ 的显式表示；二是训练时 $D$ 必须与 $G$ 保持良好同步（尤其是，不能在不更新 $D$ 的情况下过多地训练 $G$，以免出现“Helvetica 情形”，即 $G$ 把太多不同的 $\boldsymbol{z}$ 坍缩到同一个 $\boldsymbol{x}$ 上，失去对 $p_{\text{data}}$ 建模所需的多样性），这就像玻尔兹曼机的负链必须在学习步之间保持更新一样。优点是：完全不需要马尔可夫链，只用反向传播就能得到梯度，学习过程中不需要推断，并且模型中可以纳入种类广泛的函数。表 2 总结了生成对抗网络与其他生成建模方法的比较。

The aforementioned advantages are primarily computational. Adversarial models may also gain some statistical advantage from the generator network not being updated directly with data examples, but only with gradients flowing through the discriminator. This means that components of the input are not copied directly into the generator’s parameters. Another advantage of adversarial networks is that they can represent very sharp, even degenerate distributions, while methods based on Markov chains require that the distribution be somewhat blurry in order for the chains to be able to mix between modes.

上述优点主要是计算上的。对抗模型可能还有一些统计上的优势：生成器网络不直接用数据样本更新，只用流经判别器的梯度来更新。这意味着输入的各个成分不会被直接复制到生成器的参数里。对抗网络的另一个优点是，它能表示非常尖锐甚至退化的分布；而基于马尔可夫链的方法要求分布有一定程度的模糊，链才能在各个模式之间混合。

7 Conclusions and future work

7 结论与未来工作

This framework admits many straightforward extensions:

1. A conditional generative model $p(\boldsymbol{x} \mid \boldsymbol{c})$ can be obtained by adding $\boldsymbol{c}$ as input to both $G$ and $D$.
2. Learned approximate inference can be performed by training an auxiliary network to predict $\boldsymbol{z}$ given $\boldsymbol{x}$. This is similar to the inference net trained by the wake-sleep algorithm [15] but with the advantage that the inference net may be trained for a fixed generator net after the generator net has finished training.
3. One can approximately model all conditionals $p(\boldsymbol{x}_S \mid \boldsymbol{x}_{\not S})$ where $S$ is a subset of the indices of $\boldsymbol{x}$ by training a family of conditional models that share parameters. Essentially, one can use adversarial nets to implement a stochastic extension of the deterministic MP-DBM [11].
4. Semi-supervised learning: features from the discriminator or inference net could improve performance of classifiers when limited labeled data is available.
5. Efficiency improvements: training could be accelerated greatly by divising better methods for coordinating $G$ and $D$ or determining better distributions to sample $\boldsymbol{z}$ from during training.

这个框架可以直接做许多扩展：

1. 把 $\boldsymbol{c}$ 同时作为 $G$ 和 $D$ 的输入，就能得到条件生成模型 $p(\boldsymbol{x} \mid \boldsymbol{c})$。
2. 训练一个辅助网络，在给定 $\boldsymbol{x}$ 时预测 $\boldsymbol{z}$，就能实现学习得到的近似推断。这与 wake-sleep 算法 [15] 训练的推断网络类似，但好处是可以在生成器网络训练完成后，针对固定的生成器网络来训练推断网络。
3. 训练一族共享参数的条件模型，就能近似地对所有条件分布 $p(\boldsymbol{x}_S \mid \boldsymbol{x}_{\not S})$ 建模，其中 $S$ 是 $\boldsymbol{x}$ 下标的一个子集。本质上，可以用对抗网络实现确定性 MP-DBM [11] 的随机扩展。
4. 半监督学习：在标注数据有限时，来自判别器或推断网络的特征可以提升分类器的性能。
5. 提高效率：设计更好的方法来协调 $G$ 和 $D$，或者为训练中采样 $\boldsymbol{z}$ 找到更好的分布，都可能大大加快训练。

This paper has demonstrated the viability of the adversarial modeling framework, suggesting that these research directions could prove useful.

本文证明了对抗建模框架的可行性，这表明上述研究方向可能会有用处。

Acknowledgments

致谢

We would like to acknowledge Patrice Marcotte, Olivier Delalleau, Kyunghyun Cho, Guillaume Alain and Jason Yosinski for helpful discussions. Yann Dauphin shared his Parzen window evaluation code with us. We would like to thank the developers of Pylearn2 [12] and Theano [7, 1], particularly Frédéric Bastien who rushed a Theano feature specifically to benefit this project. Arnaud Bergeron provided much-needed support with LaTeX typesetting. We would also like to thank CIFAR, and Canada Research Chairs for funding, and Compute Canada, and Calcul Québec for providing computational resources. Ian Goodfellow is supported by the 2013 Google Fellowship in Deep Learning. Finally, we would like to thank Les Trois Brasseurs for stimulating our creativity.

感谢 Patrice Marcotte、Olivier Delalleau、Kyunghyun Cho、Guillaume Alain 和 Jason Yosinski 提供的有益讨论。Yann Dauphin 与我们分享了他的 Parzen 窗口评估代码。感谢 Pylearn2 [12] 和 Theano [7, 1] 的开发者，特别是 Frédéric Bastien，他为这个项目赶工实现了一项 Theano 功能。Arnaud Bergeron 在 LaTeX 排版上给予了急需的支持。感谢 CIFAR 和加拿大研究讲席计划提供资助，感谢 Compute Canada 和 Calcul Québec 提供计算资源。Ian Goodfellow 获得 2013 年谷歌深度学习博士奖学金资助。最后，感谢 Les Trois Brasseurs 激发了我们的创造力。

References

参考文献

[1] Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I. J., Bergeron, A., Bouchard, N., and Bengio, Y. (2012). Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop.

[2] Bengio, Y. (2009). Learning deep architectures for AI. Now Publishers.

[3] Bengio, Y., Mesnil, G., Dauphin, Y., and Rifai, S. (2013a). Better mixing via deep representations. In ICML’13.

[4] Bengio, Y., Yao, L., Alain, G., and Vincent, P. (2013b). Generalized denoising auto-encoders as generative models. In NIPS26. Nips Foundation.

[5] Bengio, Y., Thibodeau-Laufer, E., and Yosinski, J. (2014a). Deep generative stochastic networks trainable by backprop. In ICML’14.

[6] Bengio, Y., Thibodeau-Laufer, E., Alain, G., and Yosinski, J. (2014b). Deep generative stochastic networks trainable by backprop. In Proceedings of the 30th International Conference on Machine Learning (ICML’14).

[7] Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., and Bengio, Y. (2010). Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy). Oral Presentation.

[8] Breuleux, O., Bengio, Y., and Vincent, P. (2011). Quickly generating representative samples from an RBM-derived process. Neural Computation, 23(8), 2053–2073.

[9] Glorot, X., Bordes, A., and Bengio, Y. (2011). Deep sparse rectifier neural networks. In AISTATS’2011.

[10] Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. (2013a). Maxout networks. In ICML’2013.

[11] Goodfellow, I. J., Mirza, M., Courville, A., and Bengio, Y. (2013b). Multi-prediction deep Boltzmann machines. In NIPS’2013.

[12] Goodfellow, I. J., Warde-Farley, D., Lamblin, P., Dumoulin, V., Mirza, M., Pascanu, R., Bergstra, J., Bastien, F., and Bengio, Y. (2013c). Pylearn2: a machine learning research library. arXiv preprint arXiv:1308.4214.

[13] Gutmann, M. and Hyvarinen, A. (2010). Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In AISTATS’2010.

[14] Hinton, G., Deng, L., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., and Kingsbury, B. (2012a). Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 29(6), 82–97.

[15] Hinton, G. E., Dayan, P., Frey, B. J., and Neal, R. M. (1995). The wake-sleep algorithm for unsupervised neural networks. Science, 268, 1558–1161.

[16] Hinton, G. E., Osindero, S., and Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527–1554.

[17] Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2012b). Improving neural networks by preventing co-adaptation of feature detectors. Technical report, arXiv:1207.0580.

[18] Hyvärinen, A. (2005). Estimation of non-normalized statistical models using score matching. J. Machine Learning Res., 6.

[19] Jarrett, K., Kavukcuoglu, K., Ranzato, M., and LeCun, Y. (2009). What is the best multi-stage architecture for object recognition? In Proc. International Conference on Computer Vision (ICCV’09), pages 2146–2153. IEEE.

[20] Kingma, D. P. and Welling, M. (2014). Auto-encoding variational bayes. In Proceedings of the International Conference on Learning Representations (ICLR).

[21] Krizhevsky, A. and Hinton, G. (2009). Learning multiple layers of features from tiny images. Technical report, University of Toronto.

[22] Krizhevsky, A., Sutskever, I., and Hinton, G. (2012). ImageNet classification with deep convolutional neural networks. In NIPS’2012.

[23] LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

[24] Rezende, D. J., Mohamed, S., and Wierstra, D. (2014). Stochastic backpropagation and approximate inference in deep generative models. Technical report, arXiv:1401.4082.

[25] Rifai, S., Bengio, Y., Dauphin, Y., and Vincent, P. (2012). A generative process for sampling contractive auto-encoders. In ICML’12.

[26] Salakhutdinov, R. and Hinton, G. E. (2009). Deep Boltzmann machines. In AISTATS’2009, pages 448–455.

[27] Smolensky, P. (1986). Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing, volume 1, chapter 6, pages 194–281. MIT Press, Cambridge.

[28] Susskind, J., Anderson, A., and Hinton, G. E. (2010). The Toronto face dataset. Technical Report UTML TR 2010-001, U. Toronto.

[29] Tieleman, T. (2008). Training restricted Boltzmann machines using approximations to the likelihood gradient. In W. W. Cohen, A. McCallum, and S. T. Roweis, editors, ICML 2008, pages 1064–1071. ACM.

[30] Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.-A. (2008). Extracting and composing robust features with denoising autoencoders. In ICML 2008.

[31] Younes, L. (1999). On the convergence of Markovian stochastic algorithms with rapidly decreasing ergodicity rates. Stochastics and Stochastic Reports, 65(3), 177–228.