Improved Training of Wasserstein GANs

Wasserstein GAN 的改进训练

Ishaan Gulrajani$^{1*}$, Faruk Ahmed$^1$, Martin Arjovsky$^2$, Vincent Dumoulin$^1$, Aaron Courville$^{1,3}$

$^1$ Montreal Institute for Learning Algorithms

$^2$ Courant Institute of Mathematical Sciences

$^3$ CIFAR Fellow

igul222@gmail.com
{faruk.ahmed,vincent.dumoulin,aaron.courville}@umontreal.ca
ma4371@nyu.edu

• Now at Google Brain

Code for our models is available at https://github.com/igul222/improved_wgan_training.

Abstract

摘要

Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only poor samples or fail to converge. We find that these problems are often due to the use of weight clipping in WGAN to enforce a Lipschitz constraint on the critic, which can lead to undesired behavior. We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models with continuous generators. We also achieve high quality generations on CIFAR-10 and LSUN bedrooms.

生成对抗网络（GAN）是一类很强的生成模型，但训练不稳定。不久前提出的 Wasserstein GAN（WGAN）在稳定训练上前进了一步，不过有时仍然只能给出很差的样本，或者根本不收敛。我们发现，这些问题常常来自 WGAN 用权重裁剪来强制 critic 满足 Lipschitz 约束，而这种做法会带来不希望看到的行为。我们提出一种替代裁剪的办法：惩罚 critic 相对于其输入的梯度范数。该方法优于标准 WGAN，几乎不用调超参数，就能稳定训练多种多样的 GAN 结构，包括 101 层残差网络，以及生成器连续的语言模型。我们还在 CIFAR-10 和 LSUN bedrooms 上得到了高质量的生成结果。

1 Introduction

1 引言

Generative Adversarial Networks (GANs) [9] are a powerful class of generative models that cast generative modeling as a game between two networks: a generator network produces synthetic data given some noise source and a discriminator network discriminates between the generator's output and true data. GANs can produce very visually appealing samples, but are often hard to train, and much of the recent work on the subject [23, 19, 2, 21] has been devoted to finding ways of stabilizing training. Despite this, consistently stable training of GANs remains an open problem.

生成对抗网络（GAN）[9] 把生成建模写成两个网络之间的博弈：生成器从噪声出发造出合成数据，判别器则区分生成器的输出和真实数据。GAN 可以给出看起来很好的样本，但往往很难训练。近期不少工作 [23, 19, 2, 21] 都在寻找稳定训练的办法。即便如此，怎样稳定地训练 GAN，仍然没有完全解决。

In particular, [1] provides an analysis of the convergence properties of the value function being optimized by GANs. Their proposed alternative, named Wasserstein GAN (WGAN) [2], leverages the Wasserstein distance to produce a value function which has better theoretical properties than the original. WGAN requires that the discriminator (called the critic in that work) must lie within the space of 1-Lipschitz functions, which the authors enforce through weight clipping.

特别是，[1] 分析了 GAN 所优化的价值函数的收敛性质。他们提出的替代方案叫 Wasserstein GAN（WGAN）[2]，用 Wasserstein 距离构造出一个理论上比原来更好的价值函数。WGAN 要求判别器（该文称作 critic）必须落在 1-Lipschitz 函数空间里，原作者用权重裁剪来实现这一点。

Our contributions are as follows: (1) On toy datasets, we demonstrate how critic weight clipping can lead to undesired behavior. (2) We propose gradient penalty (WGAN-GP), which does not suffer from the same problems. (3) We demonstrate stable training of varied GAN architectures, performance improvements over weight clipping, high-quality image generation, and a character-level GAN language model without any discrete sampling.

本文的贡献如下：（1）在玩具数据上说明，对 critic 做权重裁剪会带来哪些不希望看到的行为。（2）提出梯度惩罚（WGAN-GP），它没有同样的问题。（3）展示了多种 GAN 结构的稳定训练、相对权重裁剪的性能提升、高质量图像生成，以及不需要任何离散采样的字符级 GAN 语言模型。

2 Background

2 背景

2.1 Generative adversarial networks

2.1 生成对抗网络

The GAN training strategy is to define a game between two competing networks. The generator network maps a source of noise to the input space. The discriminator network receives either a generated sample or a true data sample and must distinguish between the two. The generator is trained to fool the discriminator.

GAN 的训练策略是定义两个相互竞争的网络之间的博弈。生成器把噪声映射到输入空间；判别器拿到的要么是生成样本，要么是真实样本，它必须把两者区分开。生成器则被训练去骗过判别器。

Formally, the game between the generator $G$ and the discriminator $D$ is the minimax objective:

形式上，生成器 $G$ 与判别器 $D$ 之间的博弈是如下极小极大目标：

$$\min_G \max_D \mathbb{E}_{x\sim\mathbb{P}_r}[\log(D(x))] + \mathbb{E}_{\tilde{x}\sim\mathbb{P}_g}[\log(1-D(\tilde{x}))]\qquad(1)$$

where $\mathbb{P}_r$ is the data distribution and $\mathbb{P}_g$ is the model distribution implicitly defined by $\tilde{x}=G(z)$, $z\sim p(z)$ (the input $z$ to the generator is sampled from some simple noise distribution $p$, such as the uniform distribution or a spherical Gaussian distribution).

其中 $\mathbb{P}_r$ 是数据分布，$\mathbb{P}_g$ 是由 $\tilde{x}=G(z)$、$z\sim p(z)$ 隐式定义的模型分布（生成器的输入 $z$ 从某种简单噪声分布 $p$ 中采样，例如均匀分布或球面高斯分布）。

If the discriminator is trained to optimality before each generator parameter update, then minimizing the value function amounts to minimizing the Jensen-Shannon divergence between $\mathbb{P}_r$ and $\mathbb{P}_g$ [9], but doing so often leads to vanishing gradients as the discriminator saturates. In practice, [9] advocates that the generator be instead trained to maximize $\mathbb{E}_{\tilde{x}\sim\mathbb{P}_g}[\log(D(\tilde{x}))]$, which goes some way to circumvent this difficulty. However, even this modified loss function can misbehave in the presence of a good discriminator [1].

如果在每次更新生成器参数之前都把判别器训练到最优，那么最小化该价值函数就相当于最小化 $\mathbb{P}_r$ 与 $\mathbb{P}_g$ 之间的 Jensen–Shannon 散度 [9]；但判别器饱和时，这样做常常会让梯度消失。实践中，[9] 主张改为让生成器最大化 $\mathbb{E}_{\tilde{x}\sim\mathbb{P}_g}[\log(D(\tilde{x}))]$，这能部分绕开上述困难。不过，即便改过的损失函数，在判别器很强时仍然可能表现失常 [1]。

2.2 Wasserstein GANs

2.2 Wasserstein GAN

[2] argues that the divergences which GANs typically minimize are potentially not continuous with respect to the generator's parameters, leading to training difficulty. They propose instead using the Earth-Mover (also called Wasserstein-1) distance $W(q,p)$, which is informally defined as the minimum cost of transporting mass in order to transform the distribution $q$ into the distribution $p$ (where the cost is mass times transport distance). Under mild assumptions, $W(q,p)$ is continuous everywhere and differentiable almost everywhere.

[2] 指出，GAN 通常最小化的那些散度，对生成器参数未必连续，因而会带来训练困难。他们转而使用推土机距离（也叫 Wasserstein-1 距离）$W(q,p)$：直观上，它是把分布 $q$ 搬成分布 $p$ 所需的最小运费（费用是质量乘以搬运距离）。在温和的假定下，$W(q,p)$ 处处连续，并且几乎处处可微。

The WGAN value function is constructed using the Kantorovich-Rubinstein duality [25] to obtain

WGAN 的价值函数借助 Kantorovich–Rubinstein 对偶 [25] 写成

$$\min_G \max_{D\in\mathcal{D}} \mathbb{E}_{x\sim\mathbb{P}_r}[D(x)] - \mathbb{E}_{\tilde{x}\sim\mathbb{P}_g}[D(\tilde{x})]\qquad(2)$$

where $\mathcal{D}$ is the set of 1-Lipschitz functions and $\mathbb{P}_g$ is once again the model distribution implicitly defined by $\tilde{x}=G(z)$, $z\sim p(z)$. In that case, under an optimal discriminator (called a critic in the paper, since it's not trained to classify), minimizing the value function with respect to the generator parameters minimizes $W(\mathbb{P}_r,\mathbb{P}_g)$.

其中 $\mathcal{D}$ 是 1-Lipschitz 函数组成的集合，$\mathbb{P}_g$ 仍然由 $\tilde{x}=G(z)$、$z\sim p(z)$ 隐式定义。此时，若判别器（该文称作 critic，因为它并不是在做分类）达到最优，再对生成器参数最小化该价值函数，就相当于最小化 $W(\mathbb{P}_r,\mathbb{P}_g)$。

The WGAN value function results in a critic function whose gradient with respect to its input is better behaved than its GAN counterpart, making optimization of the generator easier. Empirically, it was also observed that the WGAN value function appears to correlate with sample quality, which is not the case for GANs [2]. To enforce the Lipschitz constraint on the critic, [2] propose to clip the weights of the critic to lie within a compact space $[-c,c]$. The set of functions satisfying this constraint is a subset of the $k$-Lipschitz functions for some $k$ which depends on $c$ and the critic architecture. In the following sections, we demonstrate some of the issues with this approach and propose an alternative.

WGAN 的价值函数会给出一个对输入的梯度更规整的 critic，因而生成器更好优化。经验上还可以看到，WGAN 的价值函数似乎与样本质量相关，这一点 GAN 并不具备 [2]。为了强制 critic 满足 Lipschitz 约束，[2] 建议把 critic 的权重量裁剪到紧致区间 $[-c,c]$ 内。满足该约束的函数集，是某个 $k$-Lipschitz 函数类的子集，$k$ 取决于 $c$ 和 critic 的结构。后文我们说明这种做法的若干问题，并给出替代方案。

2.3 Properties of the optimal WGAN critic

2.3 最优 WGAN critic 的性质

In order to understand why weight clipping is problematic in a WGAN critic, as well as to motivate our approach, we highlight some properties of the optimal critic in the WGAN framework. We prove these in the Appendix.

为了理解权重裁剪在 WGAN 的 critic 里为什么会出问题，也为后文的方法做铺垫，我们先列出 WGAN 框架下最优 critic 的若干性质。证明放在附录里。

Proposition 1. Let $\mathbb{P}_r$ and $\mathbb{P}_g$ be two distributions in $\mathcal{X}$, a compact metric space. Then, there is a 1-Lipschitz function $f^*$ which is the optimal solution of $\max_{\|f\|_L\le 1}\mathbb{E}_{y\sim\mathbb{P}_r}[f(y)]-\mathbb{E}_{x\sim\mathbb{P}_g}[f(x)]$. Let $\pi$ be the optimal coupling between $\mathbb{P}_r$ and $\mathbb{P}_g$, defined as the minimizer of $W(\mathbb{P}_r,\mathbb{P}_g)=\inf_{\pi\in\Pi(\mathbb{P}_r,\mathbb{P}_g)}\mathbb{E}_{(x,y)\sim\pi}[\|x-y\|]$, where $\Pi(\mathbb{P}_r,\mathbb{P}_g)$ is the set of joint distributions $\pi(x,y)$ whose marginals are $\mathbb{P}_r$ and $\mathbb{P}_g$, respectively. Then, if $f^*$ is differentiable, $\pi(x=y)=0$, and $x_t=tx+(1-t)y$ with $0\le t\le 1$, it holds that $\mathbb{P}_{(x,y)\sim\pi}\big[\nabla f^*(x_t)=(y-x_t)/\|y-x_t\|\big]=1$.

命题 1. 设 $\mathbb{P}_r$ 与 $\mathbb{P}_g$ 是紧致度量空间 $\mathcal{X}$ 上的两个分布。则存在 1-Lipschitz 函数 $f^*$，它是 $\max_{\|f\|_L\le 1}\mathbb{E}_{y\sim\mathbb{P}_r}[f(y)]-\mathbb{E}_{x\sim\mathbb{P}_g}[f(x)]$ 的最优解。设 $\pi$ 是 $\mathbb{P}_r$ 与 $\mathbb{P}_g$ 之间的最优耦合，即 $W(\mathbb{P}_r,\mathbb{P}_g)=\inf_{\pi\in\Pi(\mathbb{P}_r,\mathbb{P}_g)}\mathbb{E}_{(x,y)\sim\pi}[\|x-y\|]$ 的最小化者，其中 $\Pi(\mathbb{P}_r,\mathbb{P}_g)$ 是边际分别为 $\mathbb{P}_r$ 与 $\mathbb{P}_g$ 的联合分布 $\pi(x,y)$ 的集合。若 $f^*$ 可微，$\pi(x=y)=0$，并记 $x_t=tx+(1-t)y$（$0\le t\le 1$），则有 $\mathbb{P}_{(x,y)\sim\pi}\big[\nabla f^*(x_t)=(y-x_t)/\|y-x_t\|\big]=1$。

Corollary 1. $f^*$ has gradient norm 1 almost everywhere under $\mathbb{P}_r$ and $\mathbb{P}_g$.

推论 1. 在 $\mathbb{P}_r$ 与 $\mathbb{P}_g$ 下，$f^*$ 的梯度范数几乎处处为 1。

3 Difficulties with weight constraints

3 权重约束带来的困难

We find that weight clipping in WGAN leads to optimization difficulties, and that even when optimization succeeds the resulting critic can have a pathological value surface. We explain these problems below and demonstrate their effects; however we do not claim that each one always occurs in practice, nor that they are the only such mechanisms.

我们发现，WGAN 里的权重裁剪会带来优化上的困难；即便优化成功，得到的 critic 价值面也可能病态。下面解释这些问题并展示其效果。我们并不声称它们在实践中总会发生，也不认为这是仅有的机制。

Our experiments use the specific form of weight constraint from [2] (hard clipping of the magnitude of each weight), but we also tried other weight constraints (L2 norm clipping, weight normalization), as well as soft constraints (L1 and L2 weight decay) and found that they exhibit similar problems. To some extent these problems can be mitigated with batch normalization in the critic, which [2] use in all of their experiments. However even with batch normalization, we observe that very deep WGAN critics often fail to converge.

实验里我们采用 [2] 那种具体的权重约束（把每个权重的幅度硬裁剪）。我们也试过别的权重约束（L2 范数裁剪、权重归一化）以及软约束（L1 和 L2 权重衰减），发现它们有类似的问题。若在 critic 里使用批归一化，这些问题可以减轻一些，[2] 的全部实验都用了批归一化。不过即便加上批归一化，很深的 WGAN critic 也常常不能收敛。

Figure 1: Gradient penalty in WGANs does not exhibit undesired behavior like weight clipping. (a) Value surfaces of WGAN critics trained to optimality on toy datasets using (top) weight clipping and (bottom) gradient penalty. Critics trained with weight clipping fail to capture higher moments of the data distribution. The 'generator' is held fixed at the real data plus Gaussian noise. (b) (left) Gradient norms of deep WGAN critics during training on the Swiss Roll dataset either explode or vanish when using weight clipping, but not when using a gradient penalty. (right) Weight clipping (top) pushes weights towards two values (the extremes of the clipping range), unlike gradient penalty (bottom).

图 1：WGAN 中的梯度惩罚不会表现出权重裁剪那样不受欢迎的行为。（a）在玩具数据上把 WGAN critic 训到最优后的价值面：上行是权重裁剪，下行是梯度惩罚。用权重裁剪训练的 critic 抓不住数据分布的高阶矩。这里的“生成器”固定为真实数据加高斯噪声。（b）（左）在 Swiss Roll 上训练深层 WGAN critic 时，权重裁剪会让梯度范数爆炸或消失，梯度惩罚则不会。（右）权重裁剪（上）会把权重推向两个值（裁剪区间的两端），梯度惩罚（下）则不会。

3.1 Capacity underuse

3.1 容量没有被用上

Implementing a $k$-Lipschitz constraint via weight clipping biases the critic towards much simpler functions. As stated previously in Corollary 1, the optimal WGAN critic has unit gradient norm almost everywhere under $\mathbb{P}_r$ and $\mathbb{P}_g$; under a weight-clipping constraint, we observe that our neural network architectures which try to attain their maximum gradient norm $k$ end up learning extremely simple functions.

用权重裁剪来实现 $k$-Lipschitz 约束，会把 critic 偏向简单得多的函数。推论 1 已经说明，最优 WGAN critic 在 $\mathbb{P}_r$ 与 $\mathbb{P}_g$ 下几乎处处具有单位梯度范数；加上权重裁剪之后，我们观察到：网络为了摸到允许的最大梯度范数 $k$，最后学到的往往是极其简单的函数。

To demonstrate this, we train WGAN critics with weight clipping to optimality on several toy distributions, holding the generator distribution $\mathbb{P}_g$ fixed at the real distribution plus unit-variance Gaussian noise. We plot value surfaces of the critics in Figure 1a. We omit batch normalization in the critic. In each case, the critic trained with weight clipping ignores higher moments of the data distribution and instead models very simple approximations to the optimal functions. In contrast, our approach does not suffer from this behavior.

为说明这一点，我们在若干玩具分布上把带权重裁剪的 WGAN critic 训到最优，同时把生成分布 $\mathbb{P}_g$ 固定为真实分布加单位方差高斯噪声。图 1a 画出了这些 critic 的价值面。critic 里不用批归一化。每一种情形下，用权重裁剪训练的 critic 都忽略了数据分布的高阶矩，只学到对最优函数非常简单的近似。我们的方法没有这种表现。

3.2 Exploding and vanishing gradients

3.2 梯度爆炸与消失

We observe that the WGAN optimization process is difficult because of interactions between the weight constraint and the cost function, which result in either vanishing or exploding gradients without careful tuning of the clipping threshold $c$. To demonstrate this, we train WGAN on the Swiss Roll toy dataset, varying the clipping threshold $c$ in $[10^{-1},10^{-2},10^{-3}]$, and plot the norm of the gradient of the critic loss with respect to successive layers of activations. Both generator and critic are 12-layer ReLU MLPs without batch normalization. Figure 1b shows that for each of these values, the gradient either grows or decays exponentially as we move farther back in the network. We find our method results in more stable gradients that neither vanish nor explode, allowing training of more complicated networks.

我们观察到，WGAN 的优化之所以困难，是因为权重约束和损失函数会相互作用：若不仔细调节裁剪阈值 $c$，梯度就会消失或爆炸。为说明这一点，我们在 Swiss Roll 玩具数据上训练 WGAN，把 $c$ 取成 $[10^{-1},10^{-2},10^{-3}]$，并画出 critic 损失对逐层激活的梯度范数。生成器和 critic 都是 12 层 ReLU 多层感知机，不用批归一化。图 1b 显示，对每一个取值，越往网络前面走，梯度要么指数增大，要么指数衰减。我们的方法给出更稳定、既不消失也不爆炸的梯度，因而可以训练更复杂的网络。

4 Gradient penalty

4 梯度惩罚

Algorithm 1 WGAN with gradient penalty. We use default values of $\lambda=10$, $n_{\mathrm{critic}}=5$, $\alpha=0.0001$, $\beta_1=0$, $\beta_2=0.9$.

算法 1：带梯度惩罚的 WGAN。默认取 $\lambda=10$，$n_{\mathrm{critic}}=5$，$\alpha=0.0001$，$\beta_1=0$，$\beta_2=0.9$。

Require: The gradient penalty coefficient $\lambda$, the number of critic iterations per generator iteration $n_{\mathrm{critic}}$, the batch size $m$, Adam hyperparameters $\alpha,\beta_1,\beta_2$.

Require: initial critic parameters $w_0$, initial generator parameters $\theta_0$.

1: while $\theta$ has not converged do

2:   for $t=1,\ldots,n_{\mathrm{critic}}$ do

3:     for $i=1,\ldots,m$ do

4:       Sample real data $x\sim\mathbb{P}_r$, latent variable $z\sim p(z)$, a random number $\epsilon\sim U[0,1]$.

5:       $\tilde{x}\leftarrow G_\theta(z)$

6:       $\hat{x}\leftarrow \epsilon x+(1-\epsilon)\tilde{x}$

7:       $L^{(i)}\leftarrow D_w(\tilde{x})-D_w(x)+\lambda(\|\nabla_{\hat{x}}D_w(\hat{x})\|_2-1)^2$

8:     end for

9:     $w\leftarrow \mathrm{Adam}(\nabla_w \frac{1}{m}\sum_{i=1}^m L^{(i)}, w, \alpha,\beta_1,\beta_2)$

10:   end for

11:   Sample a batch of latent variables $\{z^{(i)}\}_{i=1}^m\sim p(z)$.

12:   $\theta\leftarrow \mathrm{Adam}(\nabla_\theta \frac{1}{m}\sum_{i=1}^m -D_w(G_\theta(z^{(i)})), \theta, \alpha,\beta_1,\beta_2)$

13: end while

需要： 梯度惩罚系数 $\lambda$，每次生成器迭代对应的 critic 迭代次数 $n_{\mathrm{critic}}$，批量大小 $m$，Adam 超参数 $\alpha,\beta_1,\beta_2$。

需要： critic 的初始参数 $w_0$，生成器的初始参数 $\theta_0$。

1: while $\theta$ 尚未收敛 do

2:   for $t=1,\ldots,n_{\mathrm{critic}}$ do

3:     for $i=1,\ldots,m$ do

4:       采样真实数据 $x\sim\mathbb{P}_r$、隐变量 $z\sim p(z)$，以及随机数 $\epsilon\sim U[0,1]$。

5:       $\tilde{x}\leftarrow G_\theta(z)$

6:       $\hat{x}\leftarrow \epsilon x+(1-\epsilon)\tilde{x}$

7:       $L^{(i)}\leftarrow D_w(\tilde{x})-D_w(x)+\lambda(\|\nabla_{\hat{x}}D_w(\hat{x})\|_2-1)^2$

8:     end for

9:     $w\leftarrow \mathrm{Adam}(\nabla_w \frac{1}{m}\sum_{i=1}^m L^{(i)}, w, \alpha,\beta_1,\beta_2)$

10:   end for

11:   采样一批隐变量 $\{z^{(i)}\}_{i=1}^m\sim p(z)$。

12:   $\theta\leftarrow \mathrm{Adam}(\nabla_\theta \frac{1}{m}\sum_{i=1}^m -D_w(G_\theta(z^{(i)})), \theta, \alpha,\beta_1,\beta_2)$

13: end while

We now propose an alternative way to enforce the Lipschitz constraint. A differentiable function is 1-Lipschitz if and only if it has gradients with norm at most 1 everywhere, so we consider directly constraining the gradient norm of the critic's output with respect to its input. To circumvent tractability issues, we enforce a soft version of the constraint with a penalty on the gradient norm for random samples $\hat{x}\sim\mathbb{P}_{\hat{x}}$. Our new objective is

我们提出另一种施加 Lipschitz 约束的办法。可微函数是 1-Lipschitz 的，当且仅当它处处梯度范数至多为 1，因此我们直接约束 critic 输出对其输入的梯度范数。为了避开不可行的计算负担，我们只对随机样本 $\hat{x}\sim\mathbb{P}_{\hat{x}}$ 的梯度范数做软惩罚。新的目标是

$$L=\mathbb{E}_{\tilde{x}\sim\mathbb{P}_g}[D(\tilde{x})]-\mathbb{E}_{x\sim\mathbb{P}_r}[D(x)]+\lambda\,\mathbb{E}_{\hat{x}\sim\mathbb{P}_{\hat{x}}}\big[(\|\nabla_{\hat{x}}D(\hat{x})\|_2-1)^2\big]\qquad(3)$$

The first two terms are the original critic loss; the last term is our gradient penalty.

前两项是原来的 critic 损失；最后一项是我们的梯度惩罚。

Sampling distribution. We implicitly define $\mathbb{P}_{\hat{x}}$ sampling uniformly along straight lines between pairs of points sampled from the data distribution $\mathbb{P}_r$ and the generator distribution $\mathbb{P}_g$. This is motivated by the fact that the optimal critic contains straight lines with gradient norm 1 connecting coupled points from $\mathbb{P}_r$ and $\mathbb{P}_g$ (see Proposition 1). Given that enforcing the unit gradient norm constraint everywhere is intractable, enforcing it only along these straight lines seems sufficient and experimentally results in good performance.

采样分布。 我们隐式定义 $\mathbb{P}_{\hat{x}}$：在分别来自数据分布 $\mathbb{P}_r$ 和生成分布 $\mathbb{P}_g$ 的成对点之间的直线上均匀采样。动机是最优 critic 在 $\mathbb{P}_r$ 与 $\mathbb{P}_g$ 的耦合点之间含有梯度范数为 1 的直线（见命题 1）。既然无法在各处都施加单位梯度范数约束，只沿这些直线施加看起来就够了，实验效果也好。

Penalty coefficient. All experiments in this paper use $\lambda=10$, which we found to work well across a variety of architectures and datasets ranging from toy tasks to large ImageNet CNNs.

惩罚系数。 本文全部实验取 $\lambda=10$。我们发现，从玩具任务到较大的 ImageNet CNN，这一取值在多种结构和数据上都好用。

No critic batch normalization. Most prior GAN implementations [22, 23, 2] use batch normalization in both the generator and the discriminator to help stabilize training, but batch normalization changes the form of the discriminator's problem from mapping a single input to a single output to mapping from an entire batch of inputs to a batch of outputs [23]. Our penalized training objective is no longer valid in this setting, since we penalize the norm of the critic's gradient with respect to each input independently, and not the entire batch. To resolve this, we simply omit batch normalization in the critic in our models, finding that they perform well without it. Our method works with normalization schemes which don't introduce correlations between examples. In particular, we recommend layer normalization [3] as a drop-in replacement for batch normalization.

critic 不用批归一化。 以往大多数 GAN 实现 [22, 23, 2] 会在生成器和判别器里都用批归一化来稳定训练，但批归一化会把判别器的问题从“一个输入映射到一个输出”变成“一整批输入映射到一批输出”[23]。加上惩罚之后，目标在这种设定下不再成立，因为我们惩罚的是 critic 对每个输入各自的梯度范数，而不是对整个 batch。解决办法很简单：我们的模型在 critic 里去掉批归一化，发现没有它也能训得很好。不在样本之间引入相关的归一化仍然可用。我们特别建议用层归一化 [3] 直接替换批归一化。

Two-sided penalty. We encourage the norm of the gradient to go towards 1 (two-sided penalty) instead of just staying below 1 (one-sided penalty). Empirically this seems not to constrain the critic too much, likely because the optimal WGAN critic anyway has gradients with norm 1 almost everywhere under $\mathbb{P}_r$ and $\mathbb{P}_g$ and in large portions of the region in between (see subsection 2.3). In our early observations we found this to perform slightly better, but we don't investigate this fully. We describe experiments on the one-sided penalty in the appendix.

双侧惩罚。 我们鼓励梯度范数靠近 1（双侧惩罚），而不是仅仅让它不超过 1（单侧惩罚）。经验上看，这似乎没有把 critic 限制得过死，大概是因为最优 WGAN critic 在 $\mathbb{P}_r$ 与 $\mathbb{P}_g$ 下几乎处处、并在两者之间的很大一片区域里，梯度范数本就是 1（见 2.3 节）。我们较早的观察里，双侧略好一些，但没有做完整比较。单侧惩罚的实验写在附录里。

5 Experiments

5 实验

5.1 Training random architectures within a set

5.1 在一组结构里随机抽样并训练

We experimentally demonstrate our model's ability to train a large number of architectures which we think are useful to be able to train. Starting from the DCGAN architecture, we define a set of architecture variants by changing model settings to random corresponding values in Table 1. We believe that reliable training of many of the architectures in this set is a useful goal, but we do not claim that our set is an unbiased or representative sample of the whole space of useful architectures: it is designed to demonstrate a successful regime of our method, and readers should evaluate whether it contains architectures similar to their intended application.

我们用实验说明：有一大批我们认为值得训得起来的结构，现在可以训起来。从 DCGAN 结构出发，按表 1 把模型设置随机改成对应取值，得到一组变体。我们认为，能可靠地训练其中许多结构，本身就是有用的目标；但并不声称这组结构是全部有用结构空间里无偏或有代表性的样本：它用来展示本方法能够成功的范围，读者应自行判断其中是否包含与自己应用相近的结构。

Table 1: We evaluate WGAN-GP's ability to train the architectures in this set.

Setting	Values
Nonlinearity (G)	[ReLU, LeakyReLU, $\mathrm{softplus}(2x+2)/2-1$, tanh]
Nonlinearity (D)	[ReLU, LeakyReLU, $\mathrm{softplus}(2x+2)/2-1$, tanh]
Depth (G)	[4, 8, 12, 20]
Depth (D)	[4, 8, 12, 20]
Batch norm (G)	[True, False]
Batch norm (D; layer norm for WGAN-GP)	[True, False]
Base filter count (G)	[32, 64, 128]
Base filter count (D)	[32, 64, 128]

表 1：我们评估 WGAN-GP 训练这组结构的能力。

设置	取值
非线性（G）	[ReLU, LeakyReLU, $\mathrm{softplus}(2x+2)/2-1$, tanh]
非线性（D）	[ReLU, LeakyReLU, $\mathrm{softplus}(2x+2)/2-1$, tanh]
深度（G）	[4, 8, 12, 20]
深度（D）	[4, 8, 12, 20]
批归一化（G）	[True, False]
批归一化（D；WGAN-GP 则用层归一化）	[True, False]
基础卷积通道数（G）	[32, 64, 128]
基础卷积通道数（D）	[32, 64, 128]

From this set, we sample 200 architectures and train each on $32\times 32$ ImageNet with both WGAN-GP and the standard GAN objectives. Table 2 lists the number of instances where either: only the standard GAN succeeded, only WGAN-GP succeeded, both succeeded, or both failed, where success is defined as inception score $>$ min score. For most choices of score threshold, WGAN-GP successfully trains many architectures from this set which we were unable to train with the standard GAN objective. We give more experimental details in the appendix.

我们从这组里抽出 200 个结构，各自在 $32\times 32$ ImageNet 上用 WGAN-GP 和标准 GAN 目标各训一遍。表 2 统计了四种情形的个数：只有标准 GAN 成功、只有 WGAN-GP 成功、两者都成功、两者都失败；成功定义为 Inception score 大于某个最低分数。对大多数分数阈值，WGAN-GP 都能训起许多标准 GAN 训不起来的结构。更多实验细节见附录。

Table 2: Outcomes of training 200 random architectures, for different success thresholds. For comparison, our standard DCGAN scored 7.24.

Min. score	Only GAN	Only WGAN-GP	Both succeeded	Both failed
1.0	0	8	192	0
3.0	1	88	110	1
5.0	0	147	42	11
7.0	1	104	5	90
9.0	0	0	0	200

表 2：在不同成功阈值下，训练 200 个随机结构的结果。对照用的标准 DCGAN 分数为 7.24。

最低分数	仅 GAN	仅 WGAN-GP	两者都成功	两者都失败
1.0	0	8	192	0
3.0	1	88	110	1
5.0	0	147	42	11
7.0	1	104	5	90
9.0	0	0	0	200

5.2 Training varied architectures on LSUN bedrooms

5.2 在 LSUN bedrooms 上训练多种结构

One of the most unique practical advantages of WGAN-GP compared to other objectives is its ability to train many architectures with almost no hyperparameter tuning. To demonstrate this, we trained the following architecture variants by changing model settings to random corresponding values relative to a DCGAN baseline (see section 4). We train each model for 200K iterations and present samples in Figure 2. We only succeeded in training every architecture with a shared set of hyperparameters using WGAN-GP.

与其他目标相比，WGAN-GP 最独特的实际优点之一，是几乎不用调超参数就能训练许多结构。为说明这一点，我们相对 DCGAN 基线改动模型设置，训练了下列结构变体（见第 4 节）。每个模型训练 20 万次迭代，样本见图 2。只有用 WGAN-GP，我们才能在同一套超参下把每一种结构都训起来。

Figure 2: Different GAN architectures trained with different methods. We only succeeded in training every architecture with a shared set of hyperparameters using WGAN-GP.

图 2：用不同方法训练的多种 GAN 结构。只有用 WGAN-GP，我们才能在同一套超参下把每一种结构都训起来。

5.3 Improved performance over weight clipping

5.3 相对权重裁剪的性能提升

We investigate whether our method actually improves over the weight clipping WGAN in terms of sample quality and training speed. We train WGANs with weight clipping and gradient penalty on CIFAR-10 [13] and plot Inception scores [23] over the course of training in Figure 3. For WGAN-GP, we train one model with the same optimizer (RMSProp) and learning rate as WGAN with weight clipping, and one model with Adam. WGAN-GP significantly outperforms weight clipping and performs comparably to DCGAN.

我们考察该方法在样本质量和训练速度上是否真的优于权重裁剪的 WGAN。在 CIFAR-10 [13] 上分别用权重裁剪和梯度惩罚训练 WGAN，并在图 3 中画出训练过程中的 Inception score [23]。对 WGAN-GP，我们训了两个模型：一个与带权重裁剪的 WGAN 使用相同的优化器（RMSProp）和学习率，另一个使用 Adam。WGAN-GP 明显优于权重裁剪，并与 DCGAN 表现相当。

Figure 3: CIFAR-10 Inception score over generator iterations (left) or wall-clock time (right) for four models: WGAN with weight clipping, WGAN-GP with RMSProp and Adam (to control for the optimizer), and DCGAN. WGAN-GP significantly outperforms weight clipping and performs comparably to DCGAN.

图 3：四个模型在 CIFAR-10 上的 Inception score，横轴分别是生成器迭代次数（左）和墙钟时间（右）：带权重裁剪的 WGAN、分别用 RMSProp 和 Adam 的 WGAN-GP（用来控制优化器差异），以及 DCGAN。WGAN-GP 明显优于权重裁剪，并与 DCGAN 表现相当。

5.4 Sample quality on CIFAR-10 and LSUN bedrooms

5.4 在 CIFAR-10 和 LSUN bedrooms 上的样本质量

For equivalent architectures, our method achieves comparable sample quality to the standard GAN objective. However the increased stability allows us to improve sample quality by exploring a wider range of architectures. To demonstrate this, we find an architecture which establishes a new state of the art Inception score on unsupervised CIFAR-10 (Table 3). When we add label information (using the method in [20]), the same architecture outperforms all other published models except for SGAN.

在结构相当的情况下，本方法的样本质量与标准 GAN 目标相当。不过稳定性提高之后，我们可以探索更广的结构范围，从而进一步抬高样本质量。为说明这一点，我们找到一种结构，在无监督 CIFAR-10 上刷新了当时的 Inception score（表 3）。若再加入标签信息（采用 [20] 的方法），同一结构超过了除 SGAN 以外所有已发表模型。

Table 3: Inception scores on CIFAR-10. Our unsupervised model achieves state-of-the-art performance, and our conditional model outperforms all others except SGAN.

Method (unsupervised)	Score	Method (supervised)	Score
ALI [8] (in [27])	$5.34\pm .05$	SteinGAN [26]	6.35
BEGAN [4]	5.62	DCGAN (with labels, in [26])	6.58
DCGAN [22] (in [11])	$6.16\pm .07$	Improved GAN [23]	$8.09\pm .07$
Improved GAN (-L+HA) [23]	$6.86\pm .06$	AC-GAN [20]	$8.25\pm .07$
EGAN-Ent-VI [7]	$7.07\pm .10$	SGAN-no-joint [11]	$8.37\pm .08$
DFM [27]	$7.72\pm .13$	WGAN-GP ResNet (ours)	$8.42\pm .10$
WGAN-GP ResNet (ours)	$7.86\pm .07$	SGAN [11]	$8.59\pm .12$

表 3：CIFAR-10 上的 Inception score。我们的无监督模型达到当时最好成绩，有监督（条件）模型超过除 SGAN 外的所有其他模型。

方法（无监督）	分数	方法（有监督）	分数
ALI [8]（见 [27]）	$5.34\pm .05$	SteinGAN [26]	6.35
BEGAN [4]	5.62	DCGAN（带标签，见 [26]）	6.58
DCGAN [22]（见 [11]）	$6.16\pm .07$	Improved GAN [23]	$8.09\pm .07$
Improved GAN (-L+HA) [23]	$6.86\pm .06$	AC-GAN [20]	$8.25\pm .07$
EGAN-Ent-VI [7]	$7.07\pm .10$	SGAN-no-joint [11]	$8.37\pm .08$
DFM [27]	$7.72\pm .13$	WGAN-GP ResNet（本文）	$8.42\pm .10$
WGAN-GP ResNet（本文）	$7.86\pm .07$	SGAN [11]	$8.59\pm .12$

We also train a deep ResNet on $128\times 128$ LSUN bedrooms and show samples in Figure 4. We believe these samples are at least competitive with the best reported so far on any resolution for this dataset.

我们还在 $128\times 128$ 的 LSUN bedrooms 上训练了深层残差网络，样本见图 4。我们认为这些样本至少不逊于该数据集上当时在任何分辨率下报告过的最好结果。

Figure 4: Samples of $128\times 128$ LSUN bedrooms. We believe these samples are at least comparable to the best published results so far.

图 4：$128\times 128$ LSUN bedrooms 的生成样本。我们认为这些样本至少不逊于当时已发表的最好结果。

5.5 Modeling discrete data with a continuous generator

5.5 用连续生成器对离散数据建模

To demonstrate our method's ability to model degenerate distributions, we consider the problem of modeling a complex discrete distribution with a GAN whose generator is defined over a continuous space. As an instance of this problem, we train a character-level GAN language model on the Google Billion Word dataset [6]. Our generator is a simple 1D CNN which deterministically transforms a latent vector into a sequence of 32 one-hot character vectors through 1D convolutions. We apply a softmax nonlinearity at the output, but use no sampling step: during training, the softmax output is passed directly into the critic (which, likewise, is a simple 1D CNN). When decoding samples, we just take the argmax of each output vector.

为说明本方法也能对退化分布建模，我们考虑这样一个问题：生成器定义在连续空间上，却要去拟合复杂的离散分布。作为一个实例，我们在 Google Billion Word 数据集 [6] 上训练字符级 GAN 语言模型。生成器是简单的一维卷积网络，通过一维卷积把隐向量确定性映射成 32 个 one-hot 字符向量。输出端使用 softmax 非线性，但不做采样：训练时把 softmax 输出直接送进 critic（critic 同样是简单的一维卷积网络）。解码样本时，对每个输出向量取 argmax 即可。

We present samples from the model in Table 4. Our model makes frequent spelling errors (likely because it has to output each character independently) but nonetheless manages to learn quite a lot about the statistics of language. We were unable to produce comparable results with the standard GAN objective, though we do not claim that doing so is impossible.

表 4 给出模型样本。该模型经常出现拼写错误（多半是因为它必须独立输出每一个字符），但仍然学到了不少语言统计规律。我们没能用标准 GAN 目标得到可比较的结果，不过并不声称那一定做不到。

Table 4: Samples from a WGAN-GP character-level language model trained on sentences from the Billion Word dataset, truncated to 32 characters. The model learns to directly output one-hot character embeddings from a latent vector without any discrete sampling step. We were unable to achieve comparable results with the standard GAN objective and a continuous generator.

Sample	Sample
Busino game camperate spent odea	Solice Norkedin pring in since
In the bankaway of smarling the	ThiS record ( 31. ) UBS ) and Ch
SingersMay , who kill that imvic	It was not the annuas were plogr
Keray Pents of the same Reagun D	This will be us , the ect of DAN
Manging include a tudancs shat "	These leaded as most-worsd p2 a0
His Zuith Dudget , the Denmbern	The time I paidOa South Cubry i
In during the Uitational questio	Dour Fraps higs it was these del
Divos from The ' noth ronkies of	This year out howneed allowed lo
She like Monday , of macunsuer S	Kaulna Seto consficutes to repor

表 4：在 Billion Word 句子（截断到 32 个字符）上训练的 WGAN-GP 字符级语言模型样本。模型从隐向量直接输出 one-hot 字符嵌入，中间没有任何离散采样步骤。我们没能用标准 GAN 目标和连续生成器得到可比较的结果。

样本	样本
Busino game camperate spent odea	Solice Norkedin pring in since
In the bankaway of smarling the	ThiS record ( 31. ) UBS ) and Ch
SingersMay , who kill that imvic	It was not the annuas were plogr
Keray Pents of the same Reagun D	This will be us , the ect of DAN
Manging include a tudancs shat "	These leaded as most-worsd p2 a0
His Zuith Dudget , the Denmbern	The time I paidOa South Cubry i
In during the Uitational questio	Dour Fraps higs it was these del
Divos from The ' noth ronkies of	This year out howneed allowed lo
She like Monday , of macunsuer S	Kaulna Seto consficutes to repor

The difference in performance between WGAN-GP and other GANs can be explained as follows. Consider the simplex $\Delta_n=\{p\in\mathbb{R}^n:p_i\ge 0,\sum_i p_i=1\}$, and the set of vertices on the simplex (or one-hot vectors) $V_n=\{p\in\mathbb{R}^n:p_i\in\{0,1\},\sum_i p_i=1\}\subseteq\Delta_n$. If we have a vocabulary of size $n$ and a distribution $\mathbb{P}_r$ over sequences of size $T$, then $\mathbb{P}_r$ is a distribution on $V_n^T=V_n\times\cdots\times V_n$. Since $V_n^T$ is a subset of $\Delta_n^T$, we can also treat $\mathbb{P}_r$ as a distribution on $\Delta_n^T$ (by assigning zero probability mass to all points not in $V_n^T$). $\mathbb{P}_r$ is discrete on $\Delta_n^T$, but $\mathbb{P}_g$ can easily be a continuous distribution over $\Delta_n^T$. The KL divergences between two such distributions are infinite, and so the JS divergence is saturated.

WGAN-GP 与其他 GAN 表现不同，可以这样理解。考虑单纯形 $\Delta_n=\{p\in\mathbb{R}^n:p_i\ge 0,\sum_i p_i=1\}$，以及单纯形上的顶点（也就是 one-hot 向量）$V_n=\{p\in\mathbb{R}^n:p_i\in\{0,1\},\sum_i p_i=1\}\subseteq\Delta_n$。若词表大小为 $n$，并且 $\mathbb{P}_r$ 是长度为 $T$ 的序列上的分布，则 $\mathbb{P}_r$ 是 $V_n^T=V_n\times\cdots\times V_n$ 上的分布。由于 $V_n^T$ 是 $\Delta_n^T$ 的子集，也可以把 $\mathbb{P}_r$ 看成 $\Delta_n^T$ 上的分布（对不在 $V_n^T$ 上的点赋予零概率）。$\mathbb{P}_r$ 在 $\Delta_n^T$ 上是离散的，但 $\mathbb{P}_g$ 很容易成为 $\Delta_n^T$ 上的连续分布。这两种分布之间的 KL 散度为无穷，因而 JS 散度会饱和。

Although GANs do not literally minimize these divergences [16], in practice this means a discriminator might quickly learn to reject all samples that don't lie on $V_n^T$ (sequences of one-hot vectors) and give meaningless gradients to the generator. However, it is easily seen that the conditions of Theorem 1 and Corollary 1 of [2] are satisfied even on this non-standard learning scenario with $\mathcal{X}=\Delta_n^T$. This means that $W(\mathbb{P}_r,\mathbb{P}_g)$ is still well defined, continuous everywhere and differentiable almost everywhere, and we can optimize it just like in any other continuous variable setting. The way this manifests is that in WGANs, the Lipschitz constraint forces the critic to provide a linear gradient from all $\Delta_n^T$ towards the real points in $V_n^T$. Other attempts at language modeling with GANs [32, 14, 30, 5, 15, 10] typically use discrete models and gradient estimators [28, 12, 17]. Our approach is simpler to implement, though whether it scales beyond a toy language model is unclear.

尽管 GAN 并非字面上最小化这些散度 [16]，实践中这意味着判别器可能会很快学会拒绝所有不在 $V_n^T$ 上的样本（也就是非 one-hot 序列），从而只给生成器无意义的梯度。但不难看到，即便在 $\mathcal{X}=\Delta_n^T$ 这种非标准学习设定下，[2] 的定理 1 与推论 1 的条件仍然满足。这意味着 $W(\mathbb{P}_r,\mathbb{P}_g)$ 仍然定义良好，处处连续且几乎处处可微，可以像其他连续变量设定一样优化它。具体表现是：在 WGAN 里，Lipschitz 约束迫使 critic 从整个 $\Delta_n^T$ 向 $V_n^T$ 中的真实点提供线性梯度。其他用 GAN 做语言建模的尝试 [32, 14, 30, 5, 15, 10] 通常使用离散模型和梯度估计器 [28, 12, 17]。我们的做法实现起来更简单，它能否超出玩具语言模型，目前还不清楚。

5.6 Meaningful loss curves and detecting overfitting

5.6 有意义的损失曲线与过拟合检测

An important benefit of weight-clipped WGANs is that their loss correlates with sample quality and converges toward a minimum. To show that our method preserves this property, we train a WGAN-GP on the LSUN bedrooms dataset [31] and plot the negative of the critic's loss in Figure 5a. We see that the loss converges as the generator minimizes $W(\mathbb{P}_r,\mathbb{P}_g)$.

带权重裁剪的 WGAN 有一个重要优点：损失与样本质量相关，并且会走向某个最小值。为说明本方法保留了这一性质，我们在 LSUN bedrooms [31] 上训练 WGAN-GP，并在图 5a 中画出 critic 损失的相反数。可以看到，随着生成器最小化 $W(\mathbb{P}_r,\mathbb{P}_g)$，损失在收敛。

Figure 5: (a) The negative critic loss of our model on LSUN bedrooms converges toward a minimum as the network trains. (b) WGAN training and validation losses on a random 1000-digit subset of MNIST show overfitting when using either our method (left) or weight clipping (right). In particular, with our method, the critic overfits faster than the generator, causing the training loss to increase gradually over time even as the validation loss drops.

图 5：（a）我们的模型在 LSUN bedrooms 上的负 critic 损失随训练走向某个最小值。（b）在随机抽取的 1000 张 MNIST 子集上，无论用我们的方法（左）还是权重裁剪（右），WGAN 的训练损失与验证损失都显示出过拟合。特别是用我们的方法时，critic 比生成器更快过拟合，因而即使验证损失在下降，训练损失也会逐渐上升。

Given enough capacity and too little training data, GANs will overfit. To explore the loss curve's behavior when the network overfits, we train large unregularized WGANs on a random 1000-image subset of MNIST and plot the negative critic loss on both the training and validation sets in Figure 5b. In both WGAN and WGAN-GP, the two losses diverge, suggesting that the critic overfits and provides an inaccurate estimate of $W(\mathbb{P}_r,\mathbb{P}_g)$, at which point all bets are off regarding correlation with sample quality. However in WGAN-GP, the training loss gradually increases even while the validation loss drops. [29] also measure overfitting in GANs by estimating the generator's log-likelihood. Compared to that work, our method detects overfitting in the critic (rather than the generator) and measures overfitting against the same loss that the network minimizes.

如果容量足够大而训练数据太少，GAN 会过拟合。为观察网络过拟合时损失曲线的行为，我们在随机抽取的 1000 张 MNIST 图像上训练大型、无正则的 WGAN，并在图 5b 中同时画出训练集和验证集上的负 critic 损失。WGAN 与 WGAN-GP 中两条损失都会分开，说明 critic 过拟合了，给出的 $W(\mathbb{P}_r,\mathbb{P}_g)$ 估计不准；此时损失与样本质量是否相关就不好说了。不过在 WGAN-GP 里，即便验证损失在下降，训练损失也会逐渐上升。[29] 也曾通过估计生成器的对数似然来度量 GAN 的过拟合。与那项工作相比，我们检测的是 critic（而不是生成器）的过拟合，并且度量的就是网络正在最小化的那一项损失。

6 Conclusion

6 结论

In this work, we demonstrated problems with weight clipping in WGAN and introduced an alternative in the form of a penalty term in the critic loss which does not exhibit the same problems. Using our method, we demonstrated strong modeling performance and stability across a variety of architectures. Now that we have a more stable algorithm for training GANs, we hope our work opens the path for stronger modeling performance on large-scale image datasets and language. Another interesting direction is adapting our penalty term to the standard GAN objective function, where it might stabilize training by encouraging the discriminator to learn smoother decision boundaries.

本文指出了 WGAN 中权重裁剪的问题，并在 critic 损失里加入一项惩罚作为替代，它没有同样的问题。用这一方法，我们在多种结构上展示了较强的建模能力和稳定性。既然现在有了更稳定的 GAN 训练算法，希望这项工作能为大规模图像数据和语言上更强的建模打开路子。另一个值得做的方向，是把这项惩罚接到标准 GAN 目标上：它或许能促使判别器学到更平滑的决策边界，从而稳定训练。

Acknowledgements

致谢

We would like to thank Mohamed Ishmael Belghazi, Léon Bottou, Zihang Dai, Stefan Doerr, Ian Goodfellow, Kyle Kastner, Kundan Kumar, Luke Metz, Alec Radford, Colin Raffel, Sai Rajeshwar, Aditya Ramesh, Tom Sercu, Zain Shah and Jake Zhao for insightful comments.

感谢 Mohamed Ishmael Belghazi、Léon Bottou、Zihang Dai、Stefan Doerr、Ian Goodfellow、Kyle Kastner、Kundan Kumar、Luke Metz、Alec Radford、Colin Raffel、Sai Rajeshwar、Aditya Ramesh、Tom Sercu、Zain Shah 和 Jake Zhao 提出的有见地的意见。

References

参考文献

[1] M. Arjovsky and L. Bottou. Towards principled methods for training generative adversarial networks. 2017.
[2] M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.
[3] J. L. Ba, J. R. Kiros, and G. E. Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
[4] D. Berthelot, T. Schumm, and L. Metz. Began: Boundary equilibrium generative adversarial networks. arXiv preprint arXiv:1703.10717, 2017.
[5] T. Che, Y. Li, R. Zhang, R. D. Hjelm, W. Li, Y. Song, and Y. Bengio. Maximum-likelihood augmented discrete generative adversarial networks. arXiv preprint arXiv:1702.07983, 2017.
[6] C. Chelba, T. Mikolov, M. Schuster, Q. Ge, T. Brants, P. Koehn, and T. Robinson. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005, 2013.
[7] Z. Dai, A. Almahairi, P. Bachman, E. Hovy, and A. Courville. Calibrating energy-based generative adversarial networks. arXiv preprint arXiv:1702.01691, 2017.
[8] V. Dumoulin, M. I. D. Belghazi, B. Poole, A. Lamb, M. Arjovsky, O. Mastropietro, and A. Courville. Adversarially learned inference. 2017.
[9] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
[10] R. D. Hjelm, A. P. Jacob, T. Che, K. Cho, and Y. Bengio. Boundary-seeking generative adversarial networks. arXiv preprint arXiv:1702.08431, 2017.

[11] X. Huang, Y. Li, O. Poursaeed, J. Hopcroft, and S. Belongie. Stacked generative adversarial networks. arXiv preprint arXiv:1612.04357, 2016.
[12] E. Jang, S. Gu, and B. Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016.
[13] A. Krizhevsky. Learning multiple layers of features from tiny images. 2009.
[14] J. Li, W. Monroe, T. Shi, A. Ritter, and D. Jurafsky. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547, 2017.
[15] X. Liang, Z. Hu, H. Zhang, C. Gan, and E. P. Xing. Recurrent topic-transition gan for visual paragraph generation. arXiv preprint arXiv:1703.07022, 2017.
[16] S. Liu, O. Bousquet, and K. Chaudhuri. Approximation and convergence properties of generative adversarial learning. arXiv preprint arXiv:1705.08991, 2017.
[17] C. J. Maddison, A. Mnih, and Y. W. Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.00712, 2016.
[18] X. Mao, Q. Li, H. Xie, R. Y. Lau, and Z. Wang. Least squares generative adversarial networks. arXiv preprint arXiv:1611.04076, 2016.
[19] L. Metz, B. Poole, D. Pfau, and J. Sohl-Dickstein. Unrolled generative adversarial networks. arXiv preprint arXiv:1611.02163, 2016.
[20] A. Odena, C. Olah, and J. Shlens. Conditional image synthesis with auxiliary classifier gans. arXiv preprint arXiv:1610.09585, 2016.

[21] B. Poole, A. A. Alemi, J. Sohl-Dickstein, and A. Angelova. Improved generator objectives for gans. arXiv preprint arXiv:1612.02780, 2016.
[22] A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
[23] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pages 2226–2234, 2016.
[24] A. van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves, et al. Conditional image generation with pixelcnn decoders. In Advances in Neural Information Processing Systems, pages 4790–4798, 2016.
[25] C. Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.
[26] D. Wang and Q. Liu. Learning to draw samples: With application to amortized mle for generative adversarial learning. arXiv preprint arXiv:1611.01722, 2016.
[27] D. Warde-Farley and Y. Bengio. Improving generative adversarial networks with denoising feature matching. 2017.
[28] R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229–256, 1992.
[29] Y. Wu, Y. Burda, R. Salakhutdinov, and R. Grosse. On the quantitative analysis of decoder-based generative models. arXiv preprint arXiv:1611.04273, 2016.
[30] Z. Yang, W. Chen, F. Wang, and B. Xu. Improving neural machine translation with conditional sequence generative adversarial nets. arXiv preprint arXiv:1703.04887, 2017.
[31] F. Yu, A. Seff, Y. Zhang, S. Song, T. Funkhouser, and J. Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.
[32] L. Yu, W. Zhang, J. Wang, and Y. Yu. Seqgan: sequence generative adversarial nets with policy gradient. arXiv preprint arXiv:1609.05473, 2016.

A Proof of Proposition 1

附录 A 命题 1 的证明

Proof. Since $\mathcal{X}$ is a compact space, by Theorem 5.10 of [25], part (iii), we know that there is an optimal $f^*$. By Theorem 5.10 of [25], part (ii) we know that if $\pi$ is an optimal coupling, $\mathbb{P}_{(x,y)\sim\pi}[f^*(y)-f^*(x)=\|y-x\|]=1$. Let $(x,y)$ be such that $f^*(y)-f^*(x)=\|y-x\|$. We can safely assume that $x\neq y$ as well, since this happens under $\pi$ with probability 1. Let $\psi(t)=f^*(x_t)-f^*(x)$. We claim that $\psi(t)=\|x_t-x\|=t\|y-x\|$.

证明。由于 $\mathcal{X}$ 是紧致空间，由 [25] 定理 5.10 的 (iii) 可知存在最优的 $f^*$。再由该定理的 (ii)，若 $\pi$ 是最优耦合，则 $\mathbb{P}_{(x,y)\sim\pi}[f^*(y)-f^*(x)=\|y-x\|]=1$。取满足 $f^*(y)-f^*(x)=\|y-x\|$ 的 $(x,y)$。我们可以放心地假定 $x\neq y$，因为这在 $\pi$ 下以概率 1 成立。令 $\psi(t)=f^*(x_t)-f^*(x)$。我们断言 $\psi(t)=\|x_t-x\|=t\|y-x\|$。

Let $t,t'\in[0,1]$, then $|\psi(t)-\psi(t')|=|f^*(x_t)-f^*(x_{t'})|\le\|x_t-x_{t'}\|=|t-t'|\|x-y\|$. Therefore $\psi$ is $\|x-y\|$-Lipschitz. This in turn implies $\psi(1)-\psi(0)=\psi(1)-\psi(t)+\psi(t)-\psi(0)\le(1-t)\|x-y\|+\psi(t)-\psi(0)\le(1-t)\|x-y\|+t\|x-y\|=\|x-y\|$. However $|\psi(1)-\psi(0)|=|f^*(y)-f^*(x)|=\|y-x\|$, so the inequalities have to actually be equalities. In particular $\psi(t)-\psi(0)=t\|x-y\|$ and $\psi(0)=0$, hence $\psi(t)=t\|x-y\|$.

对 $t,t'\in[0,1]$，有 $|\psi(t)-\psi(t')|=|f^*(x_t)-f^*(x_{t'})|\le\|x_t-x_{t'}\|=|t-t'|\|x-y\|$。因此 $\psi$ 是 $\|x-y\|$-Lipschitz 的。进而 $\psi(1)-\psi(0)=\psi(1)-\psi(t)+\psi(t)-\psi(0)\le(1-t)\|x-y\|+\psi(t)-\psi(0)\le(1-t)\|x-y\|+t\|x-y\|=\|x-y\|$。但 $|\psi(1)-\psi(0)|=|f^*(y)-f^*(x)|=\|y-x\|$，所以这些不等式实际上都是等式。特别地，$\psi(t)-\psi(0)=t\|x-y\|$ 且 $\psi(0)=0$，于是 $\psi(t)=t\|x-y\|$。

Let $v=(y-x_t)/\|y-x_t\|$. Direct algebra shows $v=(y-x)/\|y-x\|$, which does not depend on $t$. Since $f^*(x_t)=f^*(x)+t\|x-y\|$, the directional derivative of $f^*$ at $x_t$ along $v$ equals 1. If $f^*$ is differentiable at $x_t$, we also have $\|\nabla f^*(x_t)\|\le 1$ because it is 1-Lipschitz. By Pythagoras, $1\le\|\nabla f^*(x_t)\|^2=\langle v,\nabla f^*(x_t)\rangle^2+\|\nabla f^*(x_t)-\langle v,\nabla f^*(x_t)\rangle v\|^2=1+\|\nabla f^*(x_t)-v\|^2\le 1$, hence $\nabla f^*(x_t)=v=(y-x_t)/\|y-x_t\|$. Since this happens with probability 1 under $\pi$, we obtain $\mathbb{P}_{(x,y)\sim\pi}[\nabla f^*(x_t)=(y-x_t)/\|y-x_t\|]=1$.

令 $v=(y-x_t)/\|y-x_t\|$。直接计算可得 $v=(y-x)/\|y-x\|$，它不依赖于 $t$。由于 $f^*(x_t)=f^*(x)+t\|x-y\|$，$f^*$ 在 $x_t$ 处沿 $v$ 的方向导数等于 1。若 $f^*$ 在 $x_t$ 处可微，又因为它是 1-Lipschitz 的，故 $\|\nabla f^*(x_t)\|\le 1$。由勾股关系，$1\le\|\nabla f^*(x_t)\|^2=\langle v,\nabla f^*(x_t)\rangle^2+\|\nabla f^*(x_t)-\langle v,\nabla f^*(x_t)\rangle v\|^2=1+\|\nabla f^*(x_t)-v\|^2\le 1$，因此 $\nabla f^*(x_t)=v=(y-x_t)/\|y-x_t\|$。这在 $\pi$ 下以概率 1 成立，故 $\mathbb{P}_{(x,y)\sim\pi}[\nabla f^*(x_t)=(y-x_t)/\|y-x_t\|]=1$。

B More details for training random architectures within a set

附录 B 随机结构训练的更多细节

All models were trained on $32\times 32$ ImageNet for 100K generator iterations using Adam with hyperparameters as recommended in [22] ($\alpha=0.0002$, $\beta_1=0.5$, $\beta_2=0.999$) for the standard GAN objective and our recommended settings ($\alpha=0.0001$, $\beta_1=0$, $\beta_2=0.9$) for WGAN-GP. In the discriminator, if we use batch normalization (or layer normalization) we also apply a small weight decay ($\lambda=10^{-3}$), finding that this helps both algorithms slightly.

所有模型都在 $32\times 32$ ImageNet 上训练 10 万次生成器迭代。标准 GAN 目标使用 [22] 建议的 Adam 超参数（$\alpha=0.0002$，$\beta_1=0.5$，$\beta_2=0.999$）；WGAN-GP 使用我们建议的设定（$\alpha=0.0001$，$\beta_1=0$，$\beta_2=0.9$）。在判别器里，若使用批归一化（或层归一化），我们再加一小的权重衰减（$\lambda=10^{-3}$），发现这对两种算法都略有帮助。

Table 5: Outcomes of training 200 random architectures, for different success thresholds. For comparison, our standard DCGAN achieved a score of 7.24.

Min. score	Only GAN	Only WGAN-GP	Both succeeded	Both failed
1.0	0	8	192	0
1.5	0	50	150	0
2.0	0	60	140	0
2.5	0	74	125	1
3.0	1	88	110	1
3.5	0	111	86	3
4.0	1	126	67	6
4.5	0	136	55	9
5.0	0	147	42	11
5.5	0	148	32	20
6.0	0	145	21	34
6.5	1	131	11	57
7.0	1	104	5	90
7.5	2	67	3	128
8.0	1	34	0	165
8.5	0	6	0	194
9.0	0	0	0	200

表 5：在不同成功阈值下训练 200 个随机结构的结果。对照用的标准 DCGAN 分数为 7.24。

最低分数	仅 GAN	仅 WGAN-GP	两者都成功	两者都失败
1.0	0	8	192	0
1.5	0	50	150	0
2.0	0	60	140	0
2.5	0	74	125	1
3.0	1	88	110	1
3.5	0	111	86	3
4.0	1	126	67	6
4.5	0	136	55	9
5.0	0	147	42	11
5.5	0	148	32	20
6.0	0	145	21	34
6.5	1	131	11	57
7.0	1	104	5	90
7.5	2	67	3	128
8.0	1	34	0	165
8.5	0	6	0	194
9.0	0	0	0	200

C Experiments with one-sided penalty

附录 C 单侧惩罚实验

We considered a one-sided penalty of the form $\lambda\,\mathbb{E}_{\hat{x}\sim\mathbb{P}_{\hat{x}}}[\max(0,\|\nabla_{\hat{x}}D(\hat{x})\|_2-1)^2]$ which would penalize gradients larger than 1 but not gradients smaller than 1, but we observe that the two-sided version seems to perform slightly better. We sample 174 architectures from the set specified in Table 1 and train each architecture with the one-sided and two-sided penalty terms. The two-sided penalty achieved a higher Inception score in 100 of the trials, compared to 77 for the one-sided penalty. We note that this result is not statistically significant at $p<0.05$ and further is with respect to only one (somewhat arbitrary) metric and distribution of architectures, and it is entirely possible (likely, in fact) that there are settings where the one-sided penalty performs better, but we leave a thorough comparison for future work. Other training details are the same as in Appendix B.

我们考虑过形如 $\lambda\,\mathbb{E}_{\hat{x}\sim\mathbb{P}_{\hat{x}}}[\max(0,\|\nabla_{\hat{x}}D(\hat{x})\|_2-1)^2]$ 的单侧惩罚：它只惩罚大于 1 的梯度，不惩罚小于 1 的梯度。我们观察到双侧版本似乎略好一些。从表 1 指定的集合中抽出 174 个结构，分别用单侧和双侧惩罚训练。双侧惩罚在 100 次试验里拿到更高的 Inception score，单侧是 77 次。需要说明，这一结果在 $p<0.05$ 下并不显著，而且只对应一种（多少有些任意的）指标和结构分布；完全有可能（事实上很可能）存在单侧惩罚更好的设定，完整比较留给后续工作。其余训练细节与附录 B 相同。

D Nonsmooth activation functions

附录 D 非光滑激活函数

The gradient of our objective with respect to the discriminator's parameters contains terms which involve second derivatives of the network's activation functions. In the case of networks with ReLU or other common nonsmooth activation functions, this means the gradient is undefined at some points (albeit a measure zero set) and the gradient penalty objective might not be continuous with respect to the parameters. Gradient descent is not guaranteed to succeed in this setting, but empirically this seems not to be a problem for some common activation functions: in our random architecture and LSUN architecture experiments we find that we are able to train networks with piecewise linear activation functions (ReLU, leaky ReLU) as well as smooth activation functions. We do note that we were unable to train networks with ELU activations, whose derivative is continuous but not smooth. Replacing ELU with a very similar nonlinearity which is smooth ($\mathrm{softplus}(2x+2)/2-1$) fixed the issue.

目标函数对判别器参数的梯度里，含有网络激活函数的二阶导。若网络使用 ReLU 或其他常见的非光滑激活，梯度在某些点上没有定义（尽管这是零测集），梯度惩罚目标对参数也可能不连续。这种设定下梯度下降并不保证成功，但经验上对一些常见激活函数似乎不成问题：在随机结构和 LSUN 结构实验里，分段线性激活（ReLU、leaky ReLU）和光滑激活都能训起来。需要指出，我们没能训起使用 ELU 的网络——ELU 的导数连续但不光滑。把 ELU 换成非常相近、但是光滑的非线性 $\mathrm{softplus}(2x+2)/2-1$，问题就消失了。

E Hyperparameters used for LSUN robustness experiments

附录 E LSUN 稳健性实验所用的超参数

For each method we used the hyperparameters recommended in that method's paper. For LSGAN, we additionally searched over learning rate (because the paper did not make a specific recommendation).

• WGAN with gradient penalty: Adam ($\alpha=.0001$, $\beta_1=.5$, $\beta_2=.9$)
• WGAN with weight clipping: RMSProp ($\alpha=.00005$)
• DCGAN: Adam ($\alpha=.0002$, $\beta_1=.5$)
• LSGAN: RMSProp ($\alpha=.0001$) [chosen by search over $\alpha=.001,.0002,.0001$]

对每种方法，我们使用该方法论文里建议的超参数。对 LSGAN，我们额外搜索了学习率（因为原文没有给出具体建议）。

• 带梯度惩罚的 WGAN：Adam（$\alpha=.0001$，$\beta_1=.5$，$\beta_2=.9$）
• 带权重裁剪的 WGAN：RMSProp（$\alpha=.00005$）
• DCGAN：Adam（$\alpha=.0002$，$\beta_1=.5$）
• LSGAN：RMSProp（$\alpha=.0001$）[在 $\alpha=.001,.0002,.0001$ 中搜索得到]

F CIFAR-10 ResNet architecture

附录 F CIFAR-10 残差网络结构

The generator and critic are residual networks; we use pre-activation residual blocks with two $3\times 3$ convolutional layers each and ReLU nonlinearity. Some residual blocks perform downsampling (in the critic) using mean pooling after the second convolution, or nearest-neighbor upsampling (in the generator) before the second convolution. We use batch normalization in the generator but not the critic. We optimize using Adam with learning rate $2\times 10^{-4}$, decayed linearly to 0 over 100K generator iterations, and batch size 64. For further architectural details, please refer to our open-source implementation.

生成器和 critic 都是残差网络；我们使用预激活残差块，每块含两个 $3\times 3$ 卷积层和 ReLU 非线性。部分残差块会做下采样（在 critic 里，于第二个卷积之后做均值池化），或做最近邻上采样（在生成器里，于第二个卷积之前）。生成器使用批归一化，critic 不使用。优化器为 Adam，学习率 $2\times 10^{-4}$，在 10 万次生成器迭代内线性衰减到 0，批量大小为 64。更多结构细节见我们的开源实现。

Generator $G(z)$:

Layer	Kernel size	Resample	Output shape
$z$	—	—	128
Linear	—	—	$128\times 4\times 4$
Residual block	$[3\times 3]\times 2$	Up	$128\times 8\times 8$
Residual block	$[3\times 3]\times 2$	Up	$128\times 16\times 16$
Residual block	$[3\times 3]\times 2$	Up	$128\times 32\times 32$
Conv, tanh	$3\times 3$	—	$3\times 32\times 32$

Critic $D(x)$:

Layer	Kernel size	Resample	Output shape
Residual block	$[3\times 3]\times 2$	Down	$128\times 16\times 16$
Residual block	$[3\times 3]\times 2$	Down	$128\times 8\times 8$
Residual block	$[3\times 3]\times 2$	—	$128\times 8\times 8$
Residual block	$[3\times 3]\times 2$	—	$128\times 8\times 8$
ReLU, mean pool	—	—	128
Linear	—	—	1

生成器 $G(z)$：

层	卷积核大小	重采样	输出形状
$z$	—	—	128
线性层	—	—	$128\times 4\times 4$
残差块	$[3\times 3]\times 2$	上采样	$128\times 8\times 8$
残差块	$[3\times 3]\times 2$	上采样	$128\times 16\times 16$
残差块	$[3\times 3]\times 2$	上采样	$128\times 32\times 32$
卷积，tanh	$3\times 3$	—	$3\times 32\times 32$

Critic $D(x)$：

层	卷积核大小	重采样	输出形状
残差块	$[3\times 3]\times 2$	下采样	$128\times 16\times 16$
残差块	$[3\times 3]\times 2$	下采样	$128\times 8\times 8$
残差块	$[3\times 3]\times 2$	—	$128\times 8\times 8$
残差块	$[3\times 3]\times 2$	—	$128\times 8\times 8$
ReLU，均值池化	—	—	128
线性层	—	—	1

G CIFAR-10 ResNet samples

附录 G CIFAR-10 残差网络样本

Figure 6: (left) CIFAR-10 samples generated by our unsupervised model. (right) Conditional CIFAR-10 samples, from adding AC-GAN conditioning to our unconditional model. Samples from the same class are displayed in the same column.

图 6：（左）我们的无监督模型生成的 CIFAR-10 样本。（右）在无条件模型上加入 AC-GAN 式条件后得到的条件 CIFAR-10 样本。同一类的样本排在同一列。

H More LSUN samples

附录 H 更多 LSUN 样本

Additional generated-sample sheets for the architecture variants in Figure 2 (DCGAN, LSGAN, WGAN with clipping, and WGAN-GP) are included in the original PDF appendix, covering each combination of generator/discriminator changes discussed in Section 5.2. Those sheets are unlabeled sample grids rather than numbered figures.

图 2 中各结构变体（DCGAN、LSGAN、带裁剪的 WGAN，以及 WGAN-GP）的更多生成样本页收录在原文 PDF 附录里，覆盖第 5.2 节讨论的每一种生成器/判别器改动组合。那些页面是未编号的样本网格，而不是带图号的插图。