Auto-Encoding Variational Bayes

自编码变分贝叶斯（Auto-Encoding Variational Bayes）

Diederik P. Kingma Affiliation:  Machine Learning Group Affiliation:  Universiteit van Amsterdam Email:  dpkingma@gmail.com    Max Welling Affiliation:  Machine Learning Group Affiliation:  Universiteit van Amsterdam Email:  welling.max@gmail.com

Diederik P. Kingma（隶属：Machine Learning Group，Universiteit van Amsterdam，邮箱：dpkingma@gmail.com）、Max Welling（隶属：Machine Learning Group，Universiteit van Amsterdam，邮箱：welling.max@gmail.com）

Abstract

摘要

How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case. Our contributions are two-fold. First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods. Second, we show that for i.i.d. datasets with continuous latent variables per datapoint, posterior inference can be made especially efficient by fitting an approximate inference model (also called a recognition model) to the intractable posterior using the proposed lower bound estimator. Theoretical advantages are reflected in experimental results.

在存在连续隐变量、后验分布不可积以及大规模数据集的情况下，我们如何在有向概率模型中进行高效的推断与学习？我们提出一种随机变分推断与学习算法，该算法可扩展到大数据集，并且在一些温和的可微条件下，即便在不可积情形下也能工作。贡献有两点。第一，我们证明对变分下界做重参数化，可以得到一个可用标准随机梯度方法直接优化的下界估计器。第二，我们证明：对于每个数据点带有连续隐变量的 i.i.d. 数据集，用所提出的下界估计器去拟合一个逼近不可积后验的近似推断模型（也称识别模型），可以使后验推断特别高效。理论优势在实验结果中得到体现。

1 Introduction

1 引言

How can we perform efficient approximate inference and learning with directed probabilistic models whose continuous latent variables and/or parameters have intractable posterior distributions? The variational Bayesian (VB) approach involves the optimization of an approximation to the intractable posterior. Unfortunately, the common mean-field approach requires analytical solutions of expectations w.r.t. the approximate posterior, which are also intractable in the general case. We show how a reparameterization of the variational lower bound yields a simple differentiable unbiased estimator of the lower bound; this SGVB (Stochastic Gradient Variational Bayes) estimator can be used for efficient approximate posterior inference in almost any model with continuous latent variables and/or parameters, and is straightforward to optimize using standard stochastic gradient ascent techniques.

我们如何对连续隐变量和/或参数具有不可积后验分布的有向概率模型，进行高效的近似推断与学习？变分贝叶斯（VB）方法通过对不可积后验的近似进行优化。遗憾的是，常见的平均场方法需要就可近似后验解析地求出期望，这在一般情形下同样不可积。我们证明：对变分下界做重参数化，可以得到下界的一个简单、可微且无偏的估计器；该 SGVB（Stochastic Gradient Variational Bayes，随机梯度变分贝叶斯）估计器可用于几乎任何带连续隐变量和/或参数的模型中的高效近似后验推断，并可用标准随机梯度上升技术直接优化。

For the case of an i.i.d. dataset and continuous latent variables per datapoint, we propose the Auto-Encoding VB (AEVB) algorithm. In the AEVB algorithm we make inference and learning especially efficient by using the SGVB estimator to optimize a recognition model that allows us to perform very efficient approximate posterior inference using simple ancestral sampling, which in turn allows us to efficiently learn the model parameters, without the need of expensive iterative inference schemes (such as MCMC) per datapoint. The learned approximate posterior inference model can also be used for a host of tasks such as recognition, denoising, representation and visualization purposes. When a neural network is used for the recognition model, we arrive at the variational auto-encoder.

针对 i.i.d. 数据集且每个数据点带有连续隐变量的情形，我们提出自编码变分贝叶斯（Auto-Encoding VB，AEVB）算法。在 AEVB 中，我们用 SGVB 估计器优化识别模型，使推断与学习特别高效：该识别模型允许用简单的祖先采样做非常高效的近似后验推断，从而高效学习模型参数，而无需对每个数据点做昂贵的迭代推断（如 MCMC）。学到的近似后验推断模型还可用于识别、去噪、表示与可视化等任务。当识别模型采用神经网络时，我们就得到了变分自编码器。

2 Method

2 方法

The strategy in this section can be used to derive a lower bound estimator (a stochastic objective function) for a variety of directed graphical models with continuous latent variables. We will restrict ourselves here to the common case where we have an i.i.d. dataset with latent variables per datapoint, and where we like to perform maximum likelihood (ML) or maximum a posteriori (MAP) inference on the (global) parameters, and variational inference on the latent variables. It is, for example, straightforward to extend this scenario to the case where we also perform variational inference on the global parameters; that algorithm is put in the appendix, but experiments with that case are left to future work. Note that our method can be applied to online, non-stationary settings, e.g. streaming data, but here we assume a fixed dataset for simplicity.

本节的策略可用于为一类带连续隐变量的有向图模型推导下界估计器（随机目标函数）。我们在此限于常见情形：拥有每个数据点带隐变量的 i.i.d. 数据集，希望对（全局）参数做最大似然（ML）或最大后验（MAP）推断，并对隐变量做变分推断。例如，将其扩展到对全局参数也做变分推断是直接的；该算法放在附录中，相关实验留给未来工作。注意我们的方法可用于在线、非平稳设定（例如流式数据），但为简单起见这里假设数据集固定。

Figure 1: The type of directed graphical model under consideration. Solid lines denote the generative model $p_{\boldsymbol{\theta}}(\mathbf{z})p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$, dashed lines denote the variational approximation $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ to the intractable posterior $p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x})$. The variational parameters $\boldsymbol{\phi}$ are learned jointly with the generative model parameters $\boldsymbol{\theta}$.

图 1：所考虑的有向图模型类型。实线表示生成模型 $p_{\boldsymbol{\theta}}(\mathbf{z})p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$，虚线表示对不可积后验 $p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x})$ 的变分近似 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$。变分参数 $\boldsymbol{\phi}$ 与生成模型参数 $\boldsymbol{\theta}$ 联合学习。

2.1 Problem scenario

2.1 问题设定

Let us consider some dataset $\mathbf{X}=\{\mathbf{x}^{(i)}\}_{i=1}^{N}$ consisting of $N$ i.i.d. samples of some continuous or discrete variable $\mathbf{x}$. We assume that the data are generated by some random process, involving an unobserved continuous random variable $\mathbf{z}$. The process consists of two steps: (1) a value $\mathbf{z}^{(i)}$ is generated from some prior distribution $p_{\boldsymbol{\theta}^{*}}(\mathbf{z})$; (2) a value $\mathbf{x}^{(i)}$ is generated from some conditional distribution $p_{\boldsymbol{\theta}^{*}}(\mathbf{x}|\mathbf{z})$. We assume that the prior $p_{\boldsymbol{\theta}^{*}}(\mathbf{z})$ and likelihood $p_{\boldsymbol{\theta}^{*}}(\mathbf{x}|\mathbf{z})$ come from parametric families of distributions $p_{\boldsymbol{\theta}}(\mathbf{z})$ and $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$, and that their PDFs are differentiable almost everywhere w.r.t. both $\boldsymbol{\theta}$ and $\mathbf{z}$. Unfortunately, a lot of this process is hidden from our view: the true parameters $\boldsymbol{\theta}^{*}$ as well as the values of the latent variables $\mathbf{z}^{(i)}$ are unknown to us.

考虑由连续或离散变量 $\mathbf{x}$ 的 $N$ 个 i.i.d. 样本组成的数据集 $\mathbf{X}=\{\mathbf{x}^{(i)}\}_{i=1}^{N}$。我们假定数据由某个涉及未观测连续随机变量 $\mathbf{z}$ 的随机过程生成。该过程分两步：(1) 从某先验分布 $p_{\boldsymbol{\theta}^{*}}(\mathbf{z})$ 生成值 $\mathbf{z}^{(i)}$；(2) 从某条件分布 $p_{\boldsymbol{\theta}^{*}}(\mathbf{x}|\mathbf{z})$ 生成值 $\mathbf{x}^{(i)}$。我们假定先验 $p_{\boldsymbol{\theta}^{*}}(\mathbf{z})$ 与似然 $p_{\boldsymbol{\theta}^{*}}(\mathbf{x}|\mathbf{z})$ 来自参数族 $p_{\boldsymbol{\theta}}(\mathbf{z})$ 与 $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$，且其概率密度函数几乎处处对 $\boldsymbol{\theta}$ 与 $\mathbf{z}$ 可微。遗憾的是，这一过程有许多对我们隐藏：真实参数 $\boldsymbol{\theta}^{*}$ 以及隐变量取值 $\mathbf{z}^{(i)}$ 均未知。

Very importantly, we do not make the common simplifying assumptions about the marginal or posterior probabilities. Conversely, we are here interested in a general algorithm that even works efficiently in the case of:

非常重要的是，我们并不对边际或后验概率做常见的简化假设。相反，我们关心一种即便在下列情形下仍能高效工作的通用算法：

1. Intractability: the case where the integral of the marginal likelihood $p_{\boldsymbol{\theta}}(\mathbf{x})=\int p_{\boldsymbol{\theta}}(\mathbf{z})p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})\,d\mathbf{z}$ is intractable (so we cannot evaluate or differentiate the marginal likelihood), where the true posterior density $p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x})=p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})p_{\boldsymbol{\theta}}(\mathbf{z})/p_{\boldsymbol{\theta}}(\mathbf{x})$ is intractable (so the EM algorithm cannot be used), and where the required integrals for any reasonable mean-field VB algorithm are also intractable. These intractabilities are quite common and appear in cases of moderately complicated likelihood functions $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$, e.g. a neural network with a nonlinear hidden layer.
2. A large dataset: we have so much data that batch optimization is too costly; we would like to make parameter updates using small minibatches or even single datapoints. Sampling-based solutions, e.g. Monte Carlo EM, would in general be too slow, since it involves a typically expensive sampling loop per datapoint.

1. 不可积性：边际似然 $p_{\boldsymbol{\theta}}(\mathbf{x})=\int p_{\boldsymbol{\theta}}(\mathbf{z})p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})\,d\mathbf{z}$ 的积分不可积（因而无法评估或微分边际似然），真实后验密度 $p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x})=p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})p_{\boldsymbol{\theta}}(\mathbf{z})/p_{\boldsymbol{\theta}}(\mathbf{x})$ 不可积（因而无法使用 EM 算法），且任何合理平均场 VB 算法所需的积分也不可积。这类不可积性相当常见，出现在似然函数 $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ 适度复杂时，例如带非线性隐层的神经网络。
2. 大规模数据集：数据量大会使批量优化代价过高；我们希望用小批量甚至单个数据点做参数更新。基于采样的方案（如蒙特卡洛 EM）通常过慢，因为每个数据点都涉及通常很昂贵的采样循环。

We are interested in, and propose a solution to, three related problems in the above scenario:

我们关心并针对上述情形提出对三个相关问题的解：

1. Efficient approximate ML or MAP estimation for the parameters $\boldsymbol{\theta}$. The parameters can be of interest themselves, e.g. if we are analyzing some natural process. They also allow us to mimic the hidden random process and generate artificial data that resembles the real data.
2. Efficient approximate posterior inference of the latent variable $\mathbf{z}$ given an observed value $\mathbf{x}$ for a choice of parameters $\boldsymbol{\theta}$. This is useful for coding or data representation tasks.
3. Efficient approximate marginal inference of the variable $\mathbf{x}$. This allows us to perform all kinds of inference tasks where a prior over $\mathbf{x}$ is required. Common applications in computer vision include image denoising, inpainting and super-resolution.

1. 对参数 $\boldsymbol{\theta}$ 的高效近似 ML 或 MAP 估计。参数本身可能有兴趣，例如我们在分析某个自然过程时；它们也允许我们模仿隐藏的随机过程并生成与真实数据相似的人工数据。
2. 在选定参数 $\boldsymbol{\theta}$ 时，给定观测 $\mathbf{x}$ 对隐变量 $\mathbf{z}$ 的高效近似后验推断。这对编码或数据表示任务有用。
3. 对变量 $\mathbf{x}$ 的高效近似边际推断。这使我们能完成各类需要 $\mathbf{x}$ 上先验的推断任务。计算机视觉中的常见应用包括图像去噪、补全与超分辨率。

For the purpose of solving the above problems, let us introduce a recognition model $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$: an approximation to the intractable true posterior $p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x})$. Note that in contrast with the approximate posterior in mean-field variational inference, it is not necessarily factorial and its parameters $\boldsymbol{\phi}$ are not computed from some closed-form expectation. Instead, we’ll introduce a method for learning the recognition model parameters $\boldsymbol{\phi}$ jointly with the generative model parameters $\boldsymbol{\theta}$.

为解决上述问题，引入识别模型 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$：对不可积真实后验 $p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x})$ 的近似。注意与平均场变分推断中的近似后验不同，它不必是因子化的，其参数 $\boldsymbol{\phi}$ 也不是由某个闭式期望算出。相反，我们将引入一种把识别模型参数 $\boldsymbol{\phi}$ 与生成模型参数 $\boldsymbol{\theta}$ 联合学习的方法。

From a coding theory perspective, the unobserved variables $\mathbf{z}$ have an interpretation as a latent representation or code. In this paper we will therefore also refer to the recognition model $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ as a probabilistic encoder, since given a datapoint $\mathbf{x}$ it produces a distribution (e.g. a Gaussian) over the possible values of the code $\mathbf{z}$ from which the datapoint $\mathbf{x}$ could have been generated. In a similar vein we will refer to $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ as a probabilistic decoder, since given a code $\mathbf{z}$ it produces a distribution over the possible corresponding values of $\mathbf{x}$.

从编码理论角度看，未观测变量 $\mathbf{z}$ 可解释为潜在表示或码。因此本文也把识别模型 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 称为概率编码器：给定数据点 $\mathbf{x}$，它在可能生成该点的码 $\mathbf{z}$ 上产出一个分布（例如高斯）。同理，我们把 $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ 称为概率解码器：给定码 $\mathbf{z}$，它在可能对应的 $\mathbf{x}$ 值上产出一个分布。

2.2 The variational bound

2.2 变分下界

The marginal likelihood is composed of a sum over the marginal likelihoods of individual datapoints $\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(1)},\cdots,\mathbf{x}^{(N)})=\sum_{i=1}^{N}\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)})$, which can each be rewritten as:

边际似然由各数据点边际似然之和构成 $\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(1)},\cdots,\mathbf{x}^{(N)})=\sum_{i=1}^{N}\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)})$，每一个可改写为：

$$\displaystyle\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)})=D_{KL}(q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})||p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x}^{(i)}))+\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)}) \tag{1}$$

The first RHS term is the KL divergence of the approximate from the true posterior. Since this KL-divergence is non-negative, the second RHS term $\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})$ is called the (variational) lower bound on the marginal likelihood of datapoint $i$, and can be written as:

右边第一项是近似后验相对真实后验的 KL 散度。由于该 KL 散度非负，右边第二项 $\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})$ 称为数据点 $i$ 边际似然的（变分）下界，并可写成：

$$\displaystyle\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)})\geq\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)}) \\ \displaystyle=\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})}\left[-\log q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})+\log p_{\boldsymbol{\theta}}(\mathbf{x},\mathbf{z})\right] \tag{2}$$

which can also be written as:

也可写成：

$$\displaystyle\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})=-D_{KL}(q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})||p_{\boldsymbol{\theta}}(\mathbf{z}))+\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})}\left[\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z})\right] \tag{3}$$

We want to differentiate and optimize the lower bound $\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})$ w.r.t. both the variational parameters $\boldsymbol{\phi}$ and generative parameters $\boldsymbol{\theta}$. However, the gradient of the lower bound w.r.t. $\boldsymbol{\phi}$ is a bit problematic. The usual (naïve) Monte Carlo gradient estimator for this type of problem is: $\nabla_{\boldsymbol{\phi}}\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z})}\left[f(\mathbf{z})\right]=\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z})}\left[f(\mathbf{z})\nabla_{q_{\boldsymbol{\phi}}(\mathbf{z})}\log q_{\boldsymbol{\phi}}(\mathbf{z})\right]\simeq\frac{1}{L}\sum_{l=1}^{L}f(\mathbf{z})\nabla_{q_{\boldsymbol{\phi}}(\mathbf{z}^{(l)})}\log q_{\boldsymbol{\phi}}(\mathbf{z}^{(l)})$ where $\mathbf{z}^{(l)}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})$. This gradient estimator exhibits exhibits very high variance (see e.g.  [BJP12]) and is impractical for our purposes.

我们希望就可分参数 $\boldsymbol{\phi}$ 与生成参数 $\boldsymbol{\theta}$ 微分并优化下界 $\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})$。然而下界对 $\boldsymbol{\phi}$ 的梯度有些棘手。这类问题常用的（朴素）蒙特卡洛梯度估计器为：$\nabla_{\boldsymbol{\phi}}\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z})}\left[f(\mathbf{z})\right]=\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z})}\left[f(\mathbf{z})\nabla_{q_{\boldsymbol{\phi}}(\mathbf{z})}\log q_{\boldsymbol{\phi}}(\mathbf{z})\right]\simeq\frac{1}{L}\sum_{l=1}^{L}f(\mathbf{z})\nabla_{q_{\boldsymbol{\phi}}(\mathbf{z}^{(l)})}\log q_{\boldsymbol{\phi}}(\mathbf{z}^{(l)})$，其中 $\mathbf{z}^{(l)}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})$。该梯度估计器方差极高（见例如 [BJP12]），对我们的目的不实用。

2.3 The SGVB estimator and AEVB algorithm

2.3 SGVB 估计器与 AEVB 算法

In this section we introduce a practical estimator of the lower bound and its derivatives w.r.t. the parameters. We assume an approximate posterior in the form $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$, but please note that the technique can be applied to the case $q_{\boldsymbol{\phi}}(\mathbf{z})$, i.e. where we do not condition on $\mathbf{x}$, as well. The fully variational Bayesian method for inferring a posterior over the parameters is given in the appendix.

本节引入下界及其对参数导数的实用估计器。我们假定近似后验形式为 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$，但请注意该技术同样适用于 $q_{\boldsymbol{\phi}}(\mathbf{z})$，即不条件于 $\mathbf{x}$ 的情形。对参数做后验推断的完全变分贝叶斯方法见附录。

Under certain mild conditions outlined in section 2.4 for a chosen approximate posterior $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ we can reparameterize the random variable $\widetilde{\mathbf{z}}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ using a differentiable transformation $g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$ of an (auxiliary) noise variable $\boldsymbol{\epsilon}$:

在第 2.4 节对所选近似后验 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 给出的某些温和条件下，我们可以用辅助噪声变量 $\boldsymbol{\epsilon}$ 的可微变换 $g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$ 重参数化随机变量 $\widetilde{\mathbf{z}}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$：

$$\displaystyle\widetilde{\mathbf{z}}=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})\text{\quad with \quad}\boldsymbol{\epsilon}\sim p(\boldsymbol{\epsilon}) \tag{4}$$

See section 2.4 for general strategies for chosing such an approriate distribution $p(\boldsymbol{\epsilon})$ and function $g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$. We can now form Monte Carlo estimates of expectations of some function $f(\mathbf{z})$ w.r.t. $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ as follows:

选择合适分布 $p(\boldsymbol{\epsilon})$ 与函数 $g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$ 的一般策略见第 2.4 节。现在可以把某函数 $f(\mathbf{z})$ 关于 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 的期望的蒙特卡洛估计写成：

$$\displaystyle\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})}\left[f(\mathbf{z})\right]=\mathbb{E}_{p(\boldsymbol{\epsilon})}\left[f(g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x}^{(i)}))\right] \\ \displaystyle\simeq\frac{1}{L}\sum_{l=1}^{L}{f(g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon}^{(l)},\mathbf{x}^{(i)}))}\text{\quad where \quad}\boldsymbol{\epsilon}^{(l)}\sim p(\boldsymbol{\epsilon}) \tag{5}$$

We apply this technique to the variational lower bound (eq. (2)), yielding our generic Stochastic Gradient Variational Bayes (SGVB) estimator $\widetilde{\mathcal{L}}^{A}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})\simeq\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})$:

我们将该技术用于变分下界（式 (2)），得到通用的随机梯度变分贝叶斯（SGVB）估计器 $\widetilde{\mathcal{L}}^{A}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})\simeq\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})$：

$$\displaystyle\widetilde{\mathcal{L}}^{A}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)}) \\ \displaystyle=\frac{1}{L}\sum_{l=1}^{L}\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)},\mathbf{z}^{(i,l)})-\log q_{\boldsymbol{\phi}}(\mathbf{z}^{(i,l)}|\mathbf{x}^{(i)}) \\ \displaystyle\text{where \quad}\mathbf{z}^{(i,l)} \\ \displaystyle=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon}^{(i,l)},\mathbf{x}^{(i)})\text{\quad and \quad}\boldsymbol{\epsilon}^{(l)}\sim p(\boldsymbol{\epsilon}) \tag{6}$$

Often, the KL-divergence $D_{KL}(q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})||p_{\boldsymbol{\theta}}(\mathbf{z}))$ of eq. (3) can be integrated analytically (see appendix B), such that only the expected reconstruction error $\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})}\left[\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z})\right]$ requires estimation by sampling. The KL-divergence term can then be interpreted as regularizing $\boldsymbol{\phi}$, encouraging the approximate posterior to be close to the prior $p_{\boldsymbol{\theta}}(\mathbf{z})$. This yields a second version of the SGVB estimator $\widetilde{\mathcal{L}}^{B}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})\simeq\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})$, corresponding to eq. (3), which typically has less variance than the generic estimator:

常常地，式 (3) 中的 KL 散度 $D_{KL}(q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})||p_{\boldsymbol{\theta}}(\mathbf{z}))$ 可以解析积分（见附录 B），因而只需用采样估计期望重构误差 $\mathbb{E}_{q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})}\left[\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z})\right]$。此时 KL 项可解释为对 $\boldsymbol{\phi}$ 的正则，鼓励近似后验靠近先验 $p_{\boldsymbol{\theta}}(\mathbf{z})$。由此得到对应式 (3) 的第二版 SGVB 估计器 $\widetilde{\mathcal{L}}^{B}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})\simeq\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)})$，通常比通用估计器方差更低：

$$\displaystyle\widetilde{\mathcal{L}}^{B}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)}) \\ \displaystyle=-D_{KL}(q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})||p_{\boldsymbol{\theta}}(\mathbf{z}))+\frac{1}{L}\sum_{l=1}^{L}(\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z}^{(i,l)})) \\ \displaystyle\text{where \quad}\mathbf{z}^{(i,l)} \\ \displaystyle=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon}^{(i,l)},\mathbf{x}^{(i)})\text{\quad and \quad}\boldsymbol{\epsilon}^{(l)}\sim p(\boldsymbol{\epsilon}) \tag{7}$$

Given multiple datapoints from a dataset $\mathbf{X}$ with $N$ datapoints, we can construct an estimator of the marginal likelihood lower bound of the full dataset, based on minibatches:

给定来自含 $N$ 个数据点的数据集 $\mathbf{X}$ 的多个数据点，我们可以基于小批量构造全数据集边际似然下界的估计器：

$$\displaystyle\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{X})\simeq\widetilde{\mathcal{L}}^{M}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{X}^{M})=\frac{N}{M}\sum_{i=1}^{M}\widetilde{\mathcal{L}}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)}) \tag{8}$$

where the minibatch $\mathbf{X}^{M}=\{\mathbf{x}^{(i)}\}_{i=1}^{M}$ is a randomly drawn sample of $M$ datapoints from the full dataset $\mathbf{X}$ with $N$ datapoints. In our experiments we found that the number of samples $L$ per datapoint can be set to $1$ as long as the minibatch size $M$ was large enough, e.g. $M=100$. Derivatives $\nabla_{\boldsymbol{\theta},\boldsymbol{\phi}}\widetilde{\mathcal{L}}(\boldsymbol{\theta};\mathbf{X}^{M})$ can be taken, and the resulting gradients can be used in conjunction with stochastic optimization methods such as SGD or Adagrad [DHS10]. See algorithm 1 for a basic approach to compute the stochastic gradients.

其中小批量 $\mathbf{X}^{M}=\{\mathbf{x}^{(i)}\}_{i=1}^{M}$ 是从全数据集 $\mathbf{X}$（含 $N$ 个点）中随机抽取的 $M$ 个数据点。实验中我们发现，只要小批量大小 $M$ 足够大（例如 $M=100$），每个数据点的样本数 $L$ 可设为 $1$。可以求导数 $\nabla_{\boldsymbol{\theta},\boldsymbol{\phi}}\widetilde{\mathcal{L}}(\boldsymbol{\theta};\mathbf{X}^{M})$，并与 SGD 或 Adagrad [DHS10] 等随机优化方法联用。计算随机梯度的基本做法见算法 1。

A connection with auto-encoders becomes clear when looking at the objective function given at eq. (7). The first term is (the KL divergence of the approximate posterior from the prior) acts as a regularizer, while the second term is a an expected negative reconstruction error. The function $g_{\boldsymbol{\phi}}(.)$ is chosen such that it maps a datapoint $\mathbf{x}^{(i)}$ and a random noise vector $\boldsymbol{\epsilon}^{(l)}$ to a sample from the approximate posterior for that datapoint: $\mathbf{z}^{(i,l)}=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon}^{(l)},\mathbf{x}^{(i)})$ where $\mathbf{z}^{(i,l)}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})$. Subsequently, the sample $\mathbf{z}^{(i,l)}$ is then input to function $\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z}^{(i,l)})$, which equals the probability density (or mass) of datapoint $\mathbf{x}^{(i)}$ under the generative model, given $\mathbf{z}^{(i,l)}$. This term is a negative reconstruction error in auto-encoder parlance.

观察式 (7) 的目标函数，与自动编码器的联系就清楚了。第一项（近似后验相对先验的 KL 散度）起正则作用，第二项是期望负重构误差。函数 $g_{\boldsymbol{\phi}}(.)$ 被选成把数据点 $\mathbf{x}^{(i)}$ 与随机噪声向量 $\boldsymbol{\epsilon}^{(l)}$ 映射为该点近似后验的一个样本：$\mathbf{z}^{(i,l)}=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon}^{(l)},\mathbf{x}^{(i)})$，其中 $\mathbf{z}^{(i,l)}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})$。随后样本 $\mathbf{z}^{(i,l)}$ 被送入 $\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z}^{(i,l)})$，即在给定 $\mathbf{z}^{(i,l)}$ 时数据点在生成模型下的概率密度（或质量）。用自动编码器的说法，这一项就是负重构误差。

$\boldsymbol{\theta},\boldsymbol{\phi}\leftarrow$ Initialize parameters
repeat
$\mathbf{X}^{M}\leftarrow$ Random minibatch of $M$ datapoints (drawn from full dataset)
$\boldsymbol{\epsilon}\leftarrow$ Random samples from noise distribution $p(\boldsymbol{\epsilon})$
$\mathbf{g}\leftarrow\nabla_{\boldsymbol{\theta},\boldsymbol{\phi}}\widetilde{\mathcal{L}}^{M}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{X}^{M},\boldsymbol{\epsilon})$ (Gradients of minibatch estimator (8))
$\boldsymbol{\theta},\boldsymbol{\phi}\leftarrow$ Update parameters using gradients $\mathbf{g}$ (e.g. SGD or Adagrad [DHS10])
until convergence of parameters $(\boldsymbol{\theta},\boldsymbol{\phi})$
return $\boldsymbol{\theta},\boldsymbol{\phi}$

Algorithm 1 Minibatch version of the Auto-Encoding VB (AEVB) algorithm. Either of the two SGVB estimators in section 2.3 can be used. We use settings $M=100$ and $L=1$ in experiments.

$\boldsymbol{\theta},\boldsymbol{\phi}\leftarrow$ 初始化参数
repeat
$\mathbf{X}^{M}\leftarrow$ 随机小批量，含 $M$ 个数据点（从全数据集抽取）
$\boldsymbol{\epsilon}\leftarrow$ 从噪声分布 $p(\boldsymbol{\epsilon})$ 随机采样
$\mathbf{g}\leftarrow\nabla_{\boldsymbol{\theta},\boldsymbol{\phi}}\widetilde{\mathcal{L}}^{M}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{X}^{M},\boldsymbol{\epsilon})$（小批量估计器 (8) 的梯度）
$\boldsymbol{\theta},\boldsymbol{\phi}\leftarrow$ 用梯度 $\mathbf{g}$ 更新参数（例如 SGD 或 Adagrad [DHS10]）
until 参数 $(\boldsymbol{\theta},\boldsymbol{\phi})$ 收敛
return $\boldsymbol{\theta},\boldsymbol{\phi}$

算法 1 自编码变分贝叶斯（AEVB）算法的小批量版本。可使用第 2.3 节两个 SGVB 估计器中的任一个。实验中我们使用 $M=100$、$L=1$。

2.4 The reparameterization trick

2.4 重参数化技巧

In order to solve our problem we invoked an alternative method for generating samples from $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$. The essential parameterization trick is quite simple. Let $\mathbf{z}$ be a continuous random variable, and $\mathbf{z}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ be some conditional distribution. It is then often possible to express the random variable $\mathbf{z}$ as a deterministic variable $\mathbf{z}=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$, where $\boldsymbol{\epsilon}$ is an auxiliary variable with independent marginal $p(\boldsymbol{\epsilon})$, and $g_{\boldsymbol{\phi}}(.)$ is some vector-valued function parameterized by $\boldsymbol{\phi}$.

为解决问题，我们采用了从 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 生成样本的另一种方法。本质的参数化技巧相当简单。设 $\mathbf{z}$ 为连续随机变量，且 $\mathbf{z}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 为某条件分布。则常常可以把随机变量 $\mathbf{z}$ 表达为确定性变量 $\mathbf{z}=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$，其中 $\boldsymbol{\epsilon}$ 是具有独立边际 $p(\boldsymbol{\epsilon})$ 的辅助变量，$g_{\boldsymbol{\phi}}(.)$ 是由 $\boldsymbol{\phi}$ 参数化的某个向量值函数。

This reparameterization is useful for our case since it can be used to rewrite an expectation w.r.t $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ such that the Monte Carlo estimate of the expectation is differentiable w.r.t. $\boldsymbol{\phi}$. A proof is as follows. Given the deterministic mapping $\mathbf{z}=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$ we know that $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})\prod_{i}dz_{i}=p(\boldsymbol{\epsilon})\prod_{i}d\epsilon_{i}$. Therefore[^1], $\int q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})f(\mathbf{z})\,d\mathbf{z}=\int p(\boldsymbol{\epsilon})f(\mathbf{z})\,d\boldsymbol{\epsilon}=\int p(\boldsymbol{\epsilon})f(g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x}))\,d\boldsymbol{\epsilon}$. It follows that a differentiable estimator can be constructed: $\int q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})f(\mathbf{z})\,d\mathbf{z}\simeq\frac{1}{L}\sum_{l=1}^{L}f(g_{\boldsymbol{\phi}}(\mathbf{x},\boldsymbol{\epsilon}^{(l)}))$ where $\boldsymbol{\epsilon}^{(l)}\sim p(\boldsymbol{\epsilon})$. In section 2.3 we applied this trick to obtain a differentiable estimator of the variational lower bound.

这种重参数化对我们有用，因为它可以把关于 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 的期望改写，使得期望的蒙特卡洛估计对 $\boldsymbol{\phi}$ 可微。证明如下。给定确定性映射 $\mathbf{z}=g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$，有 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})\prod_{i}dz_{i}=p(\boldsymbol{\epsilon})\prod_{i}d\epsilon_{i}$。因此[^1]，$\int q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})f(\mathbf{z})\,d\mathbf{z}=\int p(\boldsymbol{\epsilon})f(\mathbf{z})\,d\boldsymbol{\epsilon}=\int p(\boldsymbol{\epsilon})f(g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x}))\,d\boldsymbol{\epsilon}$。于是可构造可微估计器：$\int q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})f(\mathbf{z})\,d\mathbf{z}\simeq\frac{1}{L}\sum_{l=1}^{L}f(g_{\boldsymbol{\phi}}(\mathbf{x},\boldsymbol{\epsilon}^{(l)}))$，其中 $\boldsymbol{\epsilon}^{(l)}\sim p(\boldsymbol{\epsilon})$。第 2.3 节正是用该技巧得到了变分下界的可微估计器。

Take, for example, the univariate Gaussian case: let $z\sim p(z|x)=\mathcal{N}(\mu,\sigma^{2})$. In this case, a valid reparameterization is $z=\mu+\sigma\epsilon$, where $\epsilon$ is an auxiliary noise variable $\epsilon\sim\mathcal{N}(0,1)$. Therefore, $\mathbb{E}_{\mathcal{N}(z;\mu,\sigma^{2})}\left[f(z)\right]=\mathbb{E}_{\mathcal{N}(\epsilon;0,1)}\left[f(\mu+\sigma\epsilon)\right]\simeq\frac{1}{L}\sum_{l=1}^{L}f(\mu+\sigma\epsilon^{(l)})$ where $\epsilon^{(l)}\sim\mathcal{N}(0,1)$.

以一元高斯为例：设 $z\sim p(z|x)=\mathcal{N}(\mu,\sigma^{2})$。此时有效重参数化为 $z=\mu+\sigma\epsilon$，其中 $\epsilon$ 为辅助噪声变量 $\epsilon\sim\mathcal{N}(0,1)$。因此 $\mathbb{E}_{\mathcal{N}(z;\mu,\sigma^{2})}\left[f(z)\right]=\mathbb{E}_{\mathcal{N}(\epsilon;0,1)}\left[f(\mu+\sigma\epsilon)\right]\simeq\frac{1}{L}\sum_{l=1}^{L}f(\mu+\sigma\epsilon^{(l)})$，其中 $\epsilon^{(l)}\sim\mathcal{N}(0,1)$。

For which $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ can we choose such a differentiable transformation $g_{\boldsymbol{\phi}}(.)$ and auxiliary variable $\boldsymbol{\epsilon}\sim p(\boldsymbol{\epsilon})$? Three basic approaches are:

对哪些 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 我们可以选择这样的可微变换 $g_{\boldsymbol{\phi}}(.)$ 与辅助变量 $\boldsymbol{\epsilon}\sim p(\boldsymbol{\epsilon})$？三种基本途径是：

1. Tractable inverse CDF. In this case, let $\boldsymbol{\epsilon}\sim\mathcal{U}(\mathbf{0},\mathbf{I})$, and let $g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$ be the inverse CDF of $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$. Examples: Exponential, Cauchy, Logistic, Rayleigh, Pareto, Weibull, Reciprocal, Gompertz, Gumbel and Erlang distributions.
2. Analogous to the Gaussian example, for any ”location-scale” family of distributions we can choose the standard distribution (with $\text{location}=0$, $\text{scale}=1$) as the auxiliary variable $\boldsymbol{\epsilon}$, and let $g(.)=\text{location}+\text{scale}\cdot\boldsymbol{\epsilon}$. Examples: Laplace, Elliptical, Student’s t, Logistic, Uniform, Triangular and Gaussian distributions.
3. Composition: It is often possible to express random variables as different transformations of auxiliary variables. Examples: Log-Normal (exponentiation of normally distributed variable), Gamma (a sum over exponentially distributed variables), Dirichlet (weighted sum of Gamma variates), Beta, Chi-Squared, and F distributions.

1. 可处理的逆 CDF。此时令 $\boldsymbol{\epsilon}\sim\mathcal{U}(\mathbf{0},\mathbf{I})$，并令 $g_{\boldsymbol{\phi}}(\boldsymbol{\epsilon},\mathbf{x})$ 为 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 的逆 CDF。例子：指数、Cauchy、Logistic、Rayleigh、Pareto、Weibull、倒数、Gompertz、Gumbel 与 Erlang 分布。
2. 类似于高斯例子，对任何“位置-尺度”族，可选标准分布（location=0，scale=1）作为辅助变量 $\boldsymbol{\epsilon}$，并令 $g(.)=\mathrm{location}+\mathrm{scale}\cdot\boldsymbol{\epsilon}$。例子：Laplace、椭圆、Student t、Logistic、均匀、三角与高斯分布。
3. 复合：常常可以把随机变量表达为辅助变量的不同变换。例子：对数正态（正态变量的指数）、Gamma（指数变量之和）、Dirichlet（Gamma 变量的加权和）、Beta、卡方与 F 分布。

When all three approaches fail, good approximations to the inverse CDF exist requiring computations with time complexity comparable to the PDF (see e.g.  [Dev86] for some methods).

当三种途径都失败时，存在与 PDF 时间复杂度相当的良好逆 CDF 近似（例如见 [Dev86] 中的一些方法）。

3 Example: Variational Auto-Encoder

3 示例：变分自编码器

In this section we’ll give an example where we use a neural network for the probabilistic encoder $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ (the approximation to the posterior of the generative model $p_{\boldsymbol{\theta}}(\mathbf{x},\mathbf{z})$) and where the parameters $\boldsymbol{\phi}$ and $\boldsymbol{\theta}$ are optimized jointly with the AEVB algorithm.

本节给出一个例子：用神经网络作为概率编码器 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$（对生成模型 $p_{\boldsymbol{\theta}}(\mathbf{x},\mathbf{z})$ 后验的近似），并用 AEVB 算法联合优化参数 $\boldsymbol{\phi}$ 与 $\boldsymbol{\theta}$。

Let the prior over the latent variables be the centered isotropic multivariate Gaussian $p_{\boldsymbol{\theta}}(\mathbf{z})=\mathcal{N}(\mathbf{z};\mathbf{0},\mathbf{I})$. Note that in this case, the prior lacks parameters. We let $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ be a multivariate Gaussian (in case of real-valued data) or Bernoulli (in case of binary data) whose distribution parameters are computed from $\mathbf{z}$ with a MLP (a fully-connected neural network with a single hidden layer, see appendix C). Note the true posterior $p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x})$ is in this case intractable. While there is much freedom in the form $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$, we’ll assume the true (but intractable) posterior takes on a approximate Gaussian form with an approximately diagonal covariance. In this case, we can let the variational approximate posterior be a multivariate Gaussian with a diagonal covariance structure[^2]:

设隐变量先验为中心各向同性多元高斯 $p_{\boldsymbol{\theta}}(\mathbf{z})=\mathcal{N}(\mathbf{z};\mathbf{0},\mathbf{I})$。注意此时先验没有参数。令 $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ 为多元高斯（实值数据）或 Bernoulli（二值数据），其分布参数由 MLP（带单个隐层的全连接神经网络，见附录 C）从 $\mathbf{z}$ 算出。注意此时真实后验 $p_{\boldsymbol{\theta}}(\mathbf{z}|\mathbf{x})$ 不可积。尽管 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 形式很自由，我们假定真实（但不可积）后验近似呈对角协方差的高斯形式。此时可令变分近似后验为具有对角协方差结构的多元高斯[^2]：

$$\displaystyle\log q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)}) \\ \displaystyle=\log\mathcal{N}(\mathbf{z};\boldsymbol{\mu}^{(i)},\boldsymbol{\sigma}^{2(i)}\mathbf{I}) \tag{9}$$

where the mean and s.d. of the approximate posterior, $\boldsymbol{\mu}^{(i)}$ and $\boldsymbol{\sigma}^{(i)}$, are outputs of the encoding MLP, i.e. nonlinear functions of datapoint $\mathbf{x}^{(i)}$ and the variational parameters $\boldsymbol{\phi}$ (see appendix C).

其中近似后验的均值与标准差 $\boldsymbol{\mu}^{(i)}$ 与 $\boldsymbol{\sigma}^{(i)}$ 是编码 MLP 的输出，即数据点 $\mathbf{x}^{(i)}$ 与变分参数 $\boldsymbol{\phi}$ 的非线性函数（见附录 C）。

As explained in section 2.4, we sample from the posterior $\mathbf{z}^{(i,l)}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})$ using $\mathbf{z}^{(i,l)}=g_{\boldsymbol{\phi}}(\mathbf{x}^{(i)},\boldsymbol{\epsilon}^{(l)})=\boldsymbol{\mu}^{(i)}+\boldsymbol{\sigma}^{(i)}\odot\boldsymbol{\epsilon}^{(l)}$ where $\boldsymbol{\epsilon}^{(l)}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$. With $\odot$ we signify an element-wise product. In this model both $p_{\boldsymbol{\theta}}(\mathbf{z})$ (the prior) and $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ are Gaussian; in this case, we can use the estimator of eq. (7) where the KL divergence can be computed and differentiated without estimation (see appendix B). The resulting estimator for this model and datapoint $\mathbf{x}^{(i)}$ is:

如第 2.4 节所释，我们用 $\mathbf{z}^{(i,l)}=g_{\boldsymbol{\phi}}(\mathbf{x}^{(i)},\boldsymbol{\epsilon}^{(l)})=\boldsymbol{\mu}^{(i)}+\boldsymbol{\sigma}^{(i)}\odot\boldsymbol{\epsilon}^{(l)}$（其中 $\boldsymbol{\epsilon}^{(l)}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$）从后验 $\mathbf{z}^{(i,l)}\sim q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})$ 采样。$\odot$ 表示逐元素乘积。该模型中 $p_{\boldsymbol{\theta}}(\mathbf{z})$（先验）与 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 均为高斯；此时可用式 (7) 的估计器，KL 散度无需估计即可计算并微分（见附录 B）。对该模型与数据点 $\mathbf{x}^{(i)}$，所得估计器为：

$$\displaystyle\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{x}^{(i)}) \\ \displaystyle\simeq\frac{1}{2}\sum_{j=1}^{J}\left(1+\log((\sigma_{j}^{(i)})^{2})-(\mu_{j}^{(i)})^{2}-(\sigma_{j}^{(i)})^{2}\right)+\frac{1}{L}\sum_{l=1}^{L}\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z}^{(i,l)}) \\ \displaystyle\text{where\quad}\mathbf{z}^{(i,l)} \\ \displaystyle=\boldsymbol{\mu}^{(i)}+\boldsymbol{\sigma}^{(i)}\odot\boldsymbol{\epsilon}^{(l)}\text{\quad and \quad}\boldsymbol{\epsilon}^{(l)}\sim\mathcal{N}(0,\mathbf{I}) \tag{10}$$

As explained above and in appendix C, the decoding term $\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z}^{(i,l)})$ is a Bernoulli or Gaussian MLP, depending on the type of data we are modelling.

如上所述及附录 C，解码项 $\log p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)}|\mathbf{z}^{(i,l)})$ 是 Bernoulli 或高斯 MLP，取决于所建模数据类型。

4 Related work

4 相关工作

The wake-sleep algorithm [HDFN95] is, to the best of our knowledge, the only other on-line learning method in the literature that is applicable to the same general class of continuous latent variable models. Like our method, the wake-sleep algorithm employs a recognition model that approximates the true posterior. A drawback of the wake-sleep algorithm is that it requires a concurrent optimization of two objective functions, which together do not correspond to optimization of (a bound of) the marginal likelihood. An advantage of wake-sleep is that it also applies to models with discrete latent variables. Wake-Sleep has the same computational complexity as AEVB per datapoint.

据我们所知，wake-sleep 算法 [HDFN95] 是文献中唯一另一种适用于同一大类连续隐变量模型的在线学习方法。与我们的方法一样，wake-sleep 使用逼近真实后验的识别模型。其缺点是需要同时优化两个目标函数，它们合在一起并不对应于（边际似然下界的）优化。其优点是也适用于离散隐变量模型。wake-sleep 与 AEVB 每个数据点的计算复杂度相同。

Stochastic variational inference [HBWP13] has recently received increasing interest. Recently, [BJP12] introduced a control variate schemes to reduce the high variance of the naïve gradient estimator discussed in section 2.1, and applied to exponential family approximations of the posterior. In [RGB13] some general methods, i.e. a control variate scheme, were introduced for reducing the variance of the original gradient estimator. In [SK13], a similar reparameterization as in this paper was used in an efficient version of a stochastic variational inference algorithm for learning the natural parameters of exponential-family approximating distributions.

随机变分推断 [HBWP13] 近来受到越来越多关注。[BJP12] 引入控制变量方案以降低第 2.1 节讨论的朴素梯度估计器的高方差，并应用于后验的指数族近似。[RGB13] 引入了一些降低原始梯度估计器方差的一般方法（即控制变量方案）。[SK13] 在学习指数族近似分布自然参数的高效随机变分推断算法中，使用了与本文类似的重参数化。

The AEVB algorithm exposes a connection between directed probabilistic models (trained with a variational objective) and auto-encoders. A connection between linear auto-encoders and a certain class of generative linear-Gaussian models has long been known. In  [Row98] it was shown that PCA corresponds to the maximum-likelihood (ML) solution of a special case of the linear-Gaussian model with a prior $p(\mathbf{z})=\mathcal{N}(0,\mathbf{I})$ and a conditional distribution $p(\mathbf{x}|\mathbf{z})=\mathcal{N}(\mathbf{x};\mathbf{W}\mathbf{z},\epsilon\mathbf{I})$, specifically the case with infinitesimally small $\epsilon$.

AEVB 算法揭示了（用变分目标训练的）有向概率模型与自动编码器之间的联系。线性自动编码器与某类生成线性-高斯模型之间的联系早已为人所知。[Row98] 证明 PCA 对应于带先验 $p(\mathbf{z})=\mathcal{N}(0,\mathbf{I})$ 与条件分布 $p(\mathbf{x}|\mathbf{z})=\mathcal{N}(\mathbf{x};\mathbf{W}\mathbf{z},\epsilon\mathbf{I})$ 的线性-高斯模型特例（具体是 $\epsilon$ 无穷小）的最大似然解。

In relevant recent work on autoencoders [VLL+10] it was shown that the training criterion of unregularized autoencoders corresponds to maximization of a lower bound (see the infomax principle [Lin89]) of the mutual information between input $X$ and latent representation $Z$. Maximizing (w.r.t. parameters) of the mutual information is equivalent to maximizing the conditional entropy, which is lower bounded by the expected loglikelihood of the data under the autoencoding model [VLL+10], i.e. the negative reconstrution error. However, it is well known that this reconstruction criterion is in itself not sufficient for learning useful representations [BCV13]. Regularization techniques have been proposed to make autoencoders learn useful representations, such as denoising, contractive and sparse autoencoder variants  [BCV13]. The SGVB objective contains a regularization term dictated by the variational bound (e.g. eq. (10)), lacking the usual nuisance regularization hyperparameter required to learn useful representations. Related are also encoder-decoder architectures such as the predictive sparse decomposition (PSD) [KRL08], from which we drew some inspiration. Also relevant are the recently introduced Generative Stochastic Networks [BTL13] where noisy auto-encoders learn the transition operator of a Markov chain that samples from the data distribution. In [SL10] a recognition model was employed for efficient learning with Deep Boltzmann Machines. These methods are targeted at either unnormalized models (i.e. undirected models like Boltzmann machines) or limited to sparse coding models, in contrast to our proposed algorithm for learning a general class of directed probabilistic models.

在自动编码器的相关近期工作中，[VLL+10] 证明无正则自动编码器的训练准则对应于输入 $X$ 与潜在表示 $Z$ 互信息下界的最大化（见信息最大化原理 [Lin89]）。最大化（对参数）互信息等价于最大化条件熵，而条件熵又被自动编码模型下数据的期望对数似然下界，即负重构误差 [VLL+10]。然而众所周知，仅靠重构准则不足以学到有用表示 [BCV13]。已有去噪、收缩与稀疏等自动编码器变体等正则技术使自动编码器学习有用表示 [BCV13]。SGVB 目标包含由变分下界规定的正则项（例如式 (10)），无需学习有用表示时常有的讨厌正则超参数。相关的还有预测稀疏分解（PSD）[KRL08] 这类编码器-解码器架构，我们从中获得了一些启发。同样相关的是近来提出的生成随机网络 [BTL13]，其中带噪自动编码器学习从数据分布采样的马尔可夫链转移算子。[SL10] 使用识别模型对深度玻尔兹曼机做高效学习。这些方法要么针对未归一化模型（即玻尔兹曼机这类无向模型），要么限于稀疏编码模型，而我们提出的是学习一般有向概率模型类的算法。

The recently proposed DARN method  [GMW13], also learns a directed probabilistic model using an auto-encoding structure, however their method applies to binary latent variables. Even more recently,  [RMW14] also make the connection between auto-encoders, directed proabilistic models and stochastic variational inference using the reparameterization trick we describe in this paper. Their work was developed independently of ours and provides an additional perspective on AEVB.

近来提出的 DARN 方法 [GMW13] 也用自动编码结构学习有向概率模型，但其方法适用于二值隐变量。更近地，[RMW14] 也建立了自动编码器、有向概率模型与使用本文重参数化技巧的随机变分推断之间的联系。他们的工作与我们独立发展，为 AEVB 提供了额外视角。

Figure 2: Comparison of our AEVB method to the wake-sleep algorithm, in terms of optimizing the lower bound, for different dimensionality of latent space ($N_{\mathbf{z}}$). Our method converged considerably faster and reached a better solution in all experiments. Interestingly enough, more latent variables does not result in more overfitting, which is explained by the regularizing effect of the lower bound. Vertical axis: the estimated average variational lower bound per datapoint. The estimator variance was small ($<1$) and omitted. Horizontal axis: amount of training points evaluated. Computation took around 20-40 minutes per million training samples with a Intel Xeon CPU running at an effective 40 GFLOPS.

图 2：就可分下界优化，在不同隐空间维数（$N_{\mathbf{z}}$）下比较我们的 AEVB 方法与 wake-sleep 算法。所有实验中我们的方法收敛明显更快并达到更好解。有趣的是，更多隐变量并未导致更多过拟合，这可由下界的正则效应解释。纵轴：每个数据点估计的平均变分下界。估计器方差很小（$<1$）故省略。横轴：已评估的训练点数。在有效约 40 GFLOPS 的 Intel Xeon CPU 上，每百万训练样本计算约需 20–40 分钟。

5 Experiments

5 实验

We trained generative models of images from the MNIST and Frey Face datasets[^3] and compared learning algorithms in terms of the variational lower bound, and the estimated marginal likelihood.

我们在 MNIST 与 Frey Face 数据集[^3] 上训练图像生成模型，并就可分下界与估计的边际似然比较学习算法。

The generative model (encoder) and variational approximation (decoder) from section 3 were used, where the described encoder and decoder have an equal number of hidden units. Since the Frey Face data are continuous, we used a decoder with Gaussian outputs, identical to the encoder, except that the means were constrained to the interval $(0,1)$ using a sigmoidal activation function at the decoder output. Note that with hidden units we refer to the hidden layer of the neural networks of the encoder and decoder.

使用第 3 节的生成模型（解码器）与变分近似（编码器），其中所述编码器与解码器具有相同数量的隐单元。由于 Frey Face 数据是连续的，我们使用带高斯输出的解码器，与编码器相同，只是均值通过解码器输出端的 sigmoid 激活约束在区间 $(0,1)$。注意这里的隐单元指编码器与解码器神经网络的隐层。

Parameters are updated using stochastic gradient ascent where gradients are computed by differentiating the lower bound estimator $\nabla_{\boldsymbol{\theta},\boldsymbol{\phi}}\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{X})$ (see algorithm  1), plus a small weight decay term corresponding to a prior $p(\boldsymbol{\theta})=\mathcal{N}(0,\mathbf{I})$. Optimization of this objective is equivalent to approximate MAP estimation, where the likelihood gradient is approximated by the gradient of the lower bound.

参数用随机梯度上升更新，梯度通过对下界估计器微分得到 $\nabla_{\boldsymbol{\theta},\boldsymbol{\phi}}\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\mathbf{X})$（见算法 1），外加对应先验 $p(\boldsymbol{\theta})=\mathcal{N}(0,\mathbf{I})$ 的小权重衰减项。优化该目标等价于近似 MAP 估计，其中似然梯度由下界梯度近似。

We compared performance of AEVB to the wake-sleep algorithm [HDFN95]. We employed the same encoder (also called recognition model) for the wake-sleep algorithm and the variational auto-encoder. All parameters, both variational and generative, were initialized by random sampling from $\mathcal{N}(0,0.01)$, and were jointly stochastically optimized using the MAP criterion. Stepsizes were adapted with Adagrad [DHS10]; the Adagrad global stepsize parameters were chosen from {0.01, 0.02, 0.1} based on performance on the training set in the first few iterations. Minibatches of size $M=100$ were used, with $L=1$ samples per datapoint.

我们将 AEVB 的表现与 wake-sleep 算法 [HDFN95] 比较。wake-sleep 与变分自编码器使用相同的编码器（也称识别模型）。所有参数（变分与生成）均从 $\mathcal{N}(0,0.01)$ 随机初始化，并用 MAP 准则联合随机优化。步长用 Adagrad [DHS10] 自适应；Adagrad 全局步长参数从 $\{0.01, 0.02, 0.1\}$ 中按前几次迭代在训练集上的表现选取。使用大小 $M=100$ 的小批量。

Likelihood lower bound

似然下界

We trained generative models (decoders) and corresponding encoders (a.k.a. recognition models) having $500$ hidden units in case of MNIST, and $200$ hidden units in case of the Frey Face dataset (to prevent overfitting, since it is a considerably smaller dataset). The chosen number of hidden units is based on prior literature on auto-encoders, and the relative performance of different algorithms was not very sensitive to these choices. Figure 2 shows the results when comparing the lower bounds. Interestingly, superfluous latent variables did not result in overfitting, which is explained by the regularizing nature of the variational bound.

我们训练生成模型（解码器）与对应编码器（即识别模型），MNIST 用 500 个隐单元，Frey Face 用 200 个隐单元（防止过拟合，因其数据集小得多）。隐单元数基于自动编码器的先前文献选取，不同算法的相对表现对这些选择不太敏感。图 2 给出下界比较结果。有趣的是，多余的隐变量并未导致过拟合。

Figure 3: Comparison of AEVB to the wake-sleep algorithm and Monte Carlo EM, in terms of the estimated marginal likelihood, for a different number of training points. Monte Carlo EM is not an on-line algorithm, and (unlike AEVB and the wake-sleep method) can’t be applied efficiently for the full MNIST dataset.

图 3：就可估计边际似然，在不同训练点数下比较 AEVB、wake-sleep 与蒙特卡洛 EM。蒙特卡洛 EM 不是在线算法，（与 AEVB 和 wake-sleep 不同）不能高效应用于完整 MNIST 数据集。

Marginal likelihood

边际似然

For very low-dimensional latent space it is possible to estimate the marginal likelihood of the learned generative models using an MCMC estimator. More information about the marginal likelihood estimator is available in the appendix. For the encoder and decoder we again used neural networks, this time with 100 hidden units, and 3 latent variables; for higher dimensional latent space the estimates became unreliable. Again, the MNIST dataset was used. The AEVB and Wake-Sleep methods were compared to Monte Carlo EM (MCEM) with a Hybrid Monte Carlo (HMC) [DKPR87] sampler; details are in the appendix. We compared the convergence speed for the three algorithms, for a small and large training set size. Results are in figure 3.

对很低维的隐空间，可以用 MCMC 估计器估计所学生成模型的边际似然。关于边际似然估计器的更多信息见附录。编码器与解码器再次使用神经网络，此次 100 个隐单元、3 个隐变量；更高维隐空间时估计变得不可靠。仍使用 MNIST。将 AEVB 与 wake-sleep 与带 Hybrid Monte Carlo（HMC）[DKPR87] 采样器的蒙特卡洛 EM（MCEM）比较。

Visualisation of high-dimensional data

高维数据的可视化

If we choose a low-dimensional latent space (e.g. 2D), we can use the learned encoders (recognition model) to project high-dimensional data to a low-dimensional manifold. See appendix A for visualisations of the 2D latent manifolds for the MNIST and Frey Face datasets.

若选择低维隐空间（例如 2 维），可用学到的编码器（识别模型）把高维数据投影到低维流形。MNIST 与 Frey Face 数据集的二维隐流形可视化见附录 A。

6 Conclusion

6 结论

We have introduced a novel estimator of the variational lower bound, Stochastic Gradient VB (SGVB), for efficient approximate inference with continuous latent variables. The proposed estimator can be straightforwardly differentiated and optimized using standard stochastic gradient methods. For the case of i.i.d. datasets and continuous latent variables per datapoint we introduce an efficient algorithm for efficient inference and learning, Auto-Encoding VB (AEVB), that learns an approximate inference model using the SGVB estimator. The theoretical advantages are reflected in experimental results.

我们引入了变分下界的新估计器——随机梯度变分贝叶斯（SGVB），用于带连续隐变量的高效近似推断。该估计器可直接微分，并用标准随机梯度方法优化。针对 i.i.d. 数据集且每个数据点带连续隐变量的情形，我们引入高效推断与学习算法——自编码变分贝叶斯（AEVB），它用 SGVB 估计器学习近似推断模型。理论优势在实验结果中得到体现。

7 Future work

7 未来工作

Since the SGVB estimator and the AEVB algorithm can be applied to almost any inference and learning problem with continuous latent variables, there are plenty of future directions: (i) learning hierarchical generative architectures with deep neural networks (e.g. convolutional networks) used for the encoders and decoders, trained jointly with AEVB; (ii) time-series models (i.e. dynamic Bayesian networks); (iii) application of SGVB to the global parameters; (iv) supervised models with latent variables, useful for learning complicated noise distributions.

由于 SGVB 估计器与 AEVB 算法几乎可用于任何带连续隐变量的推断与学习问题，未来方向很多：(i) 用深度神经网络（例如卷积网络）作编码器与解码器，并用 AEVB 联合训练的层次生成架构；(ii) 时间序列模型（即动态贝叶斯网络）；(iii) 将 SGVB 应用于全局参数；(iv) 带隐变量的监督模型，有助于学习复杂噪声分布。

References

参考文献

(a) Learned Frey Face manifold

(a) 学到的 Frey Face 流形

(a) 2-D latent space

(a) 二维隐空间

[BCV13] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. 2013.

[BJP12] David M Blei, Michael I Jordan, and John W Paisley. Variational Bayesian inference with Stochastic Search. In Proceedings of the 29th International Conference on Machine Learning (ICML-12) , pages 1367–1374, 2012.

[BTL13] Yoshua Bengio and Éric Thibodeau-Laufer. Deep generative stochastic networks trainable by backprop. arXiv preprint arXiv:1306.1091 , 2013.

[Dev86] Luc Devroye. Sample-based non-uniform random variate generation. In Proceedings of the 18th conference on Winter simulation , pages 260–265. ACM, 1986.

[DHS10] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research , 12:2121–2159, 2010.

[DKPR87] Simon Duane, Anthony D Kennedy, Brian J Pendleton, and Duncan Roweth. Hybrid monte carlo. Physics letters B , 195(2):216–222, 1987.

[GMW13] Karol Gregor, Andriy Mnih, and Daan Wierstra. Deep autoregressive networks. arXiv preprint arXiv:1310.8499 , 2013.

[HBWP13] Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. Stochastic variational inference. The Journal of Machine Learning Research , 14(1):1303–1347, 2013.

[HDFN95] Geoffrey E Hinton, Peter Dayan, Brendan J Frey, and Radford M Neal. The” wake-sleep” algorithm for unsupervised neural networks. SCIENCE , pages 1158–1158, 1995.

[KRL08] Koray Kavukcuoglu, Marc’Aurelio Ranzato, and Yann LeCun. Fast inference in sparse coding algorithms with applications to object recognition. Technical Report CBLL-TR-2008-12-01, Computational and Biological Learning Lab, Courant Institute, NYU, 2008.

[Lin89] Ralph Linsker. An application of the principle of maximum information preservation to linear systems . Morgan Kaufmann Publishers Inc., 1989.

[RGB13] Rajesh Ranganath, Sean Gerrish, and David M Blei. Black Box Variational Inference. arXiv preprint arXiv:1401.0118 , 2013.

[RMW14] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic back-propagation and variational inference in deep latent gaussian models. arXiv preprint arXiv:1401.4082 , 2014.

[Row98] Sam Roweis. EM algorithms for PCA and SPCA. Advances in neural information processing systems , pages 626–632, 1998.

[SK13] Tim Salimans and David A Knowles. Fixed-form variational posterior approximation through stochastic linear regression. Bayesian Analysis , 8(4), 2013.

[SL10] Ruslan Salakhutdinov and Hugo Larochelle. Efficient learning of deep boltzmann machines. In International Conference on Artificial Intelligence and Statistics , pages 693–700, 2010.

[VLL + 10] Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research , 9999:3371–3408, 2010.

Appendix A Visualisations

附录 A 可视化

See figures 4 and 5 for visualisations of latent space and corresponding observed space of models learned with SGVB.

用 SGVB 所学模型的隐空间与对应观测空间可视化见图 4 与图 5。

Appendix B Solution of $-D_{KL}(q_{\boldsymbol{\phi}}(\mathbf{z})||p_{\boldsymbol{\theta}}(\mathbf{z}))$, Gaussian case

附录 B $-D_{KL}(q_{\boldsymbol{\phi}}(\mathbf{z})||p_{\boldsymbol{\theta}}(\mathbf{z}))$ 的高斯情形解析解

The variational lower bound (the objective to be maximized) contains a KL term that can often be integrated analytically. Here we give the solution when both the prior $p_{\boldsymbol{\theta}}(\mathbf{z})=\mathcal{N}(0,\mathbf{I})$ and the posterior approximation $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})$ are Gaussian. Let $J$ be the dimensionality of $\mathbf{z}$. Let $\boldsymbol{\mu}$ and $\boldsymbol{\sigma}$ denote the variational mean and s.d. evaluated at datapoint $i$, and let $\mu_{j}$ and $\sigma_{j}$ simply denote the $j$-th element of these vectors. Then:

变分下界（待最大化的目标）包含一个常常可解析积分的 KL 项。这里给出先验 $p_{\boldsymbol{\theta}}(\mathbf{z})=\mathcal{N}(0,\mathbf{I})$ 与后验近似 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x}^{(i)})$ 均为高斯时的解。设 $J$ 为 $\mathbf{z}$ 的维数。令 $\boldsymbol{\mu}$ 与 $\boldsymbol{\sigma}$ 表示在数据点 $i$ 处评估的变分均值与标准差，并令 $\mu_{j}$ 与 $\sigma_{j}$ 表示这些向量的第 $j$ 个元素。则：

$$\displaystyle\int q_{\boldsymbol{\theta}}(\mathbf{z})\log p(\mathbf{z})\,d\mathbf{z} \\ \displaystyle=\int\mathcal{N}(\mathbf{z};\boldsymbol{\mu},\boldsymbol{\sigma}^{2})\log\mathcal{N}(\mathbf{z};\mathbf{0},\mathbf{I})\,d\mathbf{z} \\ \displaystyle=-\frac{J}{2}\log(2\pi)-\frac{1}{2}\sum_{j=1}^{J}(\mu_{j}^{2}+\sigma_{j}^{2})$$

And:

并且：

$$\displaystyle\int q_{\boldsymbol{\theta}}(\mathbf{z})\log q_{\boldsymbol{\theta}}(\mathbf{z})\,d\mathbf{z} \\ \displaystyle=\int\mathcal{N}(\mathbf{z};\boldsymbol{\mu},\boldsymbol{\sigma}^{2})\log\mathcal{N}(\mathbf{z};\boldsymbol{\mu},\boldsymbol{\sigma}^{2})\,d\mathbf{z} \\ \displaystyle=-\frac{J}{2}\log(2\pi)-\frac{1}{2}\sum_{j=1}^{J}(1+\log\sigma^{2}_{j})$$

Therefore:

因此：

$$\displaystyle-D_{KL}((q_{\boldsymbol{\phi}}(\mathbf{z})||p_{\boldsymbol{\theta}}(\mathbf{z})) \\ \displaystyle=\int q_{\boldsymbol{\theta}}(\mathbf{z})\left(\log p_{\boldsymbol{\theta}}(\mathbf{z})-\log q_{\boldsymbol{\theta}}(\mathbf{z})\right)\,d\mathbf{z} \\ \displaystyle=\frac{1}{2}\sum_{j=1}^{J}\left(1+\log((\sigma_{j})^{2})-(\mu_{j})^{2}-(\sigma_{j})^{2}\right)$$

When using a recognition model $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ then $\boldsymbol{\mu}$ and s.d. $\boldsymbol{\sigma}$ are simply functions of $\mathbf{x}$ and the variational parameters $\boldsymbol{\phi}$, as exemplified in the text.

当使用识别模型 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 时，$\boldsymbol{\mu}$ 与标准差 $\boldsymbol{\sigma}$ 只是 $\mathbf{x}$ 与变分参数 $\boldsymbol{\phi}$ 的函数，如正文示例。

Appendix C MLP’s as probabilistic encoders and decoders

附录 C 作为概率编码器与解码器的 MLP

In variational auto-encoders, neural networks are used as probabilistic encoders and decoders. There are many possible choices of encoders and decoders, depending on the type of data and model. In our example we used relatively simple neural networks, namely multi-layered perceptrons (MLPs). For the encoder we used a MLP with Gaussian output, while for the decoder we used MLPs with either Gaussian or Bernoulli outputs, depending on the type of data.

在变分自编码器中，神经网络被用作概率编码器与解码器。视数据类型与模型而定，编码器与解码器有许多可能选择。我们的例子使用相对简单的神经网络，即多层感知机（MLP）。编码器使用带高斯输出的 MLP，解码器视数据类型使用带高斯或 Bernoulli 输出的 MLP。

C.1 Bernoulli MLP as decoder

C.1 Bernoulli MLP 作解码器

In this case let $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ be a multivariate Bernoulli whose probabilities are computed from $\mathbf{z}$ with a fully-connected neural network with a single hidden layer:

此时令 $p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ 为多元 Bernoulli，其概率由带单个隐层的全连接神经网络从 $\mathbf{z}$ 计算：

$$\displaystyle\log p(\mathbf{x}|\mathbf{z}) \\ \displaystyle=\sum_{i=1}^{D}x_{i}\log y_{i}+(1-x_{i})\cdot\log(1-y_{i}) \\ \displaystyle\text{where\,\,\,}\mathbf{y} \\ \displaystyle=f_{\sigma}(\mathbf{W}_{2}\tanh(\mathbf{W}_{1}\mathbf{z}+\mathbf{b}_{1})+\mathbf{b}_{2}) \tag{11}$$

where $f_{\sigma}(.)$ is the elementwise sigmoid activation function, and where $\boldsymbol{\theta}=\{\mathbf{W}_{1},\mathbf{W}_{2},\mathbf{b}_{1},\mathbf{b}_{2}\}$ are the weights and biases of the MLP.

其中 $f_{\sigma}(.)$ 为逐元素 sigmoid 激活函数，$\boldsymbol{\theta}=\{\mathbf{W}_{1},\mathbf{W}_{2},\mathbf{b}_{1},\mathbf{b}_{2}\}$ 为 MLP 的权重与偏置。

C.2 Gaussian MLP as encoder or decoder

C.2 高斯 MLP 作编码器或解码器

In this case let encoder or decoder be a multivariate Gaussian with a diagonal covariance structure:

此时令编码器或解码器为具有对角协方差结构的多元高斯：

$$\displaystyle\log p(\mathbf{x}|\mathbf{z}) \\ \displaystyle=\log\mathcal{N}(\mathbf{x};\boldsymbol{\mu},\boldsymbol{\sigma}^{2}\mathbf{I}) \\ \displaystyle\text{where\,\,\,}\boldsymbol{\mu} \\ \displaystyle=\mathbf{W}_{4}\mathbf{h}+\mathbf{b}_{4} \\ \displaystyle\log\boldsymbol{\sigma}^{2} \\ \displaystyle=\mathbf{W}_{5}\mathbf{h}+\mathbf{b}_{5} \\ \displaystyle\mathbf{h} \\ \displaystyle=\tanh(\mathbf{W}_{3}\mathbf{z}+\mathbf{b}_{3}) \tag{12}$$

where $\{\mathbf{W}_{3},\mathbf{W}_{4},\mathbf{W}_{5},\mathbf{b}_{3},\mathbf{b}_{4},\mathbf{b}_{5}\}$ are the weights and biases of the MLP and part of $\boldsymbol{\theta}$ when used as decoder. Note that when this network is used as an encoder $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$, then $\mathbf{z}$ and $\mathbf{x}$ are swapped, and the weights and biases are variational parameters $\boldsymbol{\phi}$.

其中 $\{\mathbf{W}_{3},\mathbf{W}_{4},\mathbf{W}_{5},\mathbf{b}_{3},\mathbf{b}_{4},\mathbf{b}_{5}\}$ 为 MLP 的权重与偏置，用作解码器时属于 $\boldsymbol{\theta}$。注意当该网络用作编码器 $q_{\boldsymbol{\phi}}(\mathbf{z}|\mathbf{x})$ 时，$\mathbf{z}$ 与 $\mathbf{x}$ 对调，权重与偏置为变分参数 $\boldsymbol{\phi}$。

Appendix D Marginal likelihood estimator

附录 D 边际似然估计器

We derived the following marginal likelihood estimator that produces good estimates of the marginal likelihood as long as the dimensionality of the sampled space is low (less then 5 dimensions), and sufficient samples are taken. Let $p_{\boldsymbol{\theta}}(\mathbf{x},\mathbf{z})=p_{\boldsymbol{\theta}}(\mathbf{z})p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ be the generative model we are sampling from, and for a given datapoint $\mathbf{x}^{(i)}$ we would like to estimate the marginal likelihood $p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)})$.

我们推导了如下边际似然估计器：只要采样空间维数较低（小于 5 维）且样本足够，就能给出良好的边际似然估计。设 $p_{\boldsymbol{\theta}}(\mathbf{x},\mathbf{z})=p_{\boldsymbol{\theta}}(\mathbf{z})p_{\boldsymbol{\theta}}(\mathbf{x}|\mathbf{z})$ 为所采样的生成模型，对给定数据点 $\mathbf{x}^{(i)}$ 我们希望估计边际似然 $p_{\boldsymbol{\theta}}(\mathbf{x}^{(i)})$。