Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Show, Attend and Tell：带视觉注意力的神经图像描述生成

Kelvin Xu kelvin.xu@umontreal.ca

Jimmy Lei Ba jimmy@psi.utoronto.ca

Ryan Kiros rkiros@cs.toronto.edu

Kyunghyun Cho kyunghyun.cho@umontreal.ca

Aaron Courville aaron.courville@umontreal.ca

Ruslan Salakhutdinov rsalakhu@cs.toronto.edu

Richard S. Zemel zemel@cs.toronto.edu

Yoshua Bengio find-me@the.web

Abstract

摘要

Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe how we can train this model in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound. We also show through visualization how the model is able to automatically learn to fix its gaze on salient objects while generating the corresponding words in the output sequence. We validate the use of attention with state-of-the-art performance on three benchmark datasets: Flickr8k, Flickr30k and MS COCO.

受机器翻译与目标检测近期工作的启发，我们提出一种基于注意力的模型，能够自动学会描述图像内容。我们说明如何用标准反向传播以确定性方式训练该模型，以及如何通过最大化变分下界以随机方式训练。我们还通过可视化展示：模型在生成输出序列中对应词语时，能够自动把目光落到显著物体上。我们在三个基准数据集上验证了注意力的作用，并取得了当时最优表现：Flickr8k、Flickr30k 与 MS COCO。

1 Introduction

1 引言

Automatically generating captions of an image is a task very close to the heart of scene understanding — one of the primary goals of computer vision. Not only must caption generation models be powerful enough to solve the computer vision challenges of determining which objects are in an image, but they must also be capable of capturing and expressing their relationships in a natural language. For this reason, caption generation has long been viewed as a difficult problem. It is a very important challenge for machine learning algorithms, as it amounts to mimicking the remarkable human ability to compress huge amounts of salient visual infomation into descriptive language.

自动为图像生成描述，非常接近场景理解——计算机视觉的核心目标之一。描述生成模型不仅要足以应对“图中有哪些物体”这类视觉难题，还必须能捕捉它们之间的关系，并用自然语言表达出来。正因如此，描述生成长期被视为难题。对机器学习算法来说，这也是一项重要挑战，因为它相当于模仿人类把大量显著视觉信息压缩成描述性语言的能力。

Despite the challenging nature of this task, there has been a recent surge of research interest in attacking the image caption generation problem. Aided by advances in training neural networks (Krizhevsky et al., 2012) and large classification datasets (Russakovsky et al., 2014), recent work has significantly improved the quality of caption generation using a combination of convolutional neural networks (convnets) to obtain vectorial representation of images and recurrent neural networks to decode those representations into natural language sentences (see Sec. 2).

尽管任务困难，近期仍出现了一批攻克图像描述生成的研究。借助神经网络训练的进展（Krizhevsky 等，2012）以及大规模分类数据集（Russakovsky 等，2014），近期工作把卷积神经网络得到的图像向量表示，与把这些表示解码成自然语言句子的循环神经网络结合起来，显著提升了描述质量（见第 2 节）。

Figure 1: Our model learns a words/image alignment. The visualized attentional maps (3) are explained in section 3.1 & 5.4

图 1：我们的模型学习词与图像的对齐。可视化的注意力图（3）在第 3.1 节与第 5.4 节解释。

Figure 2: Attention over time. As the model generates each word, its attention changes to reflect the relevant parts of the image. “soft” (top row) vs “hard” (bottom row) attention. (Note that both models generated the same captions in this example.)

图 2：随时间变化的注意力。模型每生成一个词，注意力就会转到图像中相关的部分。上一行是“软”注意力，下一行是“硬”注意力。（注意本例中两个模型生成了相同的描述。）

Figure 3: Examples of attending to the correct object (white indicates the attended regions, underlines indicated the corresponding word)

图 3：注意到正确物体的例子（白色表示被注意的区域，下划线标出对应的词）

One of the most curious facets of the human visual system is the presence of attention (Rensink, 2000; Corbetta & Shulman, 2002). Rather than compress an entire image into a static representation, attention allows for salient features to dynamically come to the forefront as needed. This is especially important when there is a lot of clutter in an image. Using representations (such as those from the top layer of a convnet) that distill information in image down to the most salient objects is one effective solution that has been widely adopted in previous work. Unfortunately, this has one potential drawback of losing information which could be useful for richer, more descriptive captions. Using more low-level representation can help preserve this information. However working with these features necessitates a powerful mechanism to steer the model to information important to the task at hand.

人类视觉系统最令人好奇的特点之一，就是注意力的存在（Rensink，2000；Corbetta 与 Shulman，2002）。注意力不是把整张图压成一个静态表示，而是让显著特征在需要时动态走到前台。图像很杂乱时，这一点尤为重要。用卷积网络顶层这类表示，把图像信息提炼到最显著的物体，是以往工作广泛采用的有效办法。可惜它有一个潜在缺点：可能丢掉对更丰富、更细致描述有用的信息。使用更低层的表示有助于保留这些信息。但要处理这些特征，就需要一种有力的机制，把模型引导到当前任务真正重要的信息上。

In this paper, we describe approaches to caption generation that attempt to incorporate a form of attention with two variants: a “hard” attention mechanism and a “soft” attention mechanism. We also show how one advantage of including attention is the ability to visualize what the model “sees”. Encouraged by recent advances in caption generation and inspired by recent success in employing attention in machine translation (Bahdanau et al., 2014) and object recognition (Ba et al., 2014; Mnih et al., 2014), we investigate models that can attend to salient part of an image while generating its caption.

本文描述的描述生成方法试图引入一种注意力，并给出两个变体：“硬”注意力机制与“软”注意力机制。我们还说明，加入注意力的一个好处是能够可视化模型“看见”了什么。受近期描述生成进展的鼓舞，并受到机器翻译（Bahdanau 等，2014）与物体识别（Ba 等，2014；Mnih 等，2014）中注意力成功应用的启发，我们研究能在生成描述时注视图像显著部分的模型。

The contributions of this paper are the following:

本文贡献如下：

• • We introduce two attention-based image caption generators under a common framework (Sec. 3.1): 1) a “soft” deterministic attention mechanism trainable by standard back-propagation methods and 2) a “hard” stochastic attention mechanism trainable by maximizing an approximate variational lower bound or equivalently by REINFORCE (Williams, 1992).

• • 我们在同一框架下引入两种基于注意力的图像描述生成器（第 3.1 节）：1）可用标准反向传播训练的“软”确定性注意力机制；2）通过最大化近似变分下界（或等价地用 REINFORCE（Williams，1992））训练的“硬”随机注意力机制。

• • We show how we can gain insight and interpret the results of this framework by visualizing “where” and “what” the attention focused on. (see Sec. 5.4)

• • 我们展示如何通过可视化注意力聚焦在“何处”以及“何物”，来理解并解释该框架的结果。（见第 5.4 节）

• • Finally, we quantitatively validate the usefulness of attention in caption generation with state of the art performance (Sec. 5.3) on three benchmark datasets: Flickr8k (Hodosh et al., 2013) , Flickr30k (Young et al., 2014) and the MS COCO dataset (Lin et al., 2014).

• • 最后，我们在三个基准数据集上用当时最优表现定量验证注意力对描述生成的用处（第 5.3 节）：Flickr8k（Hodosh 等，2013）、Flickr30k（Young 等，2014）以及 MS COCO 数据集（Lin 等，2014）。

2 Related Work

2 相关工作

In this section we provide relevant background on previous work on image caption generation and attention. Recently, several methods have been proposed for generating image descriptions. Many of these methods are based on recurrent neural networks and inspired by the successful use of sequence to sequence training with neural networks for machine translation (Cho et al., 2014; Bahdanau et al., 2014; Sutskever et al., 2014). One major reason image caption generation is well suited to the encoder-decoder framework (Cho et al., 2014) of machine translation is because it is analogous to “translating” an image to a sentence.

本节给出图像描述生成与注意力方面的相关背景。近期已有多种生成图像描述的方法。其中许多基于循环神经网络，并受到机器翻译中序列到序列神经网络训练成功应用的启发（Cho 等，2014；Bahdanau 等，2014；Sutskever 等，2014）。描述生成很适合机器翻译的编码器–解码器框架（Cho 等，2014），一个主要原因是它类似于把图像“翻译”成句子。

The first approach to use neural networks for caption generation was Kiros et al. (2014a), who proposed a multimodal log-bilinear model that was biased by features from the image. This work was later followed by Kiros et al. (2014b) whose method was designed to explicitly allow a natural way of doing both ranking and generation. Mao et al. (2014) took a similar approach to generation but replaced a feed-forward neural language model with a recurrent one. Both Vinyals et al. (2014) and Donahue et al. (2014) use LSTM RNNs for their models. Unlike Kiros et al. (2014a) and Mao et al. (2014) whose models see the image at each time step of the output word sequence, Vinyals et al. (2014) only show the image to the RNN at the beginning. Along with images, Donahue et al. (2014) also apply LSTMs to videos, allowing their model to generate video descriptions.

最早把神经网络用于描述生成的是 Kiros 等（2014a），他们提出一种由图像特征偏置的多模态对数双线性模型。随后 Kiros 等（2014b）的方法被设计成能自然地同时做排序与生成。Mao 等（2014）采取类似的生成思路，但用循环神经语言模型替换了前馈语言模型。Vinyals 等（2014）与 Donahue 等（2014）都在模型中使用 LSTM RNN。与 Kiros 等（2014a）和 Mao 等（2014）在输出词序列的每一步都看到图像不同，Vinyals 等（2014）只在开始时把图像交给 RNN。除图像外，Donahue 等（2014）还把 LSTM 用到视频上，使模型能够生成视频描述。

All of these works represent images as a single feature vector from the top layer of a pre-trained convolutional network. Karpathy & Li (2014) instead proposed to learn a joint embedding space for ranking and generation whose model learns to score sentence and image similarity as a function of R-CNN object detections with outputs of a bidirectional RNN. Fang et al. (2014) proposed a three-step pipeline for generation by incorporating object detections. Their model first learn detectors for several visual concepts based on a multi-instance learning framework. A language model trained on captions was then applied to the detector outputs, followed by rescoring from a joint image-text embedding space. Unlike these models, our proposed attention framework does not explicitly use object detectors but instead learns latent alignments from scratch. This allows our model to go beyond “objectness” and learn to attend to abstract concepts.

这些工作都把图像表示成预训练卷积网络顶层的单个特征向量。Karpathy 与 Li（2014）则提出学习一个用于排序与生成的联合嵌入空间，其模型根据 R-CNN 物体检测与双向 RNN 的输出，给句子与图像的相似度打分。Fang 等（2014）提出三步生成流程，引入物体检测：模型先在多实例学习框架下学习若干视觉概念的检测器，再把检测器输出送入在描述上训练的语言模型，最后用联合图文嵌入空间重打分。与这些模型不同，我们提出的注意力框架并不显式使用物体检测器，而是从零开始学习潜在对齐。这使模型可以超越“物体性”，学会注意抽象概念。

Prior to the use of neural networks for generating captions, two main approaches were dominant. The first involved generating caption templates which were filled in based on the results of object detections and attribute discovery (Kulkarni et al. (2013), Li et al. (2011), Yang et al. (2011), Mitchell et al. (2012), Elliott & Keller (2013)). The second approach was based on first retrieving similar captioned images from a large database then modifying these retrieved captions to fit the query (Kuznetsova et al., 2012; Kuznetsova et al., 2014). These approaches typically involved an intermediate “generalization” step to remove the specifics of a caption that are only relevant to the retrieved image, such as the name of a city. Both of these approaches have since fallen out of favour to the now dominant neural network methods.

在用神经网络生成描述之前，主流大致有两类方法。第一类是生成描述模板，再根据物体检测与属性发现的结果填空（Kulkarni 等（2013），Li 等（2011），Yang 等（2011），Mitchell 等（2012），Elliott 与 Keller（2013））。第二类是先从大型数据库中检索相似的带描述图像，再修改这些检索到的描述以适配查询（Kuznetsova 等，2012；Kuznetsova 等，2014）。这类方法通常还需要一步中间的“泛化”，去掉只与被检索图像相关的细节，例如城市名。这两类方法后来都让位于如今占主导的神经网络方法。

There has been a long line of previous work incorporating attention into neural networks for vision related tasks. Some that share the same spirit as our work include Larochelle & Hinton (2010); Denil et al. (2012); Tang et al. (2014). In particular however, our work directly extends the work of Bahdanau et al. (2014); Mnih et al. (2014); Ba et al. (2014).

把注意力引入视觉相关神经网络已有很长的研究脉络。与我们工作精神相近的包括 Larochelle 与 Hinton（2010）；Denil 等（2012）；Tang 等（2014）。不过，本文直接延伸的是 Bahdanau 等（2014）；Mnih 等（2014）；Ba 等（2014）的工作。

3 Image Caption Generation with Attention Mechanism

3 带注意力机制的图像描述生成

3.1 Model Details

3.1 模型细节

In this section, we describe the two variants of our attention-based model by first describing their common framework. The main difference is the definition of the $\phi$ function which we describe in detail in Section 4. We denote vectors with bolded font and matrices with capital letters. In our description below, we suppress bias terms for readability.

本节先描述两种基于注意力的模型变体的共同框架。主要差别在于 $\phi$ 函数的定义，我们在第 4 节详细说明。向量用粗体，矩阵用大写字母。为便于阅读，下面的描述略去偏置项。

3.1.1 Encoder: Convolutional Features

Our model takes a single raw image and generates a caption $\mathbf{y}$ encoded as a sequence of 1-of-$K$ encoded words.

3.1.1 编码器：卷积特征

我们的模型接收一张原始图像，并生成描述 $\mathbf{y}$，它被编码为一串 1-of-$K$ 的词。

$$y=\left\{\mathbf{y}_{1},\ldots,\mathbf{y}_{C}\right\},\mbox{ }\mathbf{y}_{i}\in\mathbb{R}^{K}$$

where $K$ is the size of the vocabulary and $C$ is the length of the caption.

其中 $K$ 是词表大小，$C$ 是描述长度。

We use a convolutional neural network in order to extract a set of feature vectors which we refer to as annotation vectors. The extractor produces $L$ vectors, each of which is a D-dimensional representation corresponding to a part of the image.

我们用卷积神经网络提取一组特征向量，称为标注向量。提取器产生 $L$ 个向量，每个都是对应图像某一部分的 $D$ 维表示。

$$\displaystyle a=\left\{\mathbf{a}_{1},\ldots,\mathbf{a}_{L}\right\},\mbox{ }\mathbf{a}_{i}\in\mathbb{R}^{D}$$

In order to obtain a correspondence between the feature vectors and portions of the 2-D image, we extract features from a lower convolutional layer unlike previous work which instead used a fully connected layer. This allows the decoder to selectively focus on certain parts of an image by selecting a subset of all the feature vectors.

为了让特征向量与二维图像的各个部分对应起来，我们从较低的卷积层提取特征，而不是像以往工作那样使用全连接层。这样，解码器可以通过选择特征向量子集，有选择地聚焦图像的某些部分。

Figure 4: A LSTM cell, lines with bolded squares imply projections with a learnt weight vector. Each cell learns how to weigh its input components (input gate), while learning how to modulate that contribution to the memory (input modulator). It also learns weights which erase the memory cell (forget gate), and weights which control how this memory should be emitted (output gate).

图 4：一个 LSTM 单元，带粗方框的线表示带可学习权重向量的投影。每个单元学习如何权衡其输入分量（输入门），同时学习如何调节该贡献写入记忆（输入调制器）。它还学习擦除记忆单元的权重（遗忘门），以及控制这段记忆如何输出的权重（输出门）。

3.1.2 Decoder: Long Short-Term Memory Network

We use a long short-term memory (LSTM) network (Hochreiter & Schmidhuber, 1997) that produces a caption by generating one word at every time step conditioned on a context vector, the previous hidden state and the previously generated words. Our implementation of LSTM closely follows the one used in Zaremba et al. (2014) (see Fig. 4). Using $T_{s,t}:\mathbb{R}^{s}\rightarrow\mathbb{R}^{t}$ to denote a simple affine transformation with parameters that are learned,

3.1.2 解码器：长短期记忆网络

我们使用长短期记忆（LSTM）网络（Hochreiter 与 Schmidhuber，1997），它在每一步根据上下文向量、上一隐藏状态以及此前生成的词，生成描述中的一个词。我们的 LSTM 实现紧密沿用 Zaremba 等（2014）的做法（见图 4）。用 $T_{s,t}:\mathbb{R}^{s}\rightarrow\mathbb{R}^{t}$ 表示带可学习参数的简单仿射变换，

$$\displaystyle\begin{pmatrix}\mathbf{i}{t}\
\mathbf{f}{t}\
\mathbf{o}{t}\
\mathbf{g}{t}\
\end{pmatrix}= \ \displaystyle\begin{pmatrix}\sigma\
\sigma\
\sigma\
\tanh\
\end{pmatrix}T_{D+m+n,n}\begin{pmatrix}\mathbf{E}\mathbf{y}{t-1}\
\mathbf{h}{t-1}\
\hat{\mathbf{z}{t}}\
\end{pmatrix} \ \displaystyle\mathbf{c}{t} \ \displaystyle=\mathbf{f}{t}\odot\mathbf{c}{t-1}+\mathbf{i}{t}\odot\mathbf{g}{t} \ \displaystyle\mathbf{h}{t} \ \displaystyle=\mathbf{o}{t}\odot\tanh(\mathbf{c}_{t}). \tag{1}$$

$$\displaystyle\begin{pmatrix}\mathbf{i}_{t}\\\mathbf{f}_{t}\\\mathbf{o}_{t}\\\mathbf{g}_{t}\\\end{pmatrix}= \\ \displaystyle\begin{pmatrix}\sigma\\\sigma\\\sigma\\\tanh\\\end{pmatrix}T_{D+m+n,n}\begin{pmatrix}\mathbf{E}\mathbf{y}_{t-1}\\\mathbf{h}_{t-1}\\\hat{\mathbf{z}_{t}}\\\end{pmatrix} \\ \displaystyle\mathbf{c}_{t} \\ \displaystyle=\mathbf{f}_{t}\odot\mathbf{c}_{t-1}+\mathbf{i}_{t}\odot\mathbf{g}_{t} \\ \displaystyle\mathbf{h}_{t} \\ \displaystyle=\mathbf{o}_{t}\odot\tanh(\mathbf{c}_{t}). \tag{1}$$

Here, $\mathbf{i}_{t}$, $\mathbf{f}_{t}$, $\mathbf{c}_{t}$, $\mathbf{o}_{t}$, $\mathbf{h}_{t}$ are the input, forget, memory, output and hidden state of the LSTM, respectively. The vector $\hat{\mathbf{z}}\in\mathbb{R}^{D}$ is the context vector, capturing the visual information associated with a particular input location, as explained below. $\mathbf{E}\in\mathbb{R}^{m\times K}$ is an embedding matrix. Let $m$ and $n$ denote the embedding and LSTM dimensionality respectively and $\sigma$ and $\odot$ be the logistic sigmoid activation and element-wise multiplication respectively.

这里，$\mathbf{i}_{t}$、$\mathbf{f}_{t}$、$\mathbf{c}_{t}$、$\mathbf{o}_{t}$、$\mathbf{h}_{t}$ 分别是 LSTM 的输入、遗忘、记忆、输出与隐藏状态。向量 $\hat{\mathbf{z}}\in\mathbb{R}^{D}$ 是上下文向量，捕捉与特定输入位置相关的视觉信息，下文说明。$\mathbf{E}\in\mathbb{R}^{m\times K}$ 是嵌入矩阵。令 $m$ 与 $n$ 分别表示嵌入维数与 LSTM 维数，$\sigma$ 与 $\odot$ 分别是 logistic sigmoid 激活与逐元素乘法。

In simple terms, the context vector $\hat{\mathbf{z}}_{t}$ (equations (1)–(3)) is a dynamic representation of the relevant part of the image input at time $t$. We define a mechanism $\phi$ that computes $\hat{\mathbf{z}}_{t}$ from the annotation vectors $\mathbf{a}_{i},i=1,\ldots,L$ corresponding to the features extracted at different image locations. For each location $i$, the mechanism generates a positive weight $\alpha_{i}$ which can be interpreted either as the probability that location $i$ is the right place to focus for producing the next word (the “hard” but stochastic attention mechanism), or as the relative importance to give to location $i$ in blending the $a_{i}$’s together. The weight $\alpha_{i}$ of each annotation vector $a_{i}$ is computed by an attention model $f_{\mbox{att}}$ for which we use a multilayer perceptron conditioned on the previous hidden state $h_{t-1}$. The soft version of this attention mechanism was introduced by Bahdanau et al. (2014). For emphasis, we note that the hidden state varies as the output RNN advances in its output sequence: “where” the network looks next depends on the sequence of words that has already been generated.

简单来说，上下文向量 $\hat{\mathbf{z}}_{t}$（式（1）–（3））是时刻 $t$ 与图像输入相关部分对应的动态表示。我们定义机制 $\phi$，从对应于不同图像位置所提特征的标注向量 $\mathbf{a}_{i},i=1,\ldots,L$ 计算 $\hat{\mathbf{z}}_{t}$。对每个位置 $i$，该机制生成一个正权重 $\alpha_{i}$：它既可以解释为“位置 $i$ 是生成下一个词时该注视之处”的概率（“硬”但随机的注意力机制），也可以解释为把各个 $a_{i}$ 混合时赋予位置 $i$ 的相对重要性。每个标注向量 $a_{i}$ 的权重 $\alpha_{i}$ 由注意力模型 $f_{\mbox{att}}$ 计算，我们用一个以先前隐藏状态 $h_{t-1}$ 为条件的多层感知机来实现。这种注意力机制的软版本由 Bahdanau 等（2014）引入。需要强调的是，隐藏状态会随着输出 RNN 在输出序列上推进而变化：网络接下来看“哪里”，取决于已经生成的词序列。

$$\displaystyle e_{ti}= \\ \displaystyle f_{\mbox{att}}(\mathbf{a}_{i},\mathbf{h}_{t-1}) \\ \displaystyle\alpha_{ti}= \\ \displaystyle\frac{\exp(e_{ti})}{\sum_{k=1}^{L}\exp(e_{tk})}. \tag{4}$$

Once the weights (which sum to one) are computed, the context vector $\hat{z}_{t}$ is computed by

权重（和为 1）计算完成后，上下文向量 $\hat{z}_{t}$ 由下式得到

$$\displaystyle\hat{\mathbf{z}}_{t}=\phi\left(\left\{\mathbf{a}_{i}\right\},\left\{\alpha_{i}\right\}\right), \tag{6}$$

where $\phi$ is a function that returns a single vector given the set of annotation vectors and their corresponding weights. The details of $\phi$ function are discussed in Sec. 4.

其中 $\phi$ 根据一组标注向量及其对应权重返回单个向量。$\phi$ 函数的细节在第 4 节讨论。

The initial memory state and hidden state of the LSTM are predicted by an average of the annotation vectors fed through two separate MLPs (init,c and init,h):

LSTM 的初始记忆状态与隐藏状态，由标注向量的平均分别通过两个 MLP（init,c 与 init,h）来预测：

$$\displaystyle\mathbf{c}_{0}=f_{\text{init,c}}(\frac{1}{L}\sum_{i}^{L}\mathbf{a}_{i}) \\ \displaystyle\mathbf{h}_{0}=f_{\text{init,h}}(\frac{1}{L}\sum_{i}^{L}\mathbf{a}_{i})$$

In this work, we use a deep output layer (Pascanu et al., 2014) to compute the output word probability given the LSTM state, the context vector and the previous word:

本文用深层输出层（Pascanu 等，2014），根据 LSTM 状态、上下文向量以及前一个词来计算输出词概率：

$$\displaystyle p(\mathbf{y}_{t}|\mathbf{a},\mathbf{y}_{1}^{t-1})\propto\exp(\mathbf{L}_{o}(\mathbf{E}\mathbf{y}_{t-1}+\mathbf{L}_{h}\mathbf{h}_{t}+\mathbf{L}_{z}\hat{\mathbf{z}}_{t})) \tag{7}$$

Where $\mathbf{L}_{o}\in\mathbb{R}^{K\times m}$, $\mathbf{L}_{h}\in\mathbb{R}^{m\times n}$, $\mathbf{L}_{z}\in\mathbb{R}^{m\times D}$, and $\mathbf{E}$ are learned parameters initialized randomly.

其中 $\mathbf{L}_{o}\in\mathbb{R}^{K\times m}$、$\mathbf{L}_{h}\in\mathbb{R}^{m\times n}$、$\mathbf{L}_{z}\in\mathbb{R}^{m\times D}$ 以及 $\mathbf{E}$ 都是随机初始化的可学习参数。

4 Learning Stochastic “Hard” vs Deterministic “Soft” Attention

4 学习随机“硬”注意力与确定性“软”注意力

In this section we discuss two alternative mechanisms for the attention model $f_{\mbox{att}}$: stochastic attention and deterministic attention.

本节讨论注意力模型 $f_{\mbox{att}}$ 的两种替代机制：随机注意力与确定性注意力。

Figure 5: Examples of mistakes where we can use attention to gain intuition into what the model saw.

图 5：一些错误例子，我们可以借助注意力来直觉地理解模型当时看见了什么。

4.1 Stochastic “Hard” Attention

4.1 随机“硬”注意力

We represent the location variable $s_{t}$ as where the model decides to focus attention when generating the $t^{th}$ word. $s_{t,i}$ is an indicator one-hot variable which is set to 1 if the $i$-th location (out of $L$) is the one used to extract visual features. By treating the attention locations as intermediate latent variables, we can assign a multinoulli distribution parametrized by $\{\alpha_{i}\}$, and view $\hat{z}_{t}$ as a random variable:

我们用位置变量 $s_{t}$ 表示模型在生成第 $t$ 个词时决定把注意力放在哪里。$s_{t,i}$ 是一个指示独热变量：若第 $i$ 个位置（共 $L$ 个）被用来提取视觉特征，则取 1。把注意力位置当作中间隐变量，我们可以赋予由 $\{\alpha_{i}\}$ 参数化的多项伯努利分布，并把 $\hat{z}_{t}$ 视为随机变量：

$$\displaystyle p( \\ \displaystyle s_{t,i}=1\mid s_{j<t},\mathbf{a})=\alpha_{t,i} \\ \displaystyle\hat{\mathbf{z}}_{t} \\ \displaystyle=\sum_{i}{s}_{t,i}\mathbf{a}_{i}. \tag{8}$$

We define a new objective function $L_{s}$ that is a variational lower bound on the marginal log-likelihood $\log p(\mathbf{y}\mid\mathbf{a})$ of observing the sequence of words $\mathbf{y}$ given image features $\mathbf{a}$. The learning algorithm for the parameters $W$ of the models can be derived by directly optimizing $L_{s}$:

我们定义新的目标函数 $L_{s}$，它是在给定图像特征 $\mathbf{a}$ 时观察到词序列 $\mathbf{y}$ 的边际对数似然 $\log p(\mathbf{y}\mid\mathbf{a})$ 的变分下界。模型参数 $W$ 的学习算法可以通过直接优化 $L_{s}$ 导出：

$$\displaystyle L_{s}= \\ \displaystyle\sum_{s}p(s\mid\mathbf{a})\log p(\mathbf{y}\mid s,\mathbf{a}) \\ \displaystyle\leq \\ \displaystyle\log\sum_{s}p(s\mid\mathbf{a})p(\mathbf{y}\mid s,\mathbf{a}) \\ \displaystyle= \\ \displaystyle\log p(\mathbf{y}\mid\mathbf{a}) \tag{10}$$

$$\frac{\partial L_{s}}{\partial W}=\sum_{s}p(s\mid\mathbf{a})\bigg[\frac{\partial\log p(\mathbf{y}\mid s,\mathbf{a})}{\partial W}+\
\log p(\mathbf{y}\mid s,\mathbf{a})\frac{\partial\log p(s\mid\mathbf{a})}{\partial W}\bigg]. \tag{11}$$

$$\frac{\partial L_{s}}{\partial W}=\sum_{s}p(s\mid\mathbf{a})\bigg[\frac{\partial\log p(\mathbf{y}\mid s,\mathbf{a})}{\partial W}+\\\log p(\mathbf{y}\mid s,\mathbf{a})\frac{\partial\log p(s\mid\mathbf{a})}{\partial W}\bigg]. \tag{11}$$

Equation 11 suggests a Monte Carlo based sampling approximation of the gradient with respect to the model parameters. This can be done by sampling the location $s_{t}$ from a multinouilli distribution defined by Equation 8.

式 11 表明，可以用基于蒙特卡洛采样的方法来近似对模型参数的梯度。做法是按式 8 定义的多项伯努利分布采样位置 $s_{t}$。

$$\displaystyle\tilde{s_{t}}\sim\text{Multinoulli}_{L}(\{\alpha_{i}\})$$

$$\frac{\partial L_{s}}{\partial W}\approx\frac{1}{N}\sum_{n=1}^{N}\bigg[\frac{\partial\log p(\mathbf{y}\mid\tilde{s}^{n},\mathbf{a})}{\partial W}+\
\log p(\mathbf{y}\mid\tilde{s}^{n},\mathbf{a})\frac{\partial\log p(\tilde{s}^{n}\mid\mathbf{a})}{\partial W}\bigg] \tag{12}$$

$$\frac{\partial L_{s}}{\partial W}\approx\frac{1}{N}\sum_{n=1}^{N}\bigg[\frac{\partial\log p(\mathbf{y}\mid\tilde{s}^{n},\mathbf{a})}{\partial W}+\\\log p(\mathbf{y}\mid\tilde{s}^{n},\mathbf{a})\frac{\partial\log p(\tilde{s}^{n}\mid\mathbf{a})}{\partial W}\bigg] \tag{12}$$

A moving average baseline is used to reduce the variance in the Monte Carlo estimator of the gradient, following Weaver & Tao (2001). Similar, but more complicated variance reduction techniques have previously been used by Mnih et al. (2014) and Ba et al. (2014). Upon seeing the $k^{th}$ mini-batch, the moving average baseline is estimated as an accumulated sum of the previous log likelihoods with exponential decay:

为降低梯度蒙特卡洛估计的方差，我们沿用 Weaver 与 Tao（2001）使用滑动平均基线。Mnih 等（2014）与 Ba 等（2014）曾使用类似但更复杂的方差削减技术。看到第 $k$ 个小批次时，滑动平均基线按此前对数似然的指数衰减累加来估计：

$$\displaystyle b_{k}=0.9\times b_{k-1}+0.1\times\log p(\mathbf{y}\mid\tilde{s}_{k},\mathbf{a})$$

To further reduce the estimator variance, an entropy term on the multinouilli distribution $H[s]$ is added. Also, with probability 0.5 for a given image, we set the sampled attention location $\tilde{s}$ to its expected value $\alpha$. Both techniques improve the robustness of the stochastic attention learning algorithm. The final learning rule for the model is then the following:

为进一步降低估计方差，还加入多项伯努利分布上的熵项 $H[s]$。此外，对给定图像以 0.5 的概率把采样到的注意力位置 $\tilde{s}$ 设成其期望值 $\alpha$。两种技术都提高了随机注意力学习算法的稳健性。模型最终的学习规则如下：

$$\frac{\partial L_{s}}{\partial W}\approx\frac{1}{N}\sum_{n=1}^{N}\bigg[\frac{\partial\log p(\mathbf{y}\mid\tilde{s}^{n},\mathbf{a})}{\partial W}+\
\lambda_{r}(\log p(\mathbf{y}\mid\tilde{s}^{n},\mathbf{a})-b)\frac{\partial\log p(\tilde{s}^{n}\mid\mathbf{a})}{\partial W}+\lambda_{e}\frac{\partial H[\tilde{s}^{n}]}{\partial W}\bigg] \tag{13}$$

$$\frac{\partial L_{s}}{\partial W}\approx\frac{1}{N}\sum_{n=1}^{N}\bigg[\frac{\partial\log p(\mathbf{y}\mid\tilde{s}^{n},\mathbf{a})}{\partial W}+\\\lambda_{r}(\log p(\mathbf{y}\mid\tilde{s}^{n},\mathbf{a})-b)\frac{\partial\log p(\tilde{s}^{n}\mid\mathbf{a})}{\partial W}+\lambda_{e}\frac{\partial H[\tilde{s}^{n}]}{\partial W}\bigg] \tag{13}$$

where, $\lambda_{r}$ and $\lambda_{e}$ are two hyper-parameters set by cross-validation. As pointed out and used in Ba et al. (2014) and Mnih et al. (2014), this is formulation is equivalent to the REINFORCE learning rule (Williams, 1992), where the reward for the attention choosing a sequence of actions is a real value proportional to the log likelihood of the target sentence under the sampled attention trajectory.

其中 $\lambda_{r}$ 与 $\lambda_{e}$ 是由交叉验证设定的两个超参数。正如 Ba 等（2014）与 Mnih 等（2014）所指出并采用的，这一形式等价于 REINFORCE 学习规则（Williams，1992）：注意力选择一系列动作的奖励，是与目标句子在采样注意力轨迹下的对数似然成正比的实值。

In making a hard choice at every point, $\phi\left(\left\{\mathbf{a}_{i}\right\},\left\{\alpha_{i}\right\}\right)$ from Equation 6 is a function that returns a sampled $\mathbf{a}_{i}$ at every point in time based upon a multinouilli distribution parameterized by $\alpha$.

由于每一步都做硬选择，式 6 中的 $\phi\left(\left\{\mathbf{a}_{i}\right\},\left\{\alpha_{i}\right\}\right)$ 是一个函数：它在每一时刻根据由 $\alpha$ 参数化的多项伯努利分布，返回一个采样得到的 $\mathbf{a}_{i}$。

4.2 Deterministic “Soft” Attention

4.2 确定性“软”注意力

Learning stochastic attention requires sampling the attention location $s_{t}$ each time, instead we can take the expectation of the context vector $\hat{\mathbf{z}}_{t}$ directly,

学习随机注意力需要每次采样注意力位置 $s_{t}$，我们也可以直接取上下文向量 $\hat{\mathbf{z}}_{t}$ 的期望，

$$\displaystyle\mathbb{E}_{p(s_{t}|a)}[\hat{\mathbf{z}}_{t}]=\sum_{i=1}^{L}\alpha_{t,i}\mathbf{a}_{i} \tag{14}$$

and formulate a deterministic attention model by computing a soft attention weighted annotation vector $\phi\left(\left\{\mathbf{a}_{i}\right\},\left\{\alpha_{i}\right\}\right)=\sum_{i}^{L}\alpha_{i}\mathbf{a}_{i}$ as introduced by Bahdanau et al. (2014). This corresponds to feeding in a soft $\alpha$ weighted context into the system. The whole model is smooth and differentiable under the deterministic attention, so learning end-to-end is trivial by using standard back-propagation.

并按 Bahdanau 等（2014）引入的方式，用软注意力加权的标注向量 $\phi\left(\left\{\mathbf{a}_{i}\right\},\left\{\alpha_{i}\right\}\right)=\sum_{i}^{L}\alpha_{i}\mathbf{a}_{i}$ 来构成确定性注意力模型。这相当于把经 $\alpha$ 软加权的上下文送入系统。在确定性注意力下整个模型光滑可微，用标准反向传播即可端到端学习。

Learning the deterministic attention can also be understood as approximately optimizing the marginal likelihood in Equation 10 under the attention location random variable $s_{t}$ from Sec. 4.1. The hidden activation of LSTM $\mathbf{h}_{t}$ is a linear projection of the stochastic context vector $\hat{\mathbf{z}}_{t}$ followed by $\tanh$ non-linearity. To the first order Taylor approximation, the expected value $\mathbb{E}_{p(s_{t}|a)}[\mathbf{h}_{t}]$ is equal to computing $\mathbf{h}_{t}$ using a single forward prop with the expected context vector $\mathbb{E}_{p(s_{t}|a)}[\hat{\mathbf{z}}_{t}]$. Considering Eq. 7, let $\mathbf{n}_{t}=\mathbf{L}_{o}(\mathbf{E}\mathbf{y}_{t-1}+\mathbf{L}_{h}\mathbf{h}_{t}+\mathbf{L}_{z}\hat{\mathbf{z}}_{t})$, $\mathbf{n}_{t,i}$ denotes $\mathbf{n}_{t}$ computed by setting the random variable $\hat{\mathbf{z}}$ value to $\mathbf{a}_{i}$. We define the normalized weighted geometric mean for the softmax $k^{th}$ word prediction:

学习确定性注意力也可以理解为：在第 4.1 节的注意力位置随机变量 $s_{t}$ 下，近似优化式 10 中的边际似然。LSTM 的隐藏激活 $\mathbf{h}_{t}$ 是随机上下文向量 $\hat{\mathbf{z}}_{t}$ 的线性投影再接 $\tanh$ 非线性。按一阶泰勒近似，期望 $\mathbb{E}_{p(s_{t}|a)}[\mathbf{h}_{t}]$ 等于用期望上下文向量 $\mathbb{E}_{p(s_{t}|a)}[\hat{\mathbf{z}}_{t}]$ 做一次前向计算得到的 $\mathbf{h}_{t}$。考虑式 7，令 $\mathbf{n}_{t}=\mathbf{L}_{o}(\mathbf{E}\mathbf{y}_{t-1}+\mathbf{L}_{h}\mathbf{h}_{t}+\mathbf{L}_{z}\hat{\mathbf{z}}_{t})$，$\mathbf{n}_{t,i}$ 表示把随机变量 $\hat{\mathbf{z}}$ 取成 $\mathbf{a}_{i}$ 时算出的 $\mathbf{n}_{t}$。我们为 softmax 第 $k$ 个词的预测定义归一化加权几何平均：

$$\displaystyle NWGM[p(y_{t}=k\mid\mathbf{a})] \\ \displaystyle={\prod_{i}\exp(n_{t,k,i})^{p(s_{t,i}=1|a)}\over\sum_{j}\prod_{i}\exp(n_{t,j,i})^{p(s_{t,i}=1|a)}} \\ \displaystyle={\exp(\mathbb{E}_{p(s_{t}|a)}[n_{t,k}])\over\sum_{j}\exp(\mathbb{E}_{p(s_{t}|a)}[n_{t,j}])}$$

The equation above shows the normalized weighted geometric mean of the caption prediction can be approximated well by using the expected context vector, where $\mathbb{E}[\mathbf{n}_{t}]=\mathbf{L}_{o}(\mathbf{E}\mathbf{y}_{t-1}+\mathbf{L}_{h}\mathbb{E}[\mathbf{h}_{t}]+\mathbf{L}_{z}\mathbb{E}[\hat{\mathbf{z}}_{t}])$. It shows that the NWGM of a softmax unit is obtained by applying softmax to the expectations of the underlying linear projections. Also, from the results in Baldi & Sadowski (2014), $NWGM[p(y_{t}=k\mid\mathbf{a})]\approx\mathbb{E}[p(y_{t}=k\mid\mathbf{a})]$ under softmax activation. That means the expectation of the outputs over all possible attention locations induced by random variable $s_{t}$ is computed by simple feedforward propagation with expected context vector $\mathbb{E}[\hat{\mathbf{z}}_{t}]$. In other words, the deterministic attention model is an approximation to the marginal likelihood over the attention locations.

上式表明，描述预测的归一化加权几何平均可以用期望上下文向量很好地近似，其中 $\mathbb{E}[\mathbf{n}_{t}]=\mathbf{L}_{o}(\mathbf{E}\mathbf{y}_{t-1}+\mathbf{L}_{h}\mathbb{E}[\mathbf{h}_{t}]+\mathbf{L}_{z}\mathbb{E}[\hat{\mathbf{z}}_{t}])$。这说明 softmax 单元的 NWGM，是对底层线性投影的期望再做 softmax 得到的。此外，根据 Baldi 与 Sadowski（2014）的结果，在 softmax 激活下有 $NWGM[p(y_{t}=k\mid\mathbf{a})]\approx\mathbb{E}[p(y_{t}=k\mid\mathbf{a})]$。也就是说，对随机变量 $s_{t}$ 诱导的所有可能注意力位置，输出的期望可以通过用期望上下文向量 $\mathbb{E}[\hat{\mathbf{z}}_{t}]$ 做一次简单前馈计算得到。换句话说，确定性注意力模型是对注意力位置边际似然的一种近似。

4.2.1 Doubly Stochastic Attention

By construction, $\sum_{i}\alpha_{ti}=1$ as they are the output of a softmax. In training the deterministic version of our model we introduce a form of doubly stochastic regularization, where we also encourage $\sum_{t}\alpha_{ti}\approx 1$. This can be interpreted as encouraging the model to pay equal attention to every part of the image over the course of generation. In our experiments, we observed that this penalty was important quantitatively to improving overall BLEU score and that qualitatively this leads to more rich and descriptive captions. In addition, the soft attention model predicts a gating scalar $\beta$ from previous hidden state $\mathbf{h}_{t-1}$ at each time step $t$, such that, $\phi\left(\left\{\mathbf{a}_{i}\right\},\left\{\alpha_{i}\right\}\right)=\beta\sum_{i}^{L}\alpha_{i}\mathbf{a}_{i}$, where $\beta_{t}=\sigma(f_{\beta}(\mathbf{h}_{t-1}))$. We notice our attention weights put more emphasis on the objects in the images by including the scalar $\beta$.

4.2.1 双重随机注意力

由构造可知，$\sum_{i}\alpha_{ti}=1$，因为它们是 softmax 的输出。在训练模型的确定性版本时，我们引入一种双重随机正则：还鼓励 $\sum_{t}\alpha_{ti}\approx 1$。这可以理解为鼓励模型在整段生成过程中对图像各部分给予大致均等的注意。实验中我们观察到，这一惩罚在定量上对提高总体 BLEU 很重要，在定性上也带来更丰富、更细致的描述。此外，软注意力模型在每一时刻 $t$ 还从先前隐藏状态 $\mathbf{h}_{t-1}$ 预测一个门控标量 $\beta$，使得 $\phi\left(\left\{\mathbf{a}_{i}\right\},\left\{\alpha_{i}\right\}\right)=\beta\sum_{i}^{L}\alpha_{i}\mathbf{a}_{i}$，其中 $\beta_{t}=\sigma(f_{\beta}(\mathbf{h}_{t-1}))$。我们注意到，加入标量 $\beta$ 后，注意力权重更强调图像中的物体。

		BLEU
Dataset	Model	BLEU-1	BLEU-2	BLEU-3	BLEU-4	METEOR
Flickr8k	Google NIC(Vinyals et al., 2014)†Σ Log Bilinear (Kiros et al., 2014a)∘ Soft-Attention Hard-Attention	Google NIC(Vinyals et al., 2014)†Σ	Log Bilinear (Kiros et al., 2014a)∘	Soft-Attention	Hard-Attention	63 65.6 67 67	63	65.6	67	67	41 42.4 44.8 45.7	41	42.4	44.8	45.7	27 27.7 29.9 31.4	27	27.7	29.9	31.4	— 17.7 19.5 21.3	—	17.7	19.5	21.3	— 17.31 18.93 20.30	—	17.31	18.93	20.30
Google NIC(Vinyals et al., 2014)†Σ
Log Bilinear (Kiros et al., 2014a)∘
Soft-Attention
Hard-Attention
63
65.6
67
67
41
42.4
44.8
45.7
27
27.7
29.9
31.4
—
17.7
19.5
21.3
—
17.31
18.93
20.30
Flickr30k	Google NIC†∘Σ Log Bilinear Soft-Attention Hard-Attention	Google NIC†∘Σ	Log Bilinear	Soft-Attention	Hard-Attention	66.3 60.0 66.7 66.9	66.3	60.0	66.7	66.9	42.3 38 43.4 43.9	42.3	38	43.4	43.9	27.7 25.4 28.8 29.6	27.7	25.4	28.8	29.6	18.3 17.1 19.1 19.9	18.3	17.1	19.1	19.9	— 16.88 18.49 18.46	—	16.88	18.49	18.46
Google NIC†∘Σ
Log Bilinear
Soft-Attention
Hard-Attention
66.3
60.0
66.7
66.9
42.3
38
43.4
43.9
27.7
25.4
28.8
29.6
18.3
17.1
19.1
19.9
—
16.88
18.49
18.46
COCO	CMU/MS Research (Chen & Zitnick, 2014)a MS Research (Fang et al., 2014)†a BRNN (Karpathy & Li, 2014)∘ Google NIC†∘Σ Log Bilinear∘ Soft-Attention Hard-Attention	CMU/MS Research (Chen & Zitnick, 2014)a	MS Research (Fang et al., 2014)†a	BRNN (Karpathy & Li, 2014)∘	Google NIC†∘Σ	Log Bilinear∘	Soft-Attention	Hard-Attention	— — 64.2 66.6 70.8 70.7 71.8	—	—	64.2	66.6	70.8	70.7	71.8	— — 45.1 46.1 48.9 49.2 50.4	—	—	45.1	46.1	48.9	49.2	50.4	— — 30.4 32.9 34.4 34.4 35.7	—	—	30.4	32.9	34.4	34.4	35.7	— — 20.3 24.6 24.3 24.3 25.0	—	—	20.3	24.6	24.3	24.3	25.0	20.41 20.71 — — 20.03 23.90 23.04	20.41	20.71	—	—	20.03	23.90	23.04
CMU/MS Research (Chen & Zitnick, 2014)a
MS Research (Fang et al., 2014)†a
BRNN (Karpathy & Li, 2014)∘
Google NIC†∘Σ
Log Bilinear∘
Soft-Attention
Hard-Attention
—
—
64.2
66.6
70.8
70.7
71.8
—
—
45.1
46.1
48.9
49.2
50.4
—
—
30.4
32.9
34.4
34.4
35.7
—
—
20.3
24.6
24.3
24.3
25.0
20.41
20.71
—
—
20.03
23.90
23.04

Google NIC(Vinyals et al., 2014)†Σ
Log Bilinear (Kiros et al., 2014a)∘
Soft-Attention
Hard-Attention

63
65.6
67
67

41
42.4
44.8
45.7

27
27.7
29.9
31.4

—
17.7
19.5
21.3

—
17.31
18.93
20.30

Google NIC†∘Σ
Log Bilinear
Soft-Attention
Hard-Attention

66.3
60.0
66.7
66.9

42.3
38
43.4
43.9

27.7
25.4
28.8
29.6

18.3
17.1
19.1
19.9

—
16.88
18.49
18.46

CMU/MS Research (Chen & Zitnick, 2014)a
MS Research (Fang et al., 2014)†a
BRNN (Karpathy & Li, 2014)∘
Google NIC†∘Σ
Log Bilinear∘
Soft-Attention
Hard-Attention

—
—
64.2
66.6
70.8
70.7
71.8

—
—
45.1
46.1
48.9
49.2
50.4

—
—
30.4
32.9
34.4
34.4
35.7

—
—
20.3
24.6
24.3
24.3
25.0

20.41
20.71
—
—
20.03
23.90
23.04

Table 1: BLEU-1,2,3,4/METEOR metrics compared to other methods, $\dagger$ indicates a different split, (—) indicates an unknown metric, $\circ$ indicates the authors kindly provided missing metrics by personal communication, $\Sigma$ indicates an ensemble, $a$ indicates using AlexNet