Universal Language Model Fine-tuning for Text Classification

面向文本分类的通用语言模型微调

Jeremy Howard 1 1 footnotemark: 1 Affiliation:  fast.ai Affiliation:  University of San Francisco Email:  j@fast.ai    Sebastian Ruder 1 1 footnotemark: 1 Affiliation:  Insight Centre, NUI Galway Affiliation:  Aylien Ltd., Dublin Email:  sebastian@ruder.io

Abstract

摘要

Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch. We propose Universal Language Model Fine-tuning (ULMFiT), an effective transfer learning method that can be applied to any task in NLP, and introduce techniques that are key for fine-tuning a language model. Our method significantly outperforms the state-of-the-art on six text classification tasks, reducing the error by 18-24% on the majority of datasets. Furthermore, with only $100$ labeled examples, it matches the performance of training from scratch on $100\times$ more data. We open-source our pretrained models and code[^2].

归纳式迁移学习已深刻改变计算机视觉，但自然语言处理（NLP）里现有做法仍多要做任务定制，并从零训练。我们提出通用语言模型微调（ULMFiT）：一种可套用到任意 NLP 任务的有效迁移方法，并给出微调语言模型时的关键技巧。该方法在六项文本分类任务上显著超过当时最优结果，多数数据集上误差下降 18%–24%。仅用 $100$ 条标注样本时，其表现可与从零训练、数据量多出 $100\times$ 的模型相当。我们开源了预训练模型与代码[^2]。

1 Introduction

1 引言

Inductive transfer learning has had a large impact on computer vision (CV). Applied CV models (including object detection, classification, and segmentation) are rarely trained from scratch, but instead are fine-tuned from models that have been pretrained on ImageNet, MS-COCO, and other datasets Sharif Razavian et al. (2014); Long et al. (2015a); He et al. (2016); Huang et al. (2017).

归纳式迁移学习对计算机视觉（CV）影响很大。实际部署的视觉模型（目标检测、分类、分割等）很少从零训起，而是在 ImageNet、MS-COCO 等数据上预训练后再微调 Sharif Razavian et al. (2014); Long et al. (2015a); He et al. (2016); Huang et al. (2017)。

Text classification is a category of Natural Language Processing (NLP) tasks with real-world applications such as spam, fraud, and bot detection Jindal and Liu (2007); Ngai et al. (2011); Chu et al. (2012), emergency response Caragea et al. (2011), and commercial document classification, such as for legal discovery Roitblat et al. (2010).

文本分类是一类自然语言处理（NLP）任务，现实用途包括垃圾邮件、欺诈与机器人账号检测 Jindal and Liu (2007); Ngai et al. (2011); Chu et al. (2012)，应急响应 Caragea et al. (2011)，以及商业文档归类（如电子证据开示）Roitblat et al. (2010)。

While Deep Learning models have achieved state-of-the-art on many NLP tasks, these models are trained from scratch, requiring large datasets, and days to converge. Research in NLP focused mostly on transductive transfer Blitzer et al. (2007). For inductive transfer, fine-tuning pretrained word embeddings Mikolov et al. (2013), a simple transfer technique that only targets a model’s first layer, has had a large impact in practice and is used in most state-of-the-art models. Recent approaches that concatenate embeddings derived from other tasks with the input at different layers Peters et al. (2017); McCann et al. (2017); Peters et al. (2018) still train the main task model from scratch and treat pretrained embeddings as fixed parameters, limiting their usefulness.

深度学习虽在许多 NLP 任务上做到当时最优，这些模型仍多从零训练，需要大体量数据，收敛也要数天。NLP 研究长期更偏直推式迁移 Blitzer et al. (2007)。在归纳式迁移上，微调预训练词向量 Mikolov et al. (2013) 只动模型第一层，却已在实践中影响很大，并被多数领先模型采用。近期有人把其他任务得到的嵌入在不同层与输入拼接 Peters et al. (2017); McCann et al. (2017); Peters et al. (2018)，主任务模型仍从零训起，预训练嵌入当固定参数，用处受限。

In light of the benefits of pretraining Erhan et al. (2010), we should be able to do better than randomly initializing the remaining parameters of our models. However, inductive transfer via fine-tuning has been unsuccessful for NLP Mou et al. (2016). Dai and Le (2015) first proposed fine-tuning a language model (LM) but require millions of in-domain documents to achieve good performance, which severely limits its applicability.

鉴于预训练的好处 Erhan et al. (2010)，我们不该再把其余参数随机初始化。但通过微调做归纳迁移在 NLP 上并不成功 Mou et al. (2016)。Dai and Le (2015) 最早提出微调语言模型（LM），却要上百万篇领域内文档才有好效果，适用面很窄。

We show that not the idea of LM fine-tuning but our lack of knowledge of how to train them effectively has been hindering wider adoption. LMs overfit to small datasets and suffered catastrophic forgetting when fine-tuned with a classifier. Compared to CV, NLP models are typically more shallow and thus require different fine-tuning methods.

我们说明：阻碍更广采用的并不是「微调 LM」这个想法本身，而是缺少有效训练它们的方法。语言模型在小数据上容易过拟合，与分类器一起微调时又会灾难性遗忘。相对 CV，NLP 模型通常更浅，需要不同的微调手法。

We propose a new method, Universal Language Model Fine-tuning (ULMFiT) that addresses these issues and enables robust inductive transfer learning for any NLP task, akin to fine-tuning ImageNet models: The same 3-layer LSTM architecture—with the same hyperparameters and no additions other than tuned dropout hyperparameters—outperforms highly engineered models and transfer learning approaches on six widely studied text classification tasks. On IMDb, with $100$ labeled examples, ULMFiT matches the performance of training from scratch with $10\times$ and—given $50$k unlabeled examples—with $100\times$ more data.

我们提出通用语言模型微调（ULMFiT），正是为解决上述问题，让任意 NLP 任务都能做稳健的归纳迁移，类似微调 ImageNet 模型：同一套 3 层 LSTM 架构——超参相同，除调好的 dropout 外不加别的结构——在六项常见文本分类上超过高度定制的模型与其他迁移方法。在 IMDb 上，仅 $100$ 条标注时，ULMFiT 可追上从零训练、$10\times$ 数据的表现；若再有 $50$k 无标注样本，则可追上 $100\times$ 数据。

Contributions

Our contributions are the following: 1) We propose Universal Language Model Fine-tuning (ULMFiT), a method that can be used to achieve CV-like transfer learning for any task for NLP. 2) We propose discriminative fine-tuning, slanted triangular learning rates, and gradual unfreezing, novel techniques to retain previous knowledge and avoid catastrophic forgetting during fine-tuning. 3) We significantly outperform the state-of-the-art on six representative text classification datasets, with an error reduction of 18-24% on the majority of datasets. 4) We show that our method enables extremely sample-efficient transfer learning and perform an extensive ablation analysis. 5) We make the pretrained models and our code available to enable wider adoption.

贡献

贡献如下：1）提出 ULMFiT，使 NLP 任务也能获得接近 CV 的迁移学习。2）提出区分式微调、斜三角学习率与逐步解冻，在微调时保留旧知识、避免灾难性遗忘。3）在六个代表性文本分类数据集上显著超过当时最优，多数集上误差降 18%–24%。4）表明该方法样本效率极高，并做了充分消融。5）公开预训练模型与代码，便于推广。

2 Related work

2 相关工作

Transfer learning in CV

Features in deep neural networks in CV have been observed to transition from general to task-specific from the first to the last layer Yosinski et al. (2014). For this reason, most work in CV focuses on transferring the first layers of the model Long et al. (2015b). Sharif Razavian et al. (2014) achieve state-of-the-art results using features of an ImageNet model as input to a simple classifier. In recent years, this approach has been superseded by fine-tuning either the last Donahue et al. (2014) or several of the last layers of a pretrained model and leaving the remaining layers frozen Long et al. (2015a).

计算机视觉中的迁移学习

深度视觉网络中的特征，从第一层到最后一层会由通用转向任务相关 Yosinski et al. (2014)。因此多数 CV 工作侧重迁移前几层 Long et al. (2015b)。Sharif Razavian et al. (2014) 用 ImageNet 模型特征喂给简单分类器即达当时最优。近年主流改为微调最后一层 Donahue et al. (2014) 或最后若干层，其余冻结 Long et al. (2015a)。

Hypercolumns

In NLP, only recently have methods been proposed that go beyond transferring word embeddings. The prevailing approach is to pretrain embeddings that capture additional context via other tasks. Embeddings at different levels are then used as features, concatenated either with the word embeddings or with the inputs at intermediate layers. This method is known as hypercolumns Hariharan et al. (2015) in CV[^3] and is used by Peters et al. (2017), Peters et al. (2018), Wieting and Gimpel (2017), Conneau et al. (2017), and McCann et al. (2017) who use language modeling, paraphrasing, entailment, and Machine Translation (MT) respectively for pretraining. Specifically, Peters et al. (2018) require engineered custom architectures, while we show state-of-the-art performance with the same basic architecture across a range of tasks. In CV, hypercolumns have been nearly entirely superseded by end-to-end fine-tuning Long et al. (2015a).

超列（Hypercolumns）

NLP 里，超越词向量迁移的方法出现得较晚。通行做法是：用其他任务预训练能捕获更多上下文的嵌入，再在不同层级当特征，与词向量或中间层输入拼接。这在 CV 里叫超列 Hariharan et al. (2015)[^3]，被 Peters et al. (2017)、Peters et al. (2018)、Wieting and Gimpel (2017)、Conneau et al. (2017)、McCann et al. (2017) 分别用语言建模、释义、蕴含与机器翻译（MT）来预训练。Peters et al. (2018) 需要定制架构，我们则用同一基础架构跨任务做到当时最优。在 CV 中，超列几乎已被端到端微调取代 Long et al. (2015a)。

Multi-task learning

A related direction is multi-task learning (MTL) Caruana (1993). This is the approach taken by Rei (2017) and Liu et al. (2018) who add a language modeling objective to the model that is trained jointly with the main task model. MTL requires the tasks to be trained from scratch every time, which makes it inefficient and often requires careful weighting of the task-specific objective functions Chen et al. (2017).

多任务学习

相关方向是多任务学习（MTL）Caruana (1993)。Rei (2017) 与 Liu et al. (2018) 把语言建模目标加进与主任务联合训练的模型。MTL 每次都要从零训各任务，效率低，还常要仔细权衡任务目标 Chen et al. (2017)。

Fine-tuning

Fine-tuning has been used successfully to transfer between similar tasks, e.g. in QA Min et al. (2017), for distantly supervised sentiment analysis Severyn and Moschitti (2015), or MT domains Sennrich et al. (2015) but has been shown to fail between unrelated ones Mou et al. (2016). Dai and Le (2015) also fine-tune a language model, but overfit with $10$k labeled examples and require millions of in-domain documents for good performance. In contrast, ULMFiT leverages general-domain pretraining and novel fine-tuning techniques to prevent overfitting even with only $100$ labeled examples and achieves state-of-the-art results also on small datasets.

微调

微调已成功用于相近任务间迁移，如问答 Min et al. (2017)、远监督情感 Severyn and Moschitti (2015)、机器翻译领域适应 Sennrich et al. (2015)，但在无关任务间被证明会失败 Mou et al. (2016)。Dai and Le (2015) 也微调语言模型，却在 $10$k 标注上过拟合，并要上百万领域内文档才好。相比之下，ULMFiT 借助通用域预训练与新的微调技巧，即使只有 $100$ 条标注也不易过拟合，小数据集上同样达到当时最优。

(a) LM pre-training

(a) 语言模型预训练

3 Universal Language Model Fine-tuning

3 通用语言模型微调

We are interested in the most general inductive transfer learning setting for NLP Pan and Yang (2010): Given a static source task $\mathcal{T}_{S}$ and any target task $\mathcal{T}_{T}$ with $\mathcal{T}_{S}\neq\mathcal{T}_{T}$, we would like to improve performance on $\mathcal{T}_{T}$. Language modeling can be seen as the ideal source task and a counterpart of ImageNet for NLP: It captures many facets of language relevant for downstream tasks, such as long-term dependencies Linzen et al. (2016), hierarchical relations Gulordava et al. (2018), and sentiment Radford et al. (2017). In contrast to tasks like MT McCann et al. (2017) and entailment Conneau et al. (2017), it provides data in near-unlimited quantities for most domains and languages. Additionally, a pretrained LM can be easily adapted to the idiosyncrasies of a target task, which we show significantly improves performance (see Section 5). Moreover, language modeling already is a key component of existing tasks such as MT and dialogue modeling. Formally, language modeling induces a hypothesis space $\mathcal{H}$ that should be useful for many other NLP tasks Vapnik and Kotz (1982); Baxter (2000).

我们关注 NLP 里最一般的归纳迁移设定 Pan and Yang (2010)：给定静态源任务 $\mathcal{T}_{S}$ 与任意目标任务 $\mathcal{T}_{T}$（$\mathcal{T}_{S}\neq\mathcal{T}_{T}$），希望提升 $\mathcal{T}_{T}$ 上的表现。语言建模可视为理想源任务，相当于 NLP 的 ImageNet：它捕获下游任务需要的多面语言性质，如长程依赖 Linzen et al. (2016)、层级关系 Gulordava et al. (2018) 与情感 Radford et al. (2017)。相对机器翻译 McCann et al. (2017) 与蕴含 Conneau et al. (2017)，多数领域与语言都能提供近乎无限的数据。预训练 LM 也易于适配目标任务的特异之处，我们证明这会显著提升表现（见第 5 节）。语言建模本身也已是机器翻译、对话建模等任务的关键组件。形式上看，语言建模诱导的假设空间 $\mathcal{H}$ 应对许多其他 NLP 任务有用 Vapnik and Kotz (1982); Baxter (2000)。

We propose Universal Language Model Fine-tuning (ULMFiT), which pretrains a language model (LM) on a large general-domain corpus and fine-tunes it on the target task using novel techniques. The method is universal in the sense that it meets these practical criteria: 1) It works across tasks varying in document size, number, and label type; 2) it uses a single architecture and training process; 3) it requires no custom feature engineering or preprocessing; and 4) it does not require additional in-domain documents or labels.

我们提出通用语言模型微调（ULMFiT）：在大型通用语料上预训练语言模型，再用新技巧在目标任务上微调。称其通用，是因为它满足这些实用标准：1）适用于文档长度、数量与标签类型各异的任务；2）单一架构与训练流程；3）无需定制特征工程或预处理；4）不额外要求领域内文档或标注。

In our experiments, we use the state-of-the-art language model AWD-LSTM Merity et al. (2017a), a regular LSTM (with no attention, short-cut connections, or other sophisticated additions) with various tuned dropout hyperparameters. Analogous to CV, we expect that downstream performance can be improved by using higher-performance language models in the future.

实验中我们采用当时领先的语言模型 AWD-LSTM Merity et al. (2017a)：普通 LSTM（无注意力、捷径或其他复杂附加结构），配合多组调好的 dropout。与 CV 类似，我们预期未来用更强语言模型还能继续抬高下游表现。

ULMFiT consists of the following steps, which we show in Figure 1: a) General-domain LM pretraining (§3.1); b) target task LM fine-tuning (§3.2); and c) target task classifier fine-tuning (§3.3). We discuss these in the following sections.

ULMFiT 包含下列步骤（见图 1）：a）通用域 LM 预训练（§3.1）；b）目标任务 LM 微调（§3.2）；c）目标任务分类器微调（§3.3）。下文分节讨论。

3.1 General-domain LM pretraining

3.1 通用域语言模型预训练

An ImageNet-like corpus for language should be large and capture general properties of language. We pretrain the language model on Wikitext-103 Merity et al. (2017b) consisting of 28,595 preprocessed Wikipedia articles and 103 million words. Pretraining is most beneficial for tasks with small datasets and enables generalization even with $100$ labeled examples. We leave the exploration of more diverse pretraining corpora to future work, but expect that they would boost performance. While this stage is the most expensive, it only needs to be performed once and improves performance and convergence of downstream models.

类似 ImageNet 的语言语料应足够大，并能刻画语言的一般性质。我们在 Wikitext-103 Merity et al. (2017b) 上预训练：含 28,595 篇预处理维基文章、约 1.03 亿词。预训练对小数据任务最有益，即使只有 $100$ 条标注也能泛化。更多样的预训练语料留给未来工作，但我们预期会进一步提升。该阶段最耗时，只需做一次，却能改善下游模型的表现与收敛。

3.2 Target task LM fine-tuning

3.2 目标任务语言模型微调

No matter how diverse the general-domain data used for pretraining is, the data of the target task will likely come from a different distribution. We thus fine-tune the LM on data of the target task. Given a pretrained general-domain LM, this stage converges faster as it only needs to adapt to the idiosyncrasies of the target data, and it allows us to train a robust LM even for small datasets. We propose discriminative fine-tuning and slanted triangular learning rates for fine-tuning the LM, which we introduce in the following.

无论预训练用的通用数据多么多样，目标任务数据多半来自不同分布。因此我们在目标任务数据上再微调 LM。有了预训练的通用域 LM，这一步收敛更快：只需适配目标数据的特异之处，即使小数据集也能训出稳健 LM。我们提出区分式微调与斜三角学习率用于微调 LM，下文介绍。

Discriminative fine-tuning

As different layers capture different types of information Yosinski et al. (2014), they should be fine-tuned to different extents. To this end, we propose a novel fine-tuning method, discriminative fine-tuning[^4].

区分式微调

不同层捕获不同类型的信息 Yosinski et al. (2014)，微调幅度也应有所不同。为此我们提出新方法：区分式微调[^4]。

Instead of using the same learning rate for all layers of the model, discriminative fine-tuning allows us to tune each layer with different learning rates. For context, the regular stochastic gradient descent (SGD) update of a model’s parameters $\theta$ at time step $t$ looks like the following Ruder (2016):

不再对模型所有层用同一学习率，区分式微调允许对每一层使用不同学习率。作为背景，模型参数 $\theta$ 在时刻 $t$ 的常规随机梯度下降（SGD）更新形如 Ruder (2016)：

$$\theta_{t}=\theta_{t-1}-\eta\cdot\nabla_{\theta}J(\theta) \tag{1}$$

where $\eta$ is the learning rate and $\nabla_{\theta}J(\theta)$ is the gradient with regard to the model’s objective function. For discriminative fine-tuning, we split the parameters $\theta$ into $\{\theta^{1},\ldots,\theta^{L}\}$ where $\theta^{l}$ contains the parameters of the model at the $l$-th layer and $L$ is the number of layers of the model. Similarly, we obtain $\{\eta^{1},\ldots,\eta^{L}\}$ where $\eta^{l}$ is the learning rate of the $l$-th layer.

其中 $\eta$ 为学习率，$\nabla_{\theta}J(\theta)$ 为目标函数对参数的梯度。做区分式微调时，把参数 $\theta$ 拆成 $\{\theta^{1},\ldots,\theta^{L}\}$，$\theta^{l}$ 为第 $l$ 层参数，$L$ 为层数。同理得到 $\{\eta^{1},\ldots,\eta^{L}\}$，$\eta^{l}$ 为第 $l$ 层学习率。

The SGD update with discriminative fine-tuning is then the following:

带区分式微调的 SGD 更新即为：

$$\theta_{t}^{l}=\theta_{t-1}^{l}-\eta^{l}\cdot\nabla_{\theta^{l}}J(\theta) \tag{2}$$

We empirically found it to work well to first choose the learning rate $\eta^{L}$ of the last layer by fine-tuning only the last layer and using $\eta^{l-1}=\eta^{l}/2.6$ as the learning rate for lower layers.

经验上，先只微调最后一层以选定 $\eta^{L}$，再对更低层取 $\eta^{l-1}=\eta^{l}/2.6$，效果较好。

Slanted triangular learning rates

For adapting its parameters to task-specific features, we would like the model to quickly converge to a suitable region of the parameter space in the beginning of training and then refine its parameters. Using the same learning rate (LR) or an annealed learning rate throughout training is not the best way to achieve this behaviour. Instead, we propose slanted triangular learning rates (STLR), which first linearly increases the learning rate and then linearly decays it according to the following update schedule, which can be seen in Figure 2:

斜三角学习率

为使参数适配任务特征，我们希望训练初期快速收敛到合适参数区域，再精细调整。全程固定学习率（LR）或单调退火，都不是实现该行为的最佳方式。我们提出斜三角学习率（STLR）：先线性升高学习率，再线性衰减，更新日程如下（见图 2）：

$$\begin{split}cut&=\lfloor T\cdot cut_frac\rfloor\
p&=\begin{cases}t/cut,&\text{if}\ t<cut\
1-\frac{t-cut}{cut\cdot(1/cut_frac-1)},&\text{otherwise}\end{cases}\
\eta_{t}&=\eta_{max}\cdot\frac{1+p\cdot(ratio-1)}{ratio}\end{split} \tag{3}$$

where $T$ is the number of training iterations[^5], $cut\_frac$ is the fraction of iterations we increase the LR, $cut$ is the iteration when we switch from increasing to decreasing the LR, $p$ is the fraction of the number of iterations we have increased or will decrease the LR respectively, $ratio$ specifies how much smaller the lowest LR is from the maximum LR $\eta_{max}$, and $\eta_{t}$ is the learning rate at iteration $t$. We generally use $cut\_frac=0.1$, $ratio=32$ and $\eta_{max}=0.01$.

其中 $T$ 为训练迭代次数[^5]，$cut\_frac$ 为升高 LR 的迭代比例，$cut$ 为从升高转为降低的迭代点，$p$ 分别为已升高或将降低部分所占比例，$ratio$ 规定最低 LR 相对最大 LR $\eta_{max}$ 小多少，$\eta_{t}$ 为第 $t$ 次迭代的学习率。通常取 $cut\_frac=0.1$、$ratio=32$、$\eta_{max}=0.01$。

STLR modifies triangular learning rates Smith (2017) with a short increase and a long decay period, which we found key for good performance.[^6] In Section 5, we compare against aggressive cosine annealing, a similar schedule that has recently been used to achieve state-of-the-art performance in CV Loshchilov and Hutter (2017).[^7]

STLR 改写了三角学习率 Smith (2017)：短升长降，我们发现这对表现很关键。[^6] 第 5 节会与激进余弦退火对比；后者近期被用于 CV 中取得当时最优 Loshchilov and Hutter (2017)。[^7]

Figure 2: The slanted triangular learning rate schedule used for ULMFiT as a function of the number of training iterations.

图 2：ULMFiT 所用斜三角学习率日程，随训练迭代次数变化。

3.3 Target task classifier fine-tuning

3.3 目标任务分类器微调

Finally, for fine-tuning the classifier, we augment the pretrained language model with two additional linear blocks. Following standard practice for CV classifiers, each block uses batch normalization Ioffe and Szegedy (2015) and dropout, with ReLU activations for the intermediate layer and a softmax activation that outputs a probability distribution over target classes at the last layer. Note that the parameters in these task-specific classifier layers are the only ones that are learned from scratch. The first linear layer takes as the input the pooled last hidden layer states.

最后，微调分类器时，我们在预训练语言模型上再加两个线性块。沿用 CV 分类器惯例，每块含批归一化 Ioffe and Szegedy (2015) 与 dropout，中间层用 ReLU，最后一层 softmax 输出目标类别概率。注意：这些任务相关分类层的参数是唯一从零学习的部分。第一个线性层的输入为池化后的末隐状态。

Concat pooling

The signal in text classification tasks is often contained in a few words, which may occur anywhere in the document. As input documents can consist of hundreds of words, information may get lost if we only consider the last hidden state of the model. For this reason, we concatenate the hidden state at the last time step $\mathbf{h}_{T}$ of the document with both the max-pooled and the mean-pooled representation of the hidden states over as many time steps as fit in GPU memory $\mathbf{H}=\{\mathbf{h}_{1},\ldots,\mathbf{h}_{T}\}$:

拼接池化

文本分类的信号常集中在少数词上，且可能出现在文档任意位置。输入可达数百词，若只看模型最后隐状态，信息易丢失。因此我们把文档最后时间步隐状态 $\mathbf{h}_{T}$，与在 GPU 内存允许范围内尽可能多的时间步隐状态 $\mathbf{H}=\{\mathbf{h}_{1},\ldots,\mathbf{h}_{T}\}$ 上的最大池化与平均池化表示拼接：

$$\mathbf{h}_{c}=[\mathbf{h}_{T},\mathtt{maxpool}(\mathbf{H}),\mathtt{meanpool}(\mathbf{H})] \tag{4}$$

where $[]$ is concatenation.

其中 $[]$ 表示拼接。

Fine-tuning the target classifier is the most critical part of the transfer learning method. Overly aggressive fine-tuning will cause catastrophic forgetting, eliminating the benefit of the information captured through language modeling; too cautious fine-tuning will lead to slow convergence (and resultant overfitting). Besides discriminative fine-tuning and triangular learning rates, we propose gradual unfreezing for fine-tuning the classifier.

微调目标分类器是迁移方法里最关键的一环。微调过猛会造成灾难性遗忘，抹掉语言建模积累的信息；过谨慎则收敛慢（并容易过拟合）。除区分式微调与三角学习率外，我们还提出逐步解冻来微调分类器。

Gradual unfreezing

Rather than fine-tuning all layers at once, which risks catastrophic forgetting, we propose to gradually unfreeze the model starting from the last layer as this contains the least general knowledge Yosinski et al. (2014): We first unfreeze the last layer and fine-tune all unfrozen layers for one epoch. We then unfreeze the next lower frozen layer and repeat, until we fine-tune all layers until convergence at the last iteration. This is similar to ‘chain-thaw’ Felbo et al. (2017), except that we add a layer at a time to the set of ‘thawed’ layers, rather than only training a single layer at a time.

逐步解冻

与其一次性微调所有层（易灾难性遗忘），我们主张从最后一层开始逐步解冻：该层含最不通用的知识 Yosinski et al. (2014)。先解冻最后一层，对所有已解冻层训一个 epoch；再解冻下一层冻结层并重复，直到最后一次迭代时对全部层微调至收敛。这类似「链式解冻」Felbo et al. (2017)，不同处在于我们每次把一层加入「已解冻」集合，而不是每次只训单层。

While discriminative fine-tuning, slanted triangular learning rates, and gradual unfreezing all are beneficial on their own, we show in Section 5 that they complement each other and enable our method to perform well across diverse datasets.

区分式微调、斜三角学习率与逐步解冻各自都有益，第 5 节将表明它们相互补充，使方法在多样数据集上表现稳健。

Dataset	Type	# classes	# examples
TREC-6	Question	6	5.5k
IMDb	Sentiment	2	25k
Yelp-bi	Sentiment	2	560k
Yelp-full	Sentiment	5	650k
AG	Topic	4	120k
DBpedia	Topic	14	560k

Table 1: Text classification datasets and tasks with number of classes and training examples.

数据集	类型	类别数	样本数
TREC-6	问题	6	5.5k
IMDb	情感	2	25k
Yelp-bi	情感	2	560k
Yelp-full	情感	5	650k
AG	主题	4	120k
DBpedia	主题	14	560k

表 1：文本分类数据集与任务，含类别数与训练样本数。

BPTT for Text Classification (BPT3C)

Language models are trained with backpropagation through time (BPTT) to enable gradient propagation for large input sequences. In order to make fine-tuning a classifier for large documents feasible, we propose BPTT for Text Classification (BPT3C): We divide the document into fixed-length batches of size $b$. At the beginning of each batch, the model is initialized with the final state of the previous batch; we keep track of the hidden states for mean and max-pooling; gradients are back-propagated to the batches whose hidden states contributed to the final prediction. In practice, we use variable length backpropagation sequences Merity et al. (2017a).

面向文本分类的 BPTT（BPT3C）

语言模型用随时间反向传播（BPTT）训练，以便在长输入上传递梯度。为使长文档上的分类器微调可行，我们提出面向文本分类的 BPTT（BPT3C）：把文档切成固定长度、大小为 $b$ 的批次。每批开始时用上一批的最终状态初始化；我们跟踪用于均值与最大池化的隐状态；梯度回传到那些对最终预测有贡献的批次隐状态。实践中采用可变长度反向传播序列 Merity et al. (2017a)。

Bidirectional language model

Similar to existing work Peters et al. (2017); Peters et al. (2018), we are not limited to fine-tuning a unidirectional language model. For all our experiments, we pretrain both a forward and a backward LM. We fine-tune a classifier for each LM independently using BPT3C and average the classifier predictions.

双向语言模型

与已有工作类似 Peters et al. (2017); Peters et al. (2018)，我们不限于微调单向语言模型。全部实验中，我们同时预训练前向与后向 LM。各自用 BPT3C 独立微调分类器，再对预测取平均。

4 Experiments

4 实验

While our approach is equally applicable to sequence labeling tasks, we focus on text classification tasks in this work due to their important real-world applications.

本方法同样适用于序列标注，但本文聚焦文本分类，因其现实应用重要。

	Model	Test		Model	Test
IMDb	CoVe McCann et al. (2017)	8.2	TREC-6	CoVe McCann et al. (2017)	4.2
oh-LSTM Johnson and Zhang (2016)	5.9	TBCNN Mou et al. (2015)	4.0
Virtual Miyato et al. (2016)	5.9	LSTM-CNN Zhou et al. (2016)	3.9
ULMFiT (ours)	4.6	ULMFiT (ours)	3.6

Table 2: Test error rates (%) on two text classification datasets used by McCann et al. (2017).

	模型	测试		模型	测试
IMDb	CoVe McCann et al. (2017)	8.2	TREC-6	CoVe McCann et al. (2017)	4.2
oh-LSTM Johnson and Zhang (2016)	5.9	TBCNN Mou et al. (2015)	4.0
Virtual Miyato et al. (2016)	5.9	LSTM-CNN Zhou et al. (2016)	3.9
ULMFiT（本文）	4.6	ULMFiT（本文）	3.6

表 2：McCann et al. (2017) 所用两个文本分类数据集上的测试误差率（%）。

	AG	DBpedia	Yelp-bi	Yelp-full
Char-level CNN Zhang et al. (2015)	9.51	1.55	4.88	37.95
CNN Johnson and Zhang (2016)	6.57	0.84	2.90	32.39
DPCNN Johnson and Zhang (2017)	6.87	0.88	2.64	30.58
ULMFiT (ours)	5.01	0.80	2.16	29.98

Table 3: Test error rates (%) on text classification datasets used by Johnson and Zhang (2017).

	AG	DBpedia	Yelp-bi	Yelp-full
字符级 CNN Zhang et al. (2015)	9.51	1.55	4.88	37.95
CNN Johnson and Zhang (2016)	6.57	0.84	2.90	32.39
DPCNN Johnson and Zhang (2017)	6.87	0.88	2.64	30.58
ULMFiT（本文）	5.01	0.80	2.16	29.98

表 3：Johnson and Zhang (2017) 所用文本分类数据集上的测试误差率（%）。

4.1 Experimental setup

4.1 实验设置

Datasets and tasks

We evaluate our method on six widely-studied datasets, with varying numbers of documents and varying document length, used by state-of-the-art text classification and transfer learning approaches Johnson and Zhang (2017); McCann et al. (2017) as instances of three common text classification tasks: sentiment analysis, question classification, and topic classification. We show the statistics for each dataset and task in Table 1.

数据集与任务

我们在六个广泛研究的数据集上评估方法：文档数量与长度各异，被当时领先的文本分类与迁移工作 Johnson and Zhang (2017); McCann et al. (2017) 用作三类常见任务：情感分析、问题分类与主题分类。各数据集与任务统计见表 1。

Sentiment Analysis

For sentiment analysis, we evaluate our approach on the binary movie review IMDb dataset Maas et al. (2011) and on the binary and five-class version of the Yelp review dataset compiled by Zhang et al. (2015).

情感分析

情感分析上，我们评估二分类电影评论 IMDb Maas et al. (2011)，以及 Zhang et al. (2015) 整理的 Yelp 评论二分类与五分类版本。

Question Classification

We use the six-class version of the small TREC dataset Voorhees and Tice (1999) dataset of open-domain, fact-based questions divided into broad semantic categories.

问题分类

使用小型 TREC 数据集的六分类版本 Voorhees and Tice (1999)：开放域事实型问题，按粗粒度语义类别划分。

Topic classification

For topic classification, we evaluate on the large-scale AG news and DBpedia ontology datasets created by Zhang et al. (2015).

主题分类

主题分类上，评估 Zhang et al. (2015) 构建的大规模 AG 新闻与 DBpedia 本体数据集。

Pre-processing

We use the same pre-processing as in earlier work Johnson and Zhang (2017); McCann et al. (2017). In addition, to allow the language model to capture aspects that might be relevant for classification, we add special tokens for upper-case words, elongation, and repetition.

预处理

预处理与先前工作一致 Johnson and Zhang (2017); McCann et al. (2017)。此外，为让语言模型捕获可能与分类相关的方面，我们对大写词、拉长拼写与重复加入特殊标记。

Hyperparameters

We are interested in a model that performs robustly across a diverse set of tasks. To this end, if not mentioned otherwise, we use the same set of hyperparameters across tasks, which we tune on the IMDb validation set. We use the AWD-LSTM language model Merity et al. (2017a) with an embedding size of $400$, $3$ layers, $1150$ hidden activations per layer, and a BPTT batch size of $70$. We apply dropout of $0.4$ to layers, $0.3$ to RNN layers, $0.4$ to input embedding layers, $0.05$ to embedding layers, and weight dropout of $0.5$ to the RNN hidden-to-hidden matrix. The classifier has a hidden layer of size $50$. We use Adam with $\beta_{1}=0.7$ instead of the default $\beta_{1}=0.9$ and $\beta_{2}=0.99$, similar to Dozat and Manning (2017). We use a batch size of $64$, a base learning rate of $0.004$ and $0.01$ for fine-tuning the LM and the classifier respectively, and tune the number of epochs on the validation set of each task[^8]. We otherwise use the same practices used in Merity et al. (2017a).

超参数

我们希望模型在多样任务上稳健。除非另述，跨任务使用同一组超参，并在 IMDb 验证集上调好。采用 AWD-LSTM 语言模型 Merity et al. (2017a)：嵌入维 $400$，3 层，每层 1150 个隐激活，BPTT 批大小 70。对层、RNN 层、输入嵌入层、嵌入层分别施加 0.4、0.3、0.4、0.05 的 dropout，并对 RNN 隐到隐矩阵施加 0.5 的权重 dropout。分类器隐层大小为 50。Adam 取 $\beta_{1}=0.7$（默认常为 0.9）、$\beta_{2}=0.99$，类似 Dozat and Manning (2017)。批大小 64；微调 LM 与分类器的基础学习率分别为 0.004 与 0.01；epoch 数在各任务验证集上调[^8]。其余实践与 Merity et al. (2017a) 一致。

Baselines and comparison models

For each task, we compare against the current state-of-the-art. For the IMDb and TREC-6 datasets, we compare against CoVe McCann et al. (2017), a state-of-the-art transfer learning method for NLP. For the AG, Yelp, and DBpedia datasets, we compare against the state-of-the-art text categorization method by Johnson and Zhang (2017).

基线与对照模型

各任务对照当时最优。IMDb 与 TREC-6 对照 CoVe McCann et al. (2017)（当时领先的 NLP 迁移方法）。AG、Yelp 与 DBpedia 对照 Johnson and Zhang (2017) 的文本分类方法。

Figure 3: Validation error rates for supervised and semi-supervised ULMFiT vs. training from scratch with different numbers of training examples on IMDb, TREC-6, and AG (from left to right).

图 3：有监督与半监督 ULMFiT 相对从零训练，在不同训练样本数下的验证误差率；数据集从左到右为 IMDb、TREC-6、AG。

4.2 Results

4.2 结果

For consistency, we report all results as error rates (lower is better). We show the test error rates on the IMDb and TREC-6 datasets used by McCann et al. (2017) in Table 2. Our method outperforms both CoVe, a state-of-the-art transfer learning method based on hypercolumns, as well as the state-of-the-art on both datasets. On IMDb, we reduce the error dramatically by 43.9% and 22% with regard to CoVe and the state-of-the-art respectively. This is promising as the existing state-of-the-art requires complex architectures Peters et al. (2018), multiple forms of attention McCann et al. (2017) and sophisticated embedding schemes Johnson and Zhang (2016), while our method employs a regular LSTM with dropout. We note that the language model fine-tuning approach of Dai and Le (2015) only achieves an error of 7.64 vs. 4.6 for our method on IMDb, demonstrating the benefit of transferring knowledge from a large ImageNet-like corpus using our fine-tuning techniques. IMDb in particular is reflective of real-world datasets: Its documents are generally a few paragraphs long—similar to emails (e.g for legal discovery) and online comments (e.g for community management); and sentiment analysis is similar to many commercial applications, e.g. product response tracking and support email routing.

为一致起见，全部结果以误差率报告（越低越好）。表 2 给出 McCann et al. (2017) 所用 IMDb 与 TREC-6 上的测试误差。我们的方法同时超过基于超列的迁移方法 CoVe，以及两集上的当时最优。在 IMDb 上，相对 CoVe 与当时最优，误差分别大幅下降 43.9% 与 22%。这一点有启发性：现有最优往往依赖复杂架构 Peters et al. (2018)、多种注意力 McCann et al. (2017) 与精巧嵌入方案 Johnson and Zhang (2016)，而我们只用带 dropout 的普通 LSTM。Dai and Le (2015) 的语言模型微调在 IMDb 上误差仅 7.64，我们为 4.6，说明借助大型「类 ImageNet」语料与本文微调技巧迁移知识更有效。IMDb 尤其贴近真实数据：文档多为几段话——类似邮件（如电子证据开示）与在线评论（如社区管理）；情感分析也类似许多商业场景，如产品反馈跟踪与客服邮件路由。

On TREC-6, our improvement—similar as the improvements of state-of-the-art approaches—is not statistically significant, due to the small size of the 500-examples test set. Nevertheless, the competitive performance on TREC-6 demonstrates that our model performs well across different dataset sizes and can deal with examples that range from single sentences—in the case of TREC-6—to several paragraphs for IMDb. Note that despite pretraining on more than two orders of magnitude less data than the 7 million sentence pairs used by McCann et al. (2017), we consistently outperform their approach on both datasets.

在 TREC-6 上，我们的提升——与其他领先方法类似——因 500 条测试集过小而不具统计显著性。不过 TREC-6 上的竞争力表明：模型能跨不同数据规模工作，既能处理 TREC-6 的单句，也能处理 IMDb 的多段。尽管预训练数据比 McCann et al. (2017) 所用 700 万句对少两个数量级以上，我们在两集上仍持续超过他们。

We show the test error rates on the larger AG, DBpedia, Yelp-bi, and Yelp-full datasets in Table 3. Our method again outperforms the state-of-the-art significantly. On AG, we observe a similarly dramatic error reduction by 23.7% compared to the state-of-the-art. On DBpedia, Yelp-bi, and Yelp-full, we reduce the error by 4.8%, 18.2%, 2.0% respectively.

表 3 给出更大的 AG、DBpedia、Yelp-bi、Yelp-full 上的测试误差。方法再次显著超过当时最优。AG 上相对最优误差下降 23.7%。DBpedia、Yelp-bi、Yelp-full 上分别降 4.8%、18.2%、2.0%。

5 Analysis

5 分析

In order to assess the impact of each contribution, we perform a series of analyses and ablations. We run experiments on three corpora, IMDb, TREC-6, and AG that are representative of different tasks, genres, and sizes. For all experiments, we split off $10\%$ of the training set and report error rates on this validation set with unidirectional LMs. We fine-tune the classifier for $50$ epochs and train all methods but ULMFiT with early stopping.

为评估各项贡献的影响，我们做了一系列分析与消融。实验在 IMDb、TREC-6、AG 三个语料上进行，分别代表不同任务、体裁与规模。全部实验从训练集划出 10%，在该验证集上用单向 LM 报误差率。分类器微调 50 个 epoch；除 ULMFiT 外，其他方法均用早停。

Low-shot learning

One of the main benefits of transfer learning is being able to train a model for a task with a small number of labels. We evaluate ULMFiT on different numbers of labeled examples in two settings: only labeled examples are used for LM fine-tuning (‘supervised’); and all task data is available and can be used to fine-tune the LM (‘semi-supervised’). We compare ULMFiT to training from scratch—which is necessary for hypercolumn-based approaches. We split off balanced fractions of the training data, keep the validation set fixed, and use the same hyperparameters as before. We show the results in Figure 3.

少样本学习

迁移学习的主要好处之一，是用很少标注就能训任务模型。我们在两种设定下评估不同标注量：仅用标注样本微调 LM（「有监督」）；任务全部数据可用、可用来微调 LM（「半监督」）。对照从零训练——超列类方法也必须如此。我们按比例切出平衡子集，验证集固定，超参与前文相同。结果见图 3。

On IMDb and AG, supervised ULMFiT with only $100$ labeled examples matches the performance of training from scratch with $10\times$ and $20\times$ more data respectively, clearly demonstrating the benefit of general-domain LM pretraining. If we allow ULMFiT to also utilize unlabeled examples ($50$k for IMDb, $100$k for AG), at $100$ labeled examples, we match the performance of training from scratch with $50\times$ and $100\times$ more data on AG and IMDb respectively. On TREC-6, ULMFiT significantly improves upon training from scratch; as examples are shorter and fewer, supervised and semi-supervised ULMFiT achieve similar results.

在 IMDb 与 AG 上，有监督 ULMFiT 仅用 $100$ 条标注，即可分别追上从零训练、$10\times$ 与 $20\times$ 数据的表现，清楚体现通用域 LM 预训练的价值。若还允许使用无标注样本（IMDb 50k、AG 100k），在 $100$ 条标注时，AG 与 IMDb 上可分别追上从零训练、$50\times$ 与 $100\times$ 数据。TREC-6 上 ULMFiT 显著优于从零训练；因样本更短更少，有监督与半监督结果接近。

Impact of pretraining

We compare using no pretraining with pretraining on WikiText-103 Merity et al. (2017b) in Table 4. Pretraining is most useful for small and medium-sized datasets, which are most common in commercial applications. However, even for large datasets, pretraining improves performance.

预训练的影响

表 4 比较不做预训练与在 WikiText-103 Merity et al. (2017b) 上预训练。预训练对中小数据集最有用，而商业应用里这类数据最常见。即便大数据集，预训练仍能提升表现。

Pretraining	IMDb	TREC-6	AG
Without pretraining	5.63	10.67	5.52
With pretraining	5.00	5.69	5.38

Table 4: Validation error rates for ULMFiT with and without pretraining.

预训练	IMDb	TREC-6	AG
无预训练	5.63	10.67	5.52
有预训练	5.00	5.69	5.38

表 4：ULMFiT 有无预训练的验证误差率。

LM	IMDb	TREC-6	AG
Vanilla LM	5.98	7.41	5.76
AWD-LSTM LM	5.00	5.69	5.38

Table 5: Validation error rates for ULMFiT with a vanilla LM and the AWD-LSTM LM.

语言模型	IMDb	TREC-6	AG
普通 LM	5.98	7.41	5.76
AWD-LSTM LM	5.00	5.69	5.38

表 5：普通 LM 与 AWD-LSTM LM 下 ULMFiT 的验证误差率。

Impact of LM quality

In order to gauge the importance of choosing an appropriate LM, we compare a vanilla LM with the same hyperparameters without any dropout[^9] with the AWD-LSTM LM with tuned dropout parameters in Table 5. Using our fine-tuning techniques, even a regular LM reaches surprisingly good performance on the larger datasets. On the smaller TREC-6, a vanilla LM without dropout runs the risk of overfitting, which decreases performance.

语言模型质量的影响

为衡量选用合适 LM 的重要性，表 5 比较同超参、无任何 dropout[^9] 的普通 LM，与带调好 dropout 的 AWD-LSTM。借助本文微调技巧，即便普通 LM 在较大数据集上也表现不俗。较小的 TREC-6 上，无 dropout 的普通 LM 易过拟合，表现下降。

Impact of LM fine-tuning

We compare no fine-tuning against fine-tuning the full model Erhan et al. (2010) (‘Full’), the most commonly used fine-tuning method, with and without discriminative fine-tuning (‘Discr’) and slanted triangular learning rates (‘Stlr’) in Table 6. Fine-tuning the LM is most beneficial for larger datasets. ‘Discr’ and ‘Stlr’ improve performance across all three datasets and are necessary on the smaller TREC-6, where regular fine-tuning is not beneficial.

语言模型微调的影响

表 6 比较：不微调；微调全模型 Erhan et al. (2010)（「Full」，最常用微调方式）；以及是否加区分式微调（「Discr」）与斜三角学习率（「Stlr」）。微调 LM 对较大数据集最有益。「Discr」与「Stlr」在三个数据集上都提升表现；在较小的 TREC-6 上它们是必要的，常规微调并无益处。

LM fine-tuning	IMDb	TREC-6	AG
No LM fine-tuning	6.99	6.38	6.09
Full	5.86	6.54	5.61
Full + discr	5.55	6.36	5.47
Full + discr + stlr	5.00	5.69	5.38

Table 6: Validation error rates for ULMFiT with different variations of LM fine-tuning.

LM 微调	IMDb	TREC-6	AG
不微调 LM	6.99	6.38	6.09
Full	5.86	6.54	5.61
Full + discr	5.55	6.36	5.47
Full + discr + stlr	5.00	5.69	5.38

表 6：不同 LM 微调变体下 ULMFiT 的验证误差率。

Impact of classifier fine-tuning

We compare training from scratch, fine-tuning the full model (‘Full’), only fine-tuning the last layer (‘Last’) Donahue et al. (2014), ‘Chain-thaw’ Felbo et al. (2017), and gradual unfreezing (‘Freez’). We furthermore assess the importance of discriminative fine-tuning (‘Discr’) and slanted triangular learning rates (‘Stlr’). We compare the latter to an alternative, aggressive cosine annealing schedule (‘Cos’) Loshchilov and Hutter (2017). We use a learning rate $\eta^{L}=0.01$ for ‘Discr’, learning rates of $0.001$ and $0.0001$ for the last and all other layers respectively for ‘Chain-thaw’ as in Felbo et al. (2017), and a learning rate of $0.001$ otherwise. We show the results in Table 7.

分类器微调的影响

我们比较：从零训练；微调全模型（「Full」）；只微调最后一层（「Last」）Donahue et al. (2014)；「Chain-thaw」Felbo et al. (2017)；以及逐步解冻（「Freez」）。并评估区分式微调（「Discr」）与斜三角学习率（「Stlr」）的重要性；后者对照激进余弦退火日程（「Cos」）Loshchilov and Hutter (2017)。「Discr」取 $\eta^{L}=0.01$；「Chain-thaw」对最后一层与其余层分别取 0.001 与 0.0001，同 Felbo et al. (2017)；其余取 0.001。结果见表 7。

Classifier fine-tuning	IMDb	TREC-6	AG
From scratch	9.93	13.36	6.81
Full	6.87	6.86	5.81
Full + discr	5.57	6.21	5.62
Last	6.49	16.09	8.38
Chain-thaw	5.39	6.71	5.90
Freez	6.37	6.86	5.81
Freez + discr	5.39	5.86	6.04
Freez + stlr	5.04	6.02	5.35
Freez + cos	5.70	6.38	5.29
Freez + discr + stlr	5.00	5.69	5.38

Table 7: Validation error rates for ULMFiT with different methods to fine-tune the classifier.

分类器微调	IMDb	TREC-6	AG
从零训练	9.93	13.36	6.81
Full	6.87	6.86	5.81
Full + discr	5.57	6.21	5.62
Last	6.49	16.09	8.38
Chain-thaw	5.39	6.71	5.90
Freez	6.37	6.86	5.81
Freez + discr	5.39	5.86	6.04
Freez + stlr	5.04	6.02	5.35
Freez + cos	5.70	6.38	5.29
Freez + discr + stlr	5.00	5.69	5.38

表 7：不同分类器微调方法下 ULMFiT 的验证误差率。

Fine-tuning the classifier significantly improves over training from scratch, particularly on the small TREC-6. ‘Last’, the standard fine-tuning method in CV, severely underfits and is never able to lower the training error to $0$. ‘Chain-thaw’ achieves competitive performance on the smaller datasets, but is outperformed significantly on the large AG. ‘Freez’ provides similar performance as ‘Full’. ‘Discr’ consistently boosts the performance of ‘Full’ and ‘Freez’, except for the large AG. Cosine annealing is competitive with slanted triangular learning rates on large data, but under-performs on smaller datasets. Finally, full ULMFiT classifier fine-tuning (bottom row) achieves the best performance on IMDB and TREC-6 and competitive performance on AG. Importantly, ULMFiT is the only method that shows excellent performance across the board—and is therefore the only universal method.

微调分类器相对从零训练提升显著，尤其在小的 TREC-6 上。「Last」作为 CV 标准微调方式严重欠拟合，训练误差从未降到 0。「Chain-thaw」在较小数据集上有竞争力，但在大的 AG 上明显落后。「Freez」与「Full」表现相近。「Discr」持续抬升「Full」与「Freez」，大 AG 除外。余弦退火在大数据上与斜三角学习率接近，小数据上则偏弱。完整的 ULMFiT 分类器微调（表底行）在 IMDb 与 TREC-6 上最佳，AG 上具竞争力。重要的是，ULMFiT 是唯一全面表现优异的方法，因而也是唯一真正通用的方法。

Classifier fine-tuning behavior

While our results demonstrate that how we fine-tune the classifier makes a significant difference, fine-tuning for inductive transfer is currently under-explored in NLP as it mostly has been thought to be unhelpful Mou et al. (2016). To better understand the fine-tuning behavior of our model, we compare the validation error of the classifier fine-tuned with ULMFiT and ‘Full’ during training in Figure 4.

分类器微调行为

结果说明如何微调分类器差别很大，但 NLP 里归纳迁移的微调仍探索不足，因人们多认为它无用 Mou et al. (2016)。为更好理解微调行为，图 4 比较训练过程中 ULMFiT 与「Full」微调分类器的验证误差。

Figure 4: Validation error rate curves for fine-tuning the classifier with ULMFiT and ‘Full’ on IMDb, TREC-6, and AG (top to bottom).

图 4：用 ULMFiT 与「Full」微调分类器时的验证误差曲线；自上而下为 IMDb、TREC-6、AG。

On all datasets, fine-tuning the full model leads to the lowest error comparatively early in training, e.g. already after the first epoch on IMDb. The error then increases as the model starts to overfit and knowledge captured through pretraining is lost. In contrast, ULMFiT is more stable and suffers from no such catastrophic forgetting; performance remains similar or improves until late epochs, which shows the positive effect of the learning rate schedule.

在所有数据集上，微调全模型会在训练较早阶段达到最低误差，例如 IMDb 上第一个 epoch 之后。随后误差回升：模型开始过拟合，预训练知识丢失。相比之下，ULMFiT 更稳定，无此类灾难性遗忘；表现维持或持续改善到后期 epoch，体现学习率日程的正面作用。

Impact of bidirectionality

At the cost of training a second model, ensembling the predictions of a forward and backwards LM-classifier brings a performance boost of around $0.5$–$0.7$. On IMDb we lower the test error from $5.30$ of a single model to $4.58$ for the bidirectional model.

双向性的影响

以训练第二个模型为代价，对前向与后向 LM 分类器预测做集成，可带来约 $0.5$–$0.7$ 的提升。IMDb 上单模型测试误差 $5.30$，双向模型降至 $4.58$。

6 Discussion and future directions

6 讨论与未来方向

While we have shown that ULMFiT can achieve state-of-the-art performance on widely used text classification tasks, we believe that language model fine-tuning will be particularly useful in the following settings compared to existing transfer learning approaches Conneau et al. (2017); McCann et al. (2017); Peters et al. (2018): a) NLP for non-English languages, where training data for supervised pretraining tasks is scarce; b) new NLP tasks where no state-of-the-art architecture exists; and c) tasks with limited amounts of labeled data (and some amounts of unlabeled data).

我们已表明 ULMFiT 可在常用文本分类上达到当时最优；相对已有迁移方法 Conneau et al. (2017); McCann et al. (2017); Peters et al. (2018)，我们认为语言模型微调在下列场景尤其有用：a）非英语 NLP，有监督预训练任务数据稀缺；b）尚无成熟最优架构的新 NLP 任务；c）标注有限（并有一定无标注数据）的任务。

Given that transfer learning and particularly fine-tuning for NLP is under-explored, many future directions are possible. One possible direction is to improve language model pretraining and fine-tuning and make them more scalable: for ImageNet, predicting far fewer classes only incurs a small performance drop Huh et al. (2016), while recent work shows that an alignment between source and target task label sets is important Mahajan et al. (2018)—focusing on predicting a subset of words such as the most frequent ones might retain most of the performance while speeding up training. Language modeling can also be augmented with additional tasks in a multi-task learning fashion Caruana (1993) or enriched with additional supervision, e.g. syntax-sensitive dependencies Linzen et al. (2016) to create a model that is more general or better suited for certain downstream tasks, ideally in a weakly-supervised manner to retain its universal properties.

鉴于 NLP 的迁移学习尤其是微调仍探索不足，未来方向很多。一条是改进语言模型预训练与微调并提高可扩展性：对 ImageNet，预测远更少类别只带来小幅掉点 Huh et al. (2016)，而近期工作表明源任务与目标任务标签集对齐很重要 Mahajan et al. (2018)——若只预测词表子集（如最高频词），或可在保大部分表现的同时加速训练。也可在多任务学习框架下为语言建模附加任务 Caruana (1993)，或加入额外监督（如对句法敏感的依赖 Linzen et al. (2016)），以得到更通用或更适特定下游任务的模型，最好以弱监督方式保留其通用性。

Another direction is to apply the method to novel tasks and models. While an extension to sequence labeling is straightforward, other tasks with more complex interactions such as entailment or question answering may require novel ways to pretrain and fine-tune. Finally, while we have provided a series of analyses and ablations, more studies are required to better understand what knowledge a pretrained language model captures, how this changes during fine-tuning, and what information different tasks require.

另一方向是把方法用到新任务与新模型。扩展到序列标注较直接；蕴含、问答等交互更复杂的任务，可能需要新的预训练与微调方式。最后，尽管我们已做系列分析与消融，仍需更多研究来弄清：预训练语言模型捕获了什么知识、微调时如何变化、不同任务又需要什么信息。

7 Conclusion

7 结论

We have proposed ULMFiT, an effective and extremely sample-efficient transfer learning method that can be applied to any NLP task. We have also proposed several novel fine-tuning techniques that in conjunction prevent catastrophic forgetting and enable robust learning across a diverse range of tasks. Our method significantly outperformed existing transfer learning techniques and the state-of-the-art on six representative text classification tasks. We hope that our results will catalyze new developments in transfer learning for NLP.

我们提出了 ULMFiT：一种有效且样本效率极高、可应用于任意 NLP 任务的迁移学习方法；并提出若干新的微调技巧，合在一起可防止灾难性遗忘，并在多样任务上实现稳健学习。该方法在六个代表性文本分类任务上显著超过已有迁移技术与当时最优。我们希望这些结果能推动 NLP 迁移学习的新发展。

Acknowledgments

致谢

We thank the anonymous reviewers for their valuable feedback. Sebastian is supported by Irish Research Council Grant Number EBPPG/2014/30 and Science Foundation Ireland Grant Number SFI/12/RC/2289.

感谢匿名审稿人的宝贵意见。Sebastian 受爱尔兰研究理事会资助编号 EBPPG/2014/30 与爱尔兰科学基金会资助编号 SFI/12/RC/2289 支持。

References

参考文献

Baxter (2000) Jonathan Baxter. 2000. A Model of Inductive Bias Learning. Journal of Artificial Intelligence Research 12:149–198.

Blitzer et al. (2007) John Blitzer, Mark Dredze, and Fernando Pereira. 2007. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification . Annual Meeting-Association for Computational Linguistics 45(1):440. https://doi.org/10.1109/IRPS.2011.5784441 .

Caragea et al. (2011) Cornelia Caragea, Nathan McNeese, Anuj Jaiswal, Greg Traylor, Hyun-Woo Kim, Prasenjit Mitra, Dinghao Wu, Andrea H Tapia, Lee Giles, Bernard J Jansen, et al. 2011. Classifying text messages for the haiti earthquake. In Proceedings of the 8th international conference on information systems for crisis response and management (ISCRAM2011) . Citeseer.

Caruana (1993) Rich Caruana. 1993. Multitask learning: A knowledge-based source of inductive bias. In Proceedings of the Tenth International Conference on Machine Learning .

Chen et al. (2017) Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. 2017. GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks pages 1–10.

Chu et al. (2012) Zi Chu, Steven Gianvecchio, Haining Wang, and Sushil Jajodia. 2012. Detecting automation of twitter accounts: Are you a human, bot, or cyborg? IEEE Transactions on Dependable and Secure Computing 9(6):811–824.

Conneau et al. (2017) Alexis Conneau, Douwe Kiela, Holger Schwenk, Loïc Barrault, and Antoine Bordes. 2017. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing .

Dai and Le (2015) Andrew M. Dai and Quoc V. Le. 2015. Semi-supervised Sequence Learning . Advances in Neural Information Processing Systems (NIPS ’15) http://arxiv.org/abs/1511.01432 .

Donahue et al. (2014) Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. 2014. Decaf: A deep convolutional activation feature for generic visual recognition. In International conference on machine learning . pages 647–655.

Dozat and Manning (2017) Timothy Dozat and Christopher D. Manning. 2017. Deep Biaffine Attention for Neural Dependency Parsing. In Proceedings of ICLR 2017 .

Erhan et al. (2010) Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. 2010. Why does unsupervised pre-training help deep learning? Journal of Machine Learning Research 11(Feb):625–660.

Felbo et al. (2017) Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing .

Gulordava et al. (2018) Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless green recurrent networks dream hierarchically. In Proceedings of NAACL-HLT 2018 .

Hariharan et al. (2015) Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. 2015. Hypercolumns for object segmentation and fine-grained localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . pages 447–456.

He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition .

Huang et al. (2017) Gao Huang, Zhuang Liu, Kilian Q. Weinberger, and Laurens van der Maaten. 2017. Densely Connected Convolutional Networks. In Proceedings of CVPR 2017 .

Huh et al. (2016) Minyoung Huh, Pulkit Agrawal, and Alexei A Efros. 2016. What makes ImageNet good for transfer learning? arXiv preprint arXiv:1608.08614 .

Ioffe and Szegedy (2015) Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning . pages 448–456.

Jindal and Liu (2007) Nitin Jindal and Bing Liu. 2007. Review spam detection. In Proceedings of the 16th international conference on World Wide Web . ACM, pages 1189–1190.

Johnson and Zhang (2016) Rie Johnson and Tong Zhang. 2016. Supervised and semi-supervised text categorization using lstm for region embeddings. In International Conference on Machine Learning . pages 526–534.

Johnson and Zhang (2017) Rie Johnson and Tong Zhang. 2017. Deep pyramid convolutional neural networks for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . volume 1, pages 562–570.

Linzen et al. (2016) Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of lstms to learn syntax-sensitive dependencies. arXiv preprint arXiv:1611.01368 .

Liu et al. (2018) Liyuan Liu, Jingbo Shang, Frank Xu, Xiang Ren, Huan Gui, Jian Peng, and Jiawei Han. 2018. Empower sequence labeling with task-aware neural language model. In Proceedings of AAAI 2018 .

Long et al. (2015a) Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015a. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . pages 3431–3440.

Long et al. (2015b) Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. 2015b. Learning Transferable Features with Deep Adaptation Networks. In Proceedings of the 32nd International Conference on Machine learning (ICML ’15) . volume 37.

Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017. SGDR: Stochastic Gradient Descent with Warm Restarts. In Proceedings of the Internal Conference on Learning Representations 2017 .

Maas et al. (2011) Andrew L Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1 . Association for Computational Linguistics, pages 142–150.

Mahajan et al. (2018) Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. 2018. Exploring the Limits of Weakly Supervised Pretraining .

McCann et al. (2017) Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in Translation: Contextualized Word Vectors. In Advances in Neural Information Processing Systems .

Merity et al. (2017a) Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2017a. Regularizing and Optimizing LSTM Language Models. arXiv preprint arXiv:1708.02182 .

Merity et al. (2017b) Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017b. Pointer Sentinel Mixture Models. In Proceedings of the International Conference on Learning Representations 2017 .

Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems .

Min et al. (2017) Sewon Min, Minjoon Seo, and Hannaneh Hajishirzi. 2017. Question Answering through Transfer Learning from Large Fine-grained Supervision Data. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Short Papers) .

Miyato et al. (2016) Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2016. Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725 .

Mou et al. (2016) Lili Mou, Zhao Meng, Rui Yan, Ge Li, Yan Xu, Lu Zhang, and Zhi Jin. 2016. How Transferable are Neural Networks in NLP Applications? Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing .

Mou et al. (2015) Lili Mou, Hao Peng, Ge Li, Yan Xu, Lu Zhang, and Zhi Jin. 2015. Discriminative neural sentence modeling by tree-based convolution. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing .

Ngai et al. (2011) EWT Ngai, Yong Hu, YH Wong, Yijun Chen, and Xin Sun. 2011. The application of data mining techniques in financial fraud detection: A classification framework and an academic review of literature. Decision Support Systems 50(3):559–569.

Pan and Yang (2010) Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22(10):1345–1359.

Peters et al. (2017) Matthew E Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. In Proceedings of ACL 2017 .

Peters et al. (2018) Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of NAACL 2018 .

Radford et al. (2017) Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444 .

Rei (2017) Marek Rei. 2017. Semi-supervised multitask learning for sequence labeling. In Proceedings of ACL 2017 .

Roitblat et al. (2010) Herbert L Roitblat, Anne Kershaw, and Patrick Oot. 2010. Document categorization in legal electronic discovery: computer classification vs. manual review. Journal of the Association for Information Science and Technology 61(1):70–80.

Ruder (2016) Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 .

Salakhutdinov and Hinton (2009) Ruslan Salakhutdinov and Geoffrey Hinton. 2009. Deep boltzmann machines. In Artificial Intelligence and Statistics . pages 448–455.

Sennrich et al. (2015) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709 .

Severyn and Moschitti (2015) Aliaksei Severyn and Alessandro Moschitti. 2015. UNITN: Training Deep Convolutional Neural Network for Twitter Sentiment Classification. Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015) pages 464–469.

Sharif Razavian et al. (2014) Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. 2014. Cnn features off-the-shelf: an astounding baseline for recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . pages 806–813.

Smith (2017) Leslie N Smith. 2017. Cyclical learning rates for training neural networks. In Applications of Computer Vision (WACV), 2017 IEEE Winter Conference on . IEEE, pages 464–472.

Vapnik and Kotz (1982) Vladimir Naumovich Vapnik and Samuel Kotz. 1982. Estimation of dependences based on empirical data , volume 40. Springer-Verlag New York.

Voorhees and Tice (1999) Ellen M Voorhees and Dawn M Tice. 1999. The trec-8 question answering track evaluation. In TREC . volume 1999, page 82.

Wieting and Gimpel (2017) John Wieting and Kevin Gimpel. 2017. Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL 2017) .

Yosinski et al. (2014) Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks? In Advances in neural information processing systems . pages 3320–3328.

Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems . pages 649–657.

Zhou et al. (2016) Peng Zhou, Zhenyu Qi, Suncong Zheng, Jiaming Xu, Hongyun Bao, and Bo Xu. 2016. Text classification improved by integrating bidirectional lstm with two-dimensional max pooling. In Proceedings of COLING 2016 .