Are Emergent Abilities of Large Language Models a Mirage?

Rylan Schaeffer Affiliation:  Computer Science, Stanford University    Brando Miranda Affiliation:  Computer Science, Stanford University    Sanmi Koyejo Affiliation:  Computer Science, Stanford University

Abstract

Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from not present to present, and their unpredictability, appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, emergent abilities appear due the researcher’s choice of metric rather than due to fundamental changes in model behavior with scale. Specifically, nonlinear or discontinuous metrics produce apparent emergent abilities, whereas linear or continuous metrics produce smooth, continuous, predictable changes in model performance. We present our alternative explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities, (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show how to choose metrics to produce never-before-seen seemingly emergent abilities in multiple vision tasks across diverse deep networks. Via all three analyses, we provide evidence that alleged emergent abilities evaporate with different metrics or with better statistics, and may not be a fundamental property of scaling AI models.

近期工作声称，大语言模型会表现出「涌现能力」：在较小规模模型上不存在、却在较大规模模型上出现的能力。涌现能力之所以引人注目，主要有两点：其一是尖锐性——从「不存在」到「存在」似乎瞬间切换；其二是不可预测性——似乎会在事先难以预见的模型规模上突然出现。本文提出另一种解释：对给定任务与模型族，在固定模型输出再做分析时，所谓涌现能力主要来自研究者所选指标，而非模型行为随规模发生根本性变化。具体而言，非线性或不连续指标会制造出看似涌现的现象；线性或连续指标则呈现平滑、连续、可预测的性能变化。我们先用一个简单数学模型阐述该替代解释，再用三种互补方式检验：(1) 基于 InstructGPT/GPT-3 族，在据称存在涌现的任务上提出、检验并证实关于指标选择效应的三条预测；(2) 对 BIG-Bench 上涌现能力做元分析，提出、检验并证实关于指标选择的两条预测；(3) 展示如何通过改选指标，在多种视觉任务与多种深度网络上人为制造此前未见的「看似涌现」能力。三项分析共同表明：所谓涌现能力在换用不同指标或改进统计后会消散，未必是 AI 模型缩放的基本属性。

1 Introduction

Emergent properties of complex systems have long been studied across disciplines, from physics to biology to mathematics. The idea of emergence was popularized by Nobel Prize-winning physicist P.W. Anderson’s “More Is Different” [1], which argues that as the complexity of a system increases, new properties may materialize that cannot be predicted even from a precise quantitative understanding of the system’s microscopic details. Recently, the idea of emergence gained significant attention in machine learning due to observations that large language models (LLMs) such as GPT [3], PaLM [6] and LaMDA [30] exhibit so-called “emergent abilities” [33, 8, 28, 3] (Fig. 1).

复杂系统的涌现性质长期跨学科被研究，从物理到生物再到数学。涌现观念因诺贝尔物理学奖得主 P.W. Anderson 的「More Is Different」[1] 而广为人知：该文主张，随着系统复杂性增加，可能出现即便精确理解微观细节也难以预测的新性质。近来，涌现概念在机器学习中再度受到关注：有观察称 GPT [3]、PaLM [6]、LaMDA [30] 等大语言模型表现出所谓「涌现能力」[33, 8, 28, 3]（图 1）。

The term “emergent abilities of LLMs” was recently and crisply defined as “abilities that are not present in smaller-scale models but are present in large-scale models; thus they cannot be predicted by simply extrapolating the performance improvements on smaller-scale models” [33]. Such emergent abilities were first discovered in the GPT-3 family [3]. Subsequent work emphasized the discovery, writing that “[although model] performance is predictable at a general level, performance on a specific task can sometimes emerge quite unpredictably and abruptly at scale” [8]. These quotations collectively identify the two defining properties of emergent abilities in LLMs:

「大语言模型的涌现能力」近期被清晰定义为：「在较小规模模型中不存在、却在大规模模型中存在的能力；因而无法仅靠外推较小规模模型上的性能提升来预测」[33]。这类涌现能力最早在 GPT-3 族中被发现 [3]。后续工作强调这一发现，写道：「尽管模型性能在整体层面可预测，但在特定任务上的表现有时会在规模达到某处时相当不可预测且突然地涌现」[8]。上述表述共同刻画了 LLM 涌现能力的两个定义性特征：

1. Sharpness, transitioning seemingly instantaneously from not present to present
2. Unpredictability, transitioning at seemingly unforeseeable model scales

1. 尖锐性：从「不存在」到「存在」似乎瞬间切换
2. 不可预测性：转换似乎发生在事先难以预见的模型规模

Figure 1: Emergent abilities of large language models. Model families display sharp and unpredictable increases in performance at specific tasks as scale increases. Source: Fig. 2 from [33].

图 1：大语言模型的涌现能力。随着规模增大，模型族在特定任务上的性能出现尖锐且不可预测的跃升。来源：[33] 图 2。

These emergent abilities have garnered significant interest, raising questions such as: What controls which abilities will emerge? What controls when abilities will emerge? How can we make desirable abilities emerge faster, and ensure undesirable abilities never emerge? These questions are especially pertinent to AI safety and alignment, as emergent abilities forewarn that larger models might one day, without warning, acquire undesired mastery over dangerous capabilities [29, 10, 17, 18].

这些涌现能力引发大量兴趣，并带出诸如：什么决定哪些能力会涌现？什么决定能力何时涌现？如何让合意能力更快出现，并确保不合意能力永不出现？这些问题对 AI 安全与对齐尤为关键——涌现能力似乎在预警：更大规模模型有朝一日可能毫无征兆地掌握危险能力 [29, 10, 17, 18]。

In this paper, we call into question the claim that LLMs possess emergent abilities, by which we specifically mean sharp and unpredictable changes in model outputs as a function of model scale on specific tasks. Our doubt stems from the observation that emergent abilities seem to appear only under metrics that nonlinearly or discontinuously scale any model’s per-token error rate. For instance, as we later show, $>92\%$ of emergent abilities on BIG-Bench tasks [28] (hand-annotated by [32]) appear under either of these two metrics:

本文质疑「LLM 具有涌现能力」这一主张——此处特指：在特定任务上，模型输出随模型规模发生尖锐且不可预测的变化。我们的疑虑来自一个观察：涌现能力似乎只在那些对任何模型的逐 token 错误率做非线性或不连续缩放的指标下才会出现。例如后文所示，BIG-Bench 任务 [28] 上（经 [32] 人工标注）超过 92% 的涌现能力，都出现在以下两种指标之一下：

$$\displaystyle\defeq,\begin{cases}1&\text{if highest probability mass on correct option}\
0&\text{otherwise}\end{cases} \ \displaystyle\defeq,\begin{cases}1&\text{if output string exactly matches target string}\
0&\text{otherwise}\end{cases}$$

This raises the possibility of an alternative explanation for the origin of LLMs’ emergent abilities: sharp and unpredictable changes might be induced by the researcher’s choice of measurement, even though the model family’s per-token error rate changes smoothly, continuously and predictably with increasing scale. Specifically, our alternative posits that emergent abilities are a mirage caused primarily by the researcher choosing a metric that nonlinearly or discontinuously deforms per-token error rates, and secondarily by possessing too few test data to accurately estimate the performance of smaller models, thereby causing smaller models to appear wholly unable to perform the task.

这就引出另一种关于 LLM 涌现能力来源的解释：尖锐且不可预测的变化，可能由研究者所选度量诱导，即便模型族的逐 token 错误率随规模平滑、连续、可预测地变化。具体地，我们的替代假说认为：涌现能力主要是一种幻觉，首要原因是研究者选用了对逐 token 错误率做非线性或不连续变形的指标；次要原因是测试数据过少，难以准确估计较小模型的表现，从而使较小模型看起来完全无法完成任务。

To communicate our alternative explanation, we present it as a simple mathematical model and demonstrate how it quantitatively reproduces the evidence offered in support of emergent abilities of LLMs. We then test our alternative explanation in three complementary ways:

为传达这一替代解释，我们先把它写成简单数学模型，并展示它如何定量复现支持「LLM 涌现能力」的证据。随后用三种互补方式检验该解释：

1. We make, test and confirm three predictions based on our alternative hypotheses using the InstructGPT [24] / GPT-3 [3] model family.
2. We meta-analyze published benchmarks [28, 33] to reveal that emergent abilities only appear for specific metrics, not for model families on particular tasks, and that changing the metric causes the emergence phenomenon to evaporate.
3. We induce never-before-seen, seemingly emergent abilities in multiple architectures across various vision tasks by intentionally changing the metrics used for evaluation.

1. 基于 InstructGPT [24] / GPT-3 [3] 模型族，提出、检验并证实源自替代假说的三条预测。
2. 对已发表基准 [28, 33] 做元分析，揭示涌现能力只出现在特定指标上，而非「模型族在某任务上」本身；更换指标后涌现现象消散。
3. 通过有意改换评估指标，在多种架构与多种视觉任务上诱导此前未见的、看似涌现的能力。

2 Alternative Explanation for Emergent Abilities

Figure 2: Emergent abilities of large language models are created by the researcher’s chosen metrics, not unpredictable changes in model behavior with scale. (A) Suppose the per-token cross-entropy loss decreases monotonically with model scale, e.g., $\mathcal{L}_{CE}$ scales as a power law. (B) The per-token probability of selecting the correct token asymptotes towards 1. (C) If the researcher scores models’ outputs using a nonlinear metric such as Accuracy (which requires a sequence of tokens to all be correct), the metric choice nonlinearly scales performance, causing performance to change sharply and unpredictably in a manner that qualitatively matches published emergent abilities (inset). (D) If the researcher instead scores models’ outputs using a discontinuous metric such as Multiple Choice Grade (akin to a step function), the metric choice discontinuously scales performance, again causing performance to change sharply and unpredictably. (E) Changing from a nonlinear metric to a linear metric such as Token Edit Distance, scaling shows smooth, continuous and predictable improvements, ablating the emergent ability. (F) Changing from a discontinuous metric to a continuous metric such as Brier Score again reveals smooth, continuous and predictable improvements in task performance. Consequently, emergent abilities are created by the researcher’s choice of metrics, not fundamental changes in model family behavior on specific tasks with scale.

图 2：大语言模型的涌现能力由研究者所选指标造成，而非模型行为随规模发生不可预测的变化。(A) 假设逐 token 交叉熵损失随模型规模单调下降，例如 $\mathcal{L}_{CE}$ 呈幂律缩放。(B) 选对正确 token 的逐 token 概率渐近趋向 1。(C) 若研究者用 Accuracy 等非线性指标给模型输出打分（要求一整段 token 全部正确），指标选择会对性能做非线性缩放，使性能出现尖锐、不可预测的变化，形态上与已发表的涌现能力相符（插图）。(D) 若改用 Multiple Choice Grade 这类不连续指标（类似阶跃函数），同样会因不连续缩放而呈现尖锐、不可预测的变化。(E) 把非线性指标换成 Token Edit Distance 这类线性指标后，缩放呈现平滑、连续、可预测的改进，涌现能力被消融。(F) 把不连续指标换成 Brier Score 这类连续指标，同样揭示平滑、连续、可预测的任务性能提升。因此，涌现能力来自研究者的指标选择，而非模型族在特定任务上随规模发生根本行为变化。

How might smooth, continuous, predictable changes in model family performance appear sharp and unpredictable? The answer is that the researcher’s choice of a nonlinear or discontinuous metric can distort the model family’s performance to appear sharp and unpredictable.

模型族性能本身平滑、连续、可预测的变化，为何会显得尖锐且不可预测？答案是：研究者选用非线性或不连续指标，会把模型族表现扭曲成尖锐且不可预测的样子。

To expound, suppose that within a model family, the test loss falls smoothly, continuously and predictably with the number of model parameters. One reason to believe this is the phenomenon known as neural scaling laws: empirical observations that deep networks exhibit power law scaling in the test loss as a function of training dataset size, number of parameters or compute [13, 27, 11, 16, 9, 12, 15, 34, 14, 7, 26]. For concreteness, suppose we have a model family of different numbers of parameters $N>0$ and assume that each model’s per-token cross entropy falls as a power law with the number of parameters $N$ for constants $c>0,\alpha<0$ (Fig. 2A):

展开来说，假设在同一模型族内，测试损失随参数量平滑、连续、可预测地下降。相信这一点的一个理由是神经缩放律：经验上，深度网络的测试损失相对训练数据量、参数量或算力呈幂律缩放[13, 27, 11, 16, 9, 12, 15, 34, 14, 7, 26]。为具体起见，设模型族参数量 $N>0$，并假设各模型的逐 token 交叉熵随 $N$ 按幂律下降，常数 $c>0,\alpha<0$（图 2A）：

$$\mathcal{L}_{CE}(N)=\Big(\frac{N}{c}\Big)^{\alpha}$$

To be clear, we do not require this particular functional form to hold; rather, we use it for illustrative purposes. Let $V$ denote the set of possible tokens, $p\in\Delta^{|V|-1}$ denote the true but unknown probability distribution, and $\hat{p}_{N}\in\Delta^{|V|-1}$ denote the $N$-parameter model’s predicted probability distribution. The per-token cross entropy as a function of number of parameters $N$ is:

需要强调：我们并不要求这一具体函数形式成立；此处只作说明之用。令 $V$ 为可能 token 集合，$p\in\Delta^{|V|-1}$ 为真实但未知的概率分布，$\hat{p}_{N}\in\Delta^{|V|-1}$ 为 $N$ 参数模型的预测分布。作为参数量 $N$ 函数的逐 token 交叉熵为：

$$\mathcal{L}_{CE}(N)\;\defeq\;-\sum_{v\in V}p(v)\log\hat{p}_{N}(v)$$

In practice, $p$ is unknown, so we substitute a one-hot distribution of the observed token $v^{*}$:

实践中 $p$ 未知，于是用观测到的 token $v^{*}$ 的 one-hot 分布代替：

$$\mathcal{L}_{CE}(N)=-\log\hat{p}_{N}(v^{*})$$

A model with $N$ parameters then has a per-token probability of selecting the correct token (Fig. 2B):

于是，$N$ 参数模型选对单个正确 token 的概率为（图 2B）：

$$p(\text{single token correct})=\exp\Big(-\mathcal{L}_{CE}(N)\Big)=\exp\Big(-(N/c)^{\alpha}\Big)$$

Suppose the researcher then chooses a metric that requires selecting $L$ tokens correctly. For example, our task might be $L$-digit integer addition, and a model’s output is scored $1$ if all $L$ output digits exactly match all target digits with no additions, deletions or substitutions, $0$ otherwise. If the probability each token is correct is independent[^1], the probability of scoring $1$ is:

再假设研究者选用要求连续 $L$ 个 token 全部选对的指标。例如任务是 $L$ 位整数加法：仅当全部 $L$ 个输出数字与目标完全一致、无增删改时记 1，否则记 0。若各 token 正确的概率相互独立[^1]，则得分为 1 的概率为：

$$\text{Accuracy}(N)\approx p_{N}(\text{single token correct})^{\text{num. of tokens}}=\exp\Big(-(N/c)^{\alpha}\Big)^{L}$$

This choice of metric nonlinearly scales performance with increasing token sequence length. When plotting performance on a linear-log plot, one sees a sharp, unpredictable emergent ability on longer sequences (Fig. 2C) that closely matches claimed emergent abilities (inset). What happens if the researcher switches from a nonlinear metric like Accuracy, under which the per-token error rate scales geometrically in target length (App. A.3), to an approximately linear metric like Token Edit Distance, under which the per-token error rate scales quasi-linearly in target length (App. A.2)?

这一指标选择使性能随目标序列长度非线性缩放。在线性–对数图上，更长序列上会出现尖锐、不可预测的「涌现能力」（图 2C），与声称的涌现现象高度相似（插图）。若研究者从 Accuracy 这类非线性指标——在其下逐 token 错误率相对目标长度呈几何缩放（附录 A.3）——换成近似线性的 Token Edit Distance——在其下逐 token 错误率相对目标长度近似拟线性缩放（附录 A.2）——又会怎样？

$$\text{Token Edit Distance}(N)\approx L\,\Big(1-p_{N}(\text{single token correct})\Big)=L\,\Big(1-\exp\big(-(N/c)^{\alpha}\big)\Big)$$

The linear metric reveals smooth, continuous, predictable changes in model performance (Fig. 2E). Similarly, if the researcher uses a discontinuous metric like Multiple Choice Grade, the researcher can find emergent abilities (Fig. 2D), but switching to a continuous metric like Brier Score removes the emergent ability (Fig. 2F). In summary, sharp and unpredictable changes with increasing scale can be fully explained by three interpretable factors: (1) the researcher choosing a metric that nonlinearly or discontinuously scales the per-token error rate, (2) having insufficient resolution to estimate model performance in the smaller parameter regime, with resolution[^2] set by $1/\text{test dataset size}$, and (3) insufficiently sampling the larger parameter regime.

线性指标揭示出模型性能平滑、连续、可预测的变化（图 2E）。类似地，若使用 Multiple Choice Grade 这类不连续指标，研究者可以发现「涌现能力」（图 2D）；换成 Brier Score 这类连续指标后，涌现能力消失（图 2F）。总之，随规模增大的尖锐且不可预测变化，可由三个可解释因素完全说明：(1) 研究者选用对逐 token 错误率做非线性或不连续缩放的指标；(2) 在较小参数区间缺乏足够分辨率来估计模型性能，分辨率[^2]由 $1/\text{测试集大小}$ 决定；(3) 对较大参数区间采样不足。

3 Analyzing InstructGPT/GPT-3’s Emergent Arithmetic Abilities

Figure 3: Claimed emergent abilities evaporate upon changing the metric. Left to Right: Mathematical Model, 2-Integer 2-Digit Multiplication Task, 2-Integer 4-Digit Addition Task. Top: When performance is measured by a nonlinear metric (e.g., Accuracy), the InstructGPT/GPT-3 [3, 24] family’s performance appears sharp and unpredictable on longer target lengths. Bottom: When performance is instead measured by a linear metric (e.g., Token Edit Distance), the family exhibits smooth, predictable performance improvements.

图 3：更换指标后，声称的涌现能力消散。 从左到右：数学模型、两整数两位数乘法任务、两整数四位数加法任务。上排：用非线性指标（如 Accuracy）度量时，InstructGPT/GPT-3 [3, 24] 族在更长目标长度上表现显得尖锐且不可预测。下排：改用线性指标（如 Token Edit Distance）后，该族呈现平滑、可预测的性能提升。

Previous papers prominently claimed the GPT [3, 24] family[^3] displays emergent abilities at integer arithmetic tasks [8, 28, 33] (Fig. 2E). We chose these tasks as they were prominently presented [3, 8, 28, 33], and we focused on the GPT family due to it being publicly queryable. As explained mathematically and visually in Sec. 2, our alternative explanation makes three predictions:

先前论文突出声称 GPT [3, 24] 族[^3] 在整数算术任务上表现出涌现能力 [8, 28, 33]（图 2E）。我们选择这些任务，是因为它们被显著展示 [3, 8, 28, 33]；并聚焦 GPT 族，因其可公开查询。如第 2 节的数学与图示所说明，我们的替代解释给出三条预测：

1. Changing the metric from a nonlinear or discontinuous metric (Fig. 2CD) to a linear or continuous metric (Fig. 2 EF) should reveal smooth, continuous, predictable performance improvement with model scale.
2. For nonlinear metrics, increasing the resolution of measured model performance by increasing the test dataset size should reveal smooth, continuous, predictable model improvements commensurate with the predictable nonlinear effect of the chosen metric.
3. Regardless of metric, increasing the target string length should predictably affect the model’s performance as a function of the length-1 target performance: approximately geometrically for accuracy and approximately quasilinearly for token edit distance.

1. 把指标从非线性或不连续（图 2CD）换成线性或连续（图 2EF），应揭示随模型规模平滑、连续、可预测的性能提升。
2. 对非线性指标，通过增大测试集提高所测性能的分辨率，应揭示与所选指标可预期的非线性效应相一致的、平滑连续可预测的改进。
3. 无论指标如何，增大目标串长度，应可预期地影响模型表现，且该影响由长度为 1 的目标表现决定：Accuracy 近似几何衰减，Token Edit Distance 近似拟线性衰减。

To test these predictions, we collected outputs from the InstructGPT/GPT-3 family on two tasks: 2-shot multiplication between two 2-digit integers and 2-shot addition between two 4-digit integers.

为检验这些预测，我们收集 InstructGPT/GPT-3 族在两个任务上的输出：两位数整数的 2-shot 乘法，以及四位数整数的 2-shot 加法。

Figure 4: Claimed emergent abilities evaporate upon using better statistics. Left to Right: Mathematical Model, 2-Integer 2-Digit Multiplication Task, 2-Integer 4-Digit Addition Task. Based on the predictable effect Accuracy has on performance, measuring performance requires high resolution. Generating additional test data increases the resolution and reveals that even on Accuracy, the InstructGPT/GPT-3 family’s [3, 24] performance is above chance and improves in a smooth, continuous, predictable manner that qualitatively matches the mathematical model.

图 4：改用更好统计后，声称的涌现能力消散。 从左到右：数学模型、两整数两位数乘法、两整数四位数加法。基于 Accuracy 对性能可预期的影响，度量性能需要高分辨率。生成更多测试数据以提高分辨率后可见：即便在 Accuracy 下，InstructGPT/GPT-3 族 [3, 24] 的表现也高于随机水平，并以平滑、连续、可预测的方式提升，形态上与数学模型相符。

Prediction: Emergent Abilities Disappear With Different Metrics

On both arithmetic tasks, the GPT family displays emergent abilities if the target has 4 or 5 digits and if the metric is Accuracy (Fig. 3, top) [3, 8, 33]. However, if one changes from nonlinear Accuracy to linear Token Edit Distance while keeping the models’ outputs fixed, the family’s performance smoothly, continuously and predictably improves with increasing scale (Fig. 3, bottom). This confirms our first prediction and supports our alternative explanation that the source of emergent abilities is the researcher’s choice of metric, not changes in the model family’s outputs. We also observe that under Token Edit Distance, increasing the length of the target string from 1 to 5 predictably decreases the family’s performance in an approximately quasilinear manner, confirming the first half of our third prediction.

预测：更换指标后涌现能力消失

在两个算术任务上，若目标有 4 或 5 位且指标为 Accuracy，GPT 族会呈现涌现能力（图 3 上）[3, 8, 33]。然而，在保持模型输出不变的前提下，把非线性 Accuracy 换成线性 Token Edit Distance 后，该族性能随规模平滑、连续、可预测地提升（图 3 下）。这证实了第一条预测，并支持我们的替代解释：涌现能力的来源是研究者的指标选择，而非模型族输出本身的变化。我们还观察到：在 Token Edit Distance 下，把目标串长度从 1 增到 5 会以近似拟线性方式可预期地降低族表现，从而证实第三条预测的前半部分。

Prediction: Emergent Abilities Disappear With Better Statistics

We next tested our second prediction: that even on nonlinear metrics such as accuracy, smaller models do not have zero accuracy, but rather have non-zero above-chance accuracy commensurate with choosing to use accuracy as the metric. In order to accurately measure models’ accuracy, we increased the resolution by generating additional test data, and found that on both arithmetic tasks, all models in the InstructGPT/GPT-3 family achieve above-chance accuracy (Fig. 4). This confirms our second prediction. We also observe that as the target string length increases, the accuracy falls approximately geometrically with the length of the target string, confirming the second half of our third prediction. These results additionally demonstrate that the researcher’s choice of metric has the effect that one should predict accuracy to have, i.e., geometric decay with the target length.

预测：改用更好统计后涌现能力消失

接着检验第二条预测：即便在 Accuracy 等非线性指标上，较小模型也并非零正确率，而是具有与「选用 Accuracy」这一选择相称的、高于随机的非零正确率。为准确测量正确率，我们通过生成更多测试数据提高分辨率，发现在两个算术任务上，InstructGPT/GPT-3 族中所有模型均达到高于随机的正确率（图 4）。这证实了第二条预测。我们还观察到：随目标串长度增加，正确率近似随长度几何下降，证实了第三条预测的后半部分。这些结果还表明，研究者所选指标产生的效应，正是人们应对 Accuracy 所预期的效应，即随目标长度几何衰减。

4 Meta-Analysis of Claimed Emergent Abilities

Analyzing the GPT family is possible because the models are publicly queryable. However, other model families claimed to exhibit emergent abilities are not publicly queryable, nor are their generated outputs publicly available, meaning we are limited to analyzing the published results themselves [8, 33, 32]. Our alternative explanation makes two predictions.

分析 GPT 族之所以可行，是因为模型可公开查询。但其他被声称具有涌现能力的模型族既不可公开查询，其生成输出也未公开，因而我们只能分析已发表结果本身 [8, 33, 32]。替代解释给出两条预测。

1. At the “population level” of Task-Metric-Model Family triplets, emergent abilities should appear predominantly on specific metrics, not task-model family pairs, and specifically with nonlinear and/or discontinuous metrics.
2. On individual Task-Metric-Model Family triplets that display an emergent ability, changing the metric to a linear and/or continuous metric should remove the emergent ability.

1. 在「任务–指标–模型族」三元组的「总体」层面，涌现能力应主要出现在特定指标上，而非任务–模型族对上，且尤其出现在非线性与/或不连续指标上。
2. 对显示涌现能力的单个任务–指标–模型族三元组，把指标换成线性与/或连续指标后，涌现能力应消失。

To test these predictions, we used to claimed emergent abilities on BIG-Bench [28, 33] due to the benchmark being pertinent and publicly available.

为检验这些预测，我们利用 BIG-Bench [28, 33] 上声称的涌现能力——该基准相关且公开可得。

Prediction: Emergent Abilities Should Appear with Metrics, not Task-Model Families

If emergent abilities are real, one should expect task-model family pairs to show emergence for all reasonable metrics. However, if our alternative explanation is correct, we should expect emergent abilities to appear only under certain metrics. To test this, we analyzed on which metrics emergent abilities appear. To determine whether a task-metric-model family triplet exhibits a possible emergent ability, we used a metric from previous work [28]. Letting $y_{i}\in\mathbb{R}$ denote model performance at model scales $x_{i}\in\mathbb{R}$, sorted such that $x_{i}<x_{i+1}$, the emergence score is:

预测：涌现能力应伴随指标出现，而非任务–模型族

若涌现能力真实存在，应预期任务–模型族对在所有合理指标下都显示涌现。但若替代解释正确，则应预期涌现能力只在某些指标下出现。为此，我们分析涌现能力出现在哪些指标上。判定一个任务–指标–模型族三元组是否可能呈现涌现能力时，我们采用先前工作的度量 [28]。令 $y_{i}\in\mathbb{R}$ 为模型在规模 $x_{i}\in\mathbb{R}$ 处的表现，并按 $x_{i}<x_{i+1}$ 排序，涌现分数为：

$$\text{Emergence Score}\Big(\Big\{(x_{n},y_{n})\Big\}_{n=1}^{N}\Big)\quad\defeq\quad\frac{\text{sign}(\arg\max_{i}y_{i}-\arg\min_{i}y_{i})(\max_{i}y_{i}-\min_{i}y_{i})}{\sqrt{\text{Median}(\{(y_{i}-y_{i-1})^{2}\}_{i})}} \tag{1}$$

Figure 5: Emergent abilities appear only for specific metrics, not task-model families. (A) Possible emergent abilities appear with at most 5 out of 39 BIG-Bench metrics. (B) Hand-annotated data by [32] reveals emergent abilities appear only under 4 preferred metrics. (C) $>92\%$ of emergent abilities appear under one of two metrics: Multiple Choice Grade and Exact String Match.

图 5：涌现能力只出现在特定指标上，而非任务–模型族。(A) 可能的涌现能力最多出现在 BIG-Bench 39 个指标中的 5 个上。(B) [32] 的人工标注显示，涌现能力只出现在 4 个首选指标下。(C) 超过 92% 的涌现能力出现在两个指标之一：Multiple Choice Grade 与 Exact String Match。

We found that most metrics used in BIG-Bench have zero task-model family pairs that exhibit emergent abilities: of the 39 preferred metrics in BIG-Bench, at most 5 display emergence (Fig. 5A). Many of the 5 are nonlinear and/or discontinuous, e.g., Exact String Match, Multiple Choice Grade, ROUGE-L-Sum (App. A.4). Notably, because BIG-Bench often scores models on tasks using multiple metrics, the lack of emergent abilities under other metrics suggests that emergent abilities do not appear when model outputs are scored using other metrics.

我们发现，BIG-Bench 使用的多数指标对零个任务–模型族对呈现涌现能力：在 39 个首选指标中，最多 5 个显示涌现（图 5A）。这 5 个里不少是非线性与/或不连续的，例如 Exact String Match、Multiple Choice Grade、ROUGE-L-Sum（附录 A.4）。值得注意的是，由于 BIG-Bench 常用多个指标给同一任务打分，在其他指标下缺乏涌现，意味着当用其他指标给模型输出打分时，涌现能力并不出现。

Because emergence score only suggests emergence, we also analyzed hand-annotated task-metric-model family triplets [32], which revealed emergent abilities appear with $4/39$ metrics (Fig. 5B), and 2 metrics account for $>92\%$ of claimed emergent abilities (Fig. 5C): Multiple Choice Grade and Exact String Match. Multiple Choice Grade is discontinuous, and Exact String Match is nonlinear.

由于涌现分数仅提示涌现，我们还分析了人工标注的任务–指标–模型族三元组 [32]：结果显示涌现能力出现在 $4/39$ 个指标上（图 5B），且两个指标贡献了超过 92% 的声称涌现（图 5C）：Multiple Choice Grade 与 Exact String Match。前者不连续，后者非线性。

Figure 6: Changing the metric when evaluating task-model family pairs causes emergent abilities to disappear. Left: The LaMDA model family displays emergent abilities when measured under the discontinuous Multiple Choice Grade. Right: The LaMDA model family’s emergent abilities disappear when measured under a continuous BIG-Bench metric: Brier Score.

图 6：更换评估任务–模型族对时所用的指标后，涌现能力消失。左：在不连续的 Multiple Choice Grade 下，LaMDA 模型族显示涌现能力。右：换成连续的 BIG-Bench 指标 Brier Score 后，LaMDA 的涌现能力消失。

Prediction: Changing Metric Removes Emergent Abilities

To test our second prediction, we focused on the LaMDA family [30] because its outputs are available through BIG-Bench. For our analysis, we identified tasks on which LaMDA displays emergent abilities with Multiple Choice Grade, then asked whether LaMDA still displays emergent abilities on the same tasks with a different BIG-Bench metric: Brier Score [2]. Brier Score is a strictly proper scoring rule for predictions of mutually exclusive outcomes; for a binary outcome, the Brier Score simplifies to the mean squared error between the outcome and its predicted probability mass. LaMDA’s emergent abilities on the discontinuous Multiple Choice Grade disappeared when we changed the metric to the continuous Brier Score (Fig. 6). These results support our alternative explanation that emergent abilities are induced by the chosen metric.

预测：更换指标可消除涌现能力

为检验第二条预测，我们聚焦 LaMDA 族 [30]，因其输出可通过 BIG-Bench 获得。分析中，我们找出 LaMDA 在 Multiple Choice Grade 下显示涌现的任务，再问：在同一任务上换用另一个 BIG-Bench 指标 Brier Score [2] 后，LaMDA 是否仍显示涌现。Brier Score 是互斥结果预测的严格真评分规则；对二元结果，它简化为结果与其预测概率质量之间的均方误差。LaMDA 在不连续 Multiple Choice Grade 上的涌现能力，在换成连续 Brier Score 后消失（图 6）。这些结果支持替代解释：涌现能力由所选指标诱导。

5 Inducing Emergent Abilities in Networks on Vision Tasks

To demonstrate how emergent abilities can be induced by the researcher’s choice of metric, we show how to produce emergent abilities in deep networks of various architectures: fully connected, convolutional, self-attentional. We focus on vision tasks because abrupt transitions in vision models’ capabilities have not been observed to the best of our knowledge; this is one reason why emergence in large language models is considered so interesting. For the convolutional example, see App. B.

为展示涌现能力如何由研究者的指标选择诱导，我们说明如何在多种架构的深度网络中制造涌现能力：全连接、卷积、自注意力。我们聚焦视觉任务，因为据我们所知，视觉模型能力尚未观察到突变；这也是大语言模型中涌现被认为格外有趣的原因之一。卷积例子见附录 B。

Emergent Reconstruction of CIFAR100 Natural Images by Nonlinear Autoencoders

Figure 7: Induced emergent reconstruction ability in shallow nonlinear autoencoders. (A) A published emergent ability at the BIG-Bench Periodic Elements task [28]. (B) Shallow nonlinear autoencoders trained on CIFAR100 [19] display smoothly decreasing mean squared reconstruction error. (C) Using a newly defined Reconstructionc metric (Eqn. 2) induces an unpredictable change.

非线性自编码器对 CIFAR100 自然图像的「涌现」重构

图 7：在浅层非线性自编码器中诱导出的涌现重构能力。(A) BIG-Bench Periodic Elements 任务上已发表的涌现能力 [28]。(B) 在 CIFAR100 [19] 上训练的浅层非线性自编码器，均方重构误差平滑下降。(C) 使用新定义的 Reconstruction*$c$ 指标（式 2）会诱导出不可预测的变化。

We first induce an emergent ability to reconstruct images in shallow (i.e., single hidden layer) nonlinear autoencoders trained on CIFAR100 natural images [19]. To emphasize that the sharpness of the metric is responsible for emergent abilities, and to show that sharpness extends to metrics beyond Accuracy, we intentionally define a discontinuous metric that measures a network’s ability to reconstruct a dataset as the average number of test data with squared reconstruction error below threshold $c$:

我们首先在 CIFAR100 自然图像 [19] 上训练的浅层（即单隐层）非线性自编码器中，诱导出重构图像的「涌现」能力。为强调指标的尖锐性才是涌现的原因，并说明尖锐性可延伸到 Accuracy 以外的指标，我们有意定义一个不连续指标：以平方重构误差低于阈值 $c$ 的测试样本平均数量，度量网络重构数据集的能力：

$$\text{Reconstruction}_{c}\Big(\{x_{n}\}_{n=1}^{N}\Big)\;\defeq\;\frac{1}{N}\sum_{n}\mathbb{I}\Big[||x_{n}-\hat{x}_{n}||^{2}<c\Big] \tag{2}$$

where $\mathbb{I}(\cdot)$ denotes an indicator variable and $\hat{x}_{n}$ is the autoencoder’s reconstruction of $x_{n}$. The autoencoder family displays smoothly decreasing squared reconstruction error as the number of bottleneck units increases (Fig. 7B). Under our newly defined Reconstructionc metric and for particular choices of $c$, the autoencoder family exhibits a sharp and seemingly unpredictable image reconstruction ability (Fig. 7C) that qualitatively matches published emergent abilities (Fig. 7A).

其中 $\mathbb{I}(\cdot)$ 为指示变量，$\hat{x}_{n}$ 为自编码器对 $x_{n}$ 的重构。随瓶颈单元数增加，自编码器族的平方重构误差平滑下降（图 7B）。在我们新定义的 Reconstruction*$c$ 指标下，对特定 $c$，该族表现出尖锐且看似不可预测的图像重构能力（图 7C），形态上与已发表的涌现能力相符（图 7A）。

Emergent Classification of Omniglot Characters by Autoregressive Transformers

Figure 8: Induced emergent classification ability in autoregressive Transformers. (A) A published emergent ability on the MMLU benchmark [8]. (B) Autoregressive transformers trained to classify Omniglot images display increasing accuracy with increasing scale. (C) When accuracy is redefined as classifying all images correctly, a seemingly emergent ability appears.

自回归 Transformer 对 Omniglot 字符的「涌现」分类

图 8：在自回归 Transformer 中诱导出的涌现分类能力。(A) MMLU 基准上已发表的涌现能力 [8]。(B) 训练用于分类 Omniglot 图像的自回归 Transformer，正确率随规模上升。(C) 把正确率重定义为全部图像都分类正确后，出现看似涌现的能力。

We next induce emergent abilities in Transformers [31] trained to autoregressively classify Omniglot handwritten characters [20], in a setup inspired by recent work [5]: Omniglot images are embedded by convolutional layers, then sequences of embedded image-image class label pairs are fed into decoder-only transformers. We measure image classification performance on sequences of length $L\in[1,5]$, again via subset accuracy: $1$ if all $L$ images are classified correctly (Fig. 8B), 0 otherwise. Causal transformers display a seemingly emergent ability to correctly classify Omniglot handwritten characters (Fig. 8C) that qualitatively matches published emergent abilities (Fig. 8A).

接着，我们在受近期工作启发的设定下 [5]，于自回归分类 Omniglot 手写字符 [20] 的 Transformer [31] 中诱导涌现能力：Omniglot 图像经卷积层嵌入，再将嵌入图像–类别标签对序列送入仅解码器 Transformer。我们在长度 $L\in[1,5]$ 的序列上度量图像分类表现，同样用子集正确率：仅当全部 $L$ 张图都分类正确时记 1（图 8B），否则记 0。因果 Transformer 表现出看似涌现的 Omniglot 手写字符正确分类能力（图 8C），形态上与已发表涌现能力相符（图 8A）。

6 Related Work

Srivastava et al. [28] observed that while accuracy at a particular task can empirically appear sharp and unpredictable, cross entropy does not; the authors then hypothesized that emergent abilities may be partially attributed to the metric. Our paper converts their discussion into precise predictions, then quantitatively tests the predictions to reveal that: metric choice is likely wholly responsible for emergent abilities; well-known and widely-used metrics (including ones already used by [28]) capture graded improvements; emergent abilities do not appear only for tasks involving multiple steps, and indeed appear most commonly on the discontinuous Multiple Choice Grade; metric choice can be used to induce emergent abilities in a novel domain (vision) in diverse architectures and tasks.

Srivastava 等人 [28] 观察到：尽管特定任务上的正确率在经验上可显得尖锐且不可预测，交叉熵却并非如此；作者随后假设涌现能力可能部分归因于指标。本文将其讨论转化为精确预测，并定量检验，从而揭示：指标选择很可能完全解释涌现能力；广为人知且广泛使用的指标（包括 [28] 已使用的）能够刻画渐进改进；涌现能力并非只出现在涉及多步的任务上，事实上最常出现在不连续的 Multiple Choice Grade 上；指标选择还可用来在新领域（视觉）的多种架构与任务中诱导涌现能力。

Caballero et al. [4] explain emergence by assuming a piece-wise power law functional form; under this view, emergent abilities are real, caused by a change in the governing power law. In contrast, our work suggests that emergent abilities are induced by the researcher, even under a single power law. Michaud et al. [25] posit that emergent abilities may be real under strong data assumptions.

Caballero 等人 [4] 通过假设分段幂律函数形式来解释涌现；在该视角下，涌现能力是真实的，由支配幂律的变化引起。相对地，我们的工作表明：即便在单一幂律下，涌现能力也可由研究者诱导。Michaud 等人 [25] 认为，在强数据假设下涌现能力可能真实存在。

7 Discussion

Our paper presents an alternative explanation for claimed emergent abilities of large language models. For a fixed task and a fixed model family, the researcher can choose a metric to create an emergent ability or choose a metric to ablate an emergent ability. Ergo, emergent abilities may be creations of the researcher’s choices, not a fundamental property of the model family on the specific task. We emphasize that nothing in this paper should be interpreted as claiming that large language models cannot display emergent abilities; rather, our message is that previously claimed emergent abilities in [3, 8, 28, 33] might likely be a mirage induced by researcher analyses.

本文为「大语言模型具有涌现能力」的声称提供替代解释。对固定任务与固定模型族，研究者可以选择指标来制造涌现能力，也可以选择指标来消融涌现能力。因此，涌现能力可能是研究者选择的产物，而非模型族在特定任务上的基本属性。我们强调：本文任何内容都不应被解读为声称大语言模型不能具有涌现能力；我们的信息是：[3, 8, 28, 33] 中先前声称的涌现能力，很可能是由研究者分析诱导出的幻觉。

Our paper has several implications. Firstly, a task and a metric are distinct and meaningful choices when constructing a benchmark. Secondly, when choosing metric(s), one should consider the metric’s effect on the per-token error rate and adapt their measuring process accordingly, e.g., if one chooses accuracy, one should make sure to have sufficient data to accurately measure accuracy to avoid the risk of drawing invalid scientific conclusions. Thirdly, when making claims about capabilities of large models, including proper controls is critical. In this particular setting, emergent abilities claims are possibly infected by a failure to control for multiple comparisons. In BIG-Bench alone, there are $\geq$ 220 tasks, $\sim 40$ metrics per task, $\sim 10$ model families, for a total of $\sim 10^{6}$ task-metric-model family triplets, meaning probability that no task-metric-model family triplet exhibits an emergent ability by random chance might be small. Fourthly, scientific progress can be hampered when models and their outputs are not made public for independent scientific investigation.

本文有若干含义。第一，构建基准时，任务与指标是彼此独立且有意义的选择。第二，选择指标时，应考虑指标对逐 token 错误率的影响，并相应调整度量过程——例如若选用正确率，应确保有足够数据准确测量正确率，以免得出无效科学结论。第三，对大模型能力下结论时，纳入恰当对照至关重要。在本设定中，涌现能力主张可能受到未能控制多重比较的影响：仅 BIG-Bench 就有 $\geq$ 220 个任务、每任务约 40 个指标、约 10 个模型族，合计约 $10^{6}$ 个任务–指标–模型族三元组，因而没有任何三元组因随机而呈现涌现的概率可能很小。第四，当模型及其输出不向独立科学调查公开时，科学进展可能受阻。

References

[1] Philip W Anderson. More is different: broken symmetry and the nature of the hierarchical structure of science. Science , 177(4047):393–396, 1972.

[2] Glenn W Brier et al. Verification of forecasts expressed in terms of probability. Monthly weather review , 78(1):1–3, 1950.

[3] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877–1901, 2020.

[4] Ethan Caballero, Kshitij Gupta, Irina Rish, and David Krueger. Broken neural scaling laws. arXiv preprint arXiv:2210.14891 , 2022.

[5] Stephanie CY Chan, Adam Santoro, Andrew Kyle Lampinen, Jane X Wang, Aaditya K Singh, Pierre Harvey Richemond, James McClelland, and Felix Hill. Data distributional properties drive emergent in-context learning in transformers. In Advances in Neural Information Processing Systems , 2022.

[6] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 , 2022.

[7] Aidan Clark, Diego De Las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake Hechtman, Trevor Cai, Sebastian Borgeaud, et al. Unified scaling laws for routed language models. In International Conference on Machine Learning , pages 4057–4086. PMLR, 2022.

[8] Deep Ganguli, Danny Hernandez, Liane Lovitt, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Dassarma, Dawn Drain, Nelson Elhage, et al. Predictability and surprise in large generative models. In 2022 ACM Conference on Fairness, Accountability, and Transparency , pages 1747–1764, 2022.

[9] Mitchell A Gordon, Kevin Duh, and Jared Kaplan. Data and parameter scaling laws for neural machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 5915–5922, 2021.

[10] Dan Hendrycks. Detecting emergent behavior. 2022.

[11] Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701 , 2020.

[12] Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. arXiv preprint arXiv:2102.01293 , 2021.

[13] Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Patwary, Mostofa Ali, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409 , 2017.

[14] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 , 2022.

[15] Andy L Jones. Scaling scaling laws with board games. arXiv preprint arXiv:2104.03113 , 2021.

[16] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 , 2020.

[17] Victoria Krakovna, Vikrant Varma, Ramana Kumar, and Mary Phuong. Refining the sharp left turn threat model, part 1: claims and mechanisms. 2022.

[18] Victoria Krakovna, Vikrant Varma, Ramana Kumar, and Mary Phuong. Refining the sharp left turn threat model, part 2: applying alignment techniques. 2022.

[19] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.

[20] Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science , 350(6266):1332–1338, 2015.

[21] Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/ , 1998.

[22] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278–2324, 1998.

[23] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74–81, 2004.

[24] Ryan Lowe and Jan Leike. Aligning language models to follow instructions. 2022.

[25] Eric J. Michaud, Ziming Liu, Uzay Girit, and Max Tegmark. The quantization model of neural scaling, 2023.

[26] Oren Neumann and Claudius Gros. Scaling laws for a multi-agent reinforcement learning model. arXiv preprint arXiv:2210.00849 , 2022.

[27] Jonathan S Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, and Nir Shavit. A constructive prediction of the generalization error across scales. arXiv preprint arXiv:1909.12673 , 2019.

[28] Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615 , 2022.

[29] Jacob Steinhardt. Future ml systems will be qualitatively different. 2022.

[30] Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 , 2022.

[31] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

[32] Jason Wei. 137 emergent abilities of large language models. 2022.

[33] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 , 2022.

[34] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12104–12113, 2022.

Appendix A Approximate Behavior of Metrics on Sequential Data

How do different metrics behave when used to measure autoregressive model outputs? Precisely answering this question is tricky and possibly analytically unsolvable, so we provide an approximate answer here.

用不同指标度量自回归模型输出时，它们如何表现？精确回答既棘手也可能解析上无解，因此我们在此给出近似答案。

Notationally, we consider $N$ test data of length $L$ (here, length is measured in tokens) with targets denoted $t_{n}\defeq(t_{n1},t_{n2},...t_{nL})$, the autoregressive model has a true-but-unknown per-token error probability of $\epsilon\in[0,1]$ and the model outputs prediction $\hat{t}_{n}\defeq(\hat{t}_{n1},\hat{t}_{n2},...\hat{t}_{nL})$. This assumes that the model’s per-token error probability is constant, which is empirically false, but modeling the complex dependencies of errors is beyond our scope.

记号上，考虑长度为 $L$（以 token 计）的 $N$ 条测试数据，目标记为 $t_{n}\defeq(t_{n1},t_{n2},...t_{nL})$，自回归模型具有真实但未知的逐 token 错误概率 $\epsilon\in[0,1]$，并输出预测 $\hat{t}_{n}\defeq(\hat{t}_{n1},\hat{t}_{n2},...\hat{t}_{nL})$。这里假设模型的逐 token 错误概率恒定——经验上并不成立——但刻画错误的复杂依赖超出本文范围。

A.1 Per-Token Error Probability is Resolution-Limited

Note that because we have $N$ test data, each of length $L$, our resolution for viewing the per-token error probability $\epsilon$ is limited by $1/NL$. Here, resolution refers to “the smallest interval measurable by a scientific instrument; the resolving power.” To explain what resolution means via an example, suppose one wants to measure a coin’s probability of yielding heads. After a single coin flip, only two outcomes are possible (H, T), so the resolution-limited probability of heads is either $0$ or $1$. After two coin flips, four outcomes are possible (HH, HT, TH, TT), so the resolution-limited probability of heads is now one of $0,0.5,1$. After $F$ coin flips, we can only resolve the coin’s probability of yielding heads up to $1/F$. Consequently, we introduce a resolution-limited notation:

注意：由于有 $N$ 条测试数据、每条长度 $L$，我们观察逐 token 错误概率 $\epsilon$ 的分辨率受限于 $1/NL$。此处分辨率指「科学仪器可测的最小间隔；分辨本领」。举例说明：若要测量一枚硬币正面朝上的概率，抛一次只有两种结果（正、反），分辨率受限的正面概率只能是 $0$ 或 $1$。抛两次有四种结果，分辨率受限概率为 $0$、$0.5$、$1$ 之一。抛 $F$ 次后，我们只能以 $1/F$ 的粒度分辨正面概率。于是引入分辨率受限记号：

$$\lfloor a\rceil_{b}\defeq\text{$a$ rounded to the nearest integer multiple of $1/b$} \tag{3}$$

A.2 Token Edit Distance

We first consider an adaptation of the Levenshtein (string edit) distance for models that function on tokens rather than characters, an adaptation we term the token edit distance. The token edit distance between two token sequences $t_{n},\hat{t_{n}}$ is defined as the integer number of additions, deletions or substitutions necessary to transform $t_{n}$ into $\hat{t}_{n}$ (or vice versa).

我们首先考虑把 Levenshtein（字符串编辑）距离适配到以 token 而非字符运作的模型，称其为token 编辑距离。两段 token 序列 $t_{n},\hat{t}_{n}$ 之间的 token 编辑距离，定义为把 $t_{n}$ 变成 $\hat{t}_{n}$（或反之）所需的增、删、替换整数次数。

$$\displaystyle\text{Token Edit Distance}(t_{n},\hat{t}_{n}) \\ \displaystyle\defeq\text{Num Substitutions}+\text{Num. Additions}+\text{Num. Deletions} \\ \displaystyle=\sum_{\ell=1}^{L}\mathbb{I}[t_{n\ell}\neq\hat{t}_{n\ell}]+\text{Num. Additions}+\text{Num. Deletions} \\ \displaystyle\geq\sum_{\ell=1}^{L}\mathbb{I}[t_{n\ell}\neq\hat{t}_{n\ell}] \tag{4}$$

The expected token edit distance is therefore:

因此期望 token 编辑距离为：

$$\displaystyle\mathbb{E}[\text{Token Edit Distance}(t_{n},\hat{t}_{n})] \\ \displaystyle\geq\mathbb{E}[\sum_{\ell=1}^{L}\mathbb{I}[t_{n\ell}\neq\hat{t}_{n\ell}]] \\ \displaystyle=\sum_{\ell=1}^{L}p(t_{n\ell}\neq\hat{t}_{n\ell}) \\ \displaystyle\approx L(1-\epsilon) \tag{7}$$

The resolution-limited expected token edit distance is therefore:

于是分辨率受限的期望 token 编辑距离为：

$$\lfloor\mathbb{E}[\text{Token Edit Distance}(t_{n},\hat{t}_{n})]\rceil_{NL}\geq L\Big(1-\lfloor\epsilon\rceil_{NL}\Big) \tag{10}$$

From this, we see that the expected token edit distance scales approximately linearly with the resolution-limited per-token probability. The real rate is slightly higher than linear because additions and deletions contribute an additional non-negative cost, but modeling this requires a model of how likely the model is to overproduce or underproduce tokens, which is something we do not currently possess.

由此可见，期望 token 编辑距离近似随分辨率受限的逐 token 概率线性缩放。真实速率略高于线性，因为增删贡献额外非负代价；但要对这一点建模，需要刻画模型过生成或欠生成 token 的倾向，而这是我们目前不具备的。

A.3 Accuracy

$$\displaystyle\text{Accuracy}(t_{n},\hat{t}_{n}) \\ \displaystyle\defeq\mathbb{I}[\text{No additions}]\,\mathbb{I}[\text{No deletions}]\,\prod_{l=1}^{L}\mathbb{I}[t_{nl}=\hat{t}_{nl}] \\ \displaystyle\approx\prod_{l=1}^{L}\mathbb{I}[t_{nl}=\hat{t}_{nl}] \tag{11}$$

As with the Token Edit Distance (App. A.3), we ignore how likely the language model is to overproduce or underproduce tokens because we do not have a good model of this process. Continuing along,

与 Token Edit Distance（附录 A.3）一样，我们忽略语言模型过生成或欠生成 token 的倾向，因为缺少对该过程的良好模型。继续推导：

$$\displaystyle\mathbb{E}[\log\text{Accuracy}] \\ \displaystyle=\sum_{l}\mathbb{E}[\log\mathbb{I}[t_{nl}=\hat{t}_{nl}]] \\ \displaystyle\leq\sum_{l}\log\mathbb{E}[\mathbb{I}[t_{nl}=\hat{t}_{nl}]] \\ \displaystyle\approx L\log(1-\epsilon) \tag{13}$$

Taking an approximation that would make most mathematicians cry:

再做一个会让多数数学家落泪的近似：

$$\displaystyle\mathbb{E}[\text{Accuracy}] \\ \displaystyle\approx\exp(\mathbb{E}[\log\text{Accuracy}]) \\ \displaystyle=(1-\epsilon)^{L} \tag{16}$$

This reveals that accuracy approximately falls geometrically with target token length. The resolution-limited expected accuracy is therefore:

这表明正确率近似随目标 token 长度几何下降。于是分辨率受限的期望正确率为：

$$\lfloor\mathbb{E}[\text{Accuracy}]\rceil_{NL}=\lfloor(1-\epsilon)^{L}\rceil_{NL} \tag{19}$$

From this we can see that choosing a nonlinear metric like Accuracy is affected significantly more by limited resolution because Accuracy forces one to distinguish quantities that decay rapidly.

由此可见，选用 Accuracy 这类非线性指标会受到有限分辨率的显著更大影响，因为 Accuracy 迫使人们去区分快速衰减的量。

A.4 ROUGE-L-Sum

Figure 9: ROUGE-L-Sum is a sharp metric. Simulations show that as the per-token error probability slightly increase (e.g. from 0.05 to 0.1), the ROUGE-L-Sum metric sharply falls.

图 9：ROUGE-L-Sum 是尖锐指标。 模拟显示：当逐 token 错误概率略增（例如从 0.05 到 0.1）时，ROUGE-L-Sum 指标急剧下降。

Another BIG-Bench metric [28] is ROUGE-L-Sum [23], a metric based on the longest common subsequence (LCS) between two sequences. Section 3.2 of [23] gives the exact definition, but the key property is that ROUGE-L-Sum measures the “union” LCS, which means “stitching” together LCSs across the candidate and multiple references. As explained in the original paper: if the candidate sequence is $c=w_{1}w_{2}w_{3}w_{4}w_{5}$, and if there are two reference sequences $r_{1}=w_{1}w_{2}w_{6}w_{7}w_{8}$ and $r_{2}=w_{1}w_{3}w_{8}w_{9}w_{5}$, then $LCS(r_{1},c)=w_{1}w_{2}$ and $LCS(r_{2},c)=w_{1}w_{3}w_{5}$, then the union -LCS of $c,r_{1},r_{2}$ is $w_{1}w_{2}w_{3}w_{5}$, with length 4. Intuitively, this disproportionately benefits models with smaller error rates because their mistakes can be “stitched” across multiple references; this is confirmed in simulation (Fig. 9).

另一个 BIG-Bench 指标 [28] 是 ROUGE-L-Sum [23]，基于两段序列的最长公共子序列（LCS）。[23] 第 3.2 节给出精确定义，关键性质是：ROUGE-L-Sum 度量「并」LCS，即把候选与多个参考之间的 LCS「缝合」起来。原文解释：若候选为 $c=w_{1}w_{2}w_{3}w_{4}w_{5}$，两个参考为 $r_{1}=w_{1}w_{2}w_{6}w_{7}w_{8}$ 与 $r_{2}=w_{1}w_{3}w_{8}w_{9}w_{5}$，则 $LCS(r_{1},c)=w_{1}w_{2}$、$LCS(r_{2},c)=w_{1}w_{3}w_{5}$，于是 $c,r_{1},r_{2}$ 的并 LCS 为 $w_{1}w_{2}w_{3}w_{5}$，长度 4。直观上，这不成比例地惠及错误率更小的模型，因其错误可跨多个参考被「缝合」；模拟也证实了这一点（图 9）。

Appendix B Inducing Emergent Abilities in Networks on Vision Tasks

B.1 Emergent Classification of MNIST Handwritten Digits by Convolutional Networks

Figure 10: Induced emergent MNIST classification ability in convolutional networks. (A) A published emergent ability from the BIG-Bench Grounded Mappings task [33]. (B) LeNet trained on MNIST [21] displays a predictable, commonplace sigmoidal increase in test accuracy as model parameters increase. (C) When accuracy is redefined as correctly classifying $K$ out of $K$ independent test data, this newly defined metric induces a seemingly unpredictable change.

图 10：在卷积网络中诱导出的涌现 MNIST 分类能力。(A) BIG-Bench Grounded Mappings 任务上已发表的涌现能力 [33]。(B) 在 MNIST [21] 上训练的 LeNet，测试正确率随参数量增加呈可预期、常见的 S 形上升。(C) 当正确率被重定义为在 $K$ 条独立测试数据上全部正确分类时，这一新指标会诱导看似不可预测的变化。

We begin by inducing an emergent classification ability in a LeNet convolutional neural network family [22], trained on the MNIST handwritten digits dataset [21]. This family displays smoothly increasing test accuracy as the number of parameters increase (Fig. 10B). To emulate the accuracy metric used by emergence papers [8, 33, 28], we use subset accuracy: 1 if the network classifies $K$ out of $K$ (independent) test data correctly, 0 otherwise. Under this definition of accuracy, the model family displays an “emergent” ability to correctly classify sets of MNIST digits as $K$ increases from $1$ to $5$, especially when combined with sparse sampling of model sizes (Fig. 10C). This convolutional family’s emergent classification ability qualitatively matches published emergent abilities, e.g., at the BIG-Bench Grounded Mappings task [33] (Fig. 10A).

我们首先在 MNIST 手写数字数据集 [21] 上训练的 LeNet 卷积神经网络族 [22] 中诱导涌现分类能力。该族测试正确率随参数量增加平滑上升（图 10B）。为模拟涌现论文所用的正确率 [8, 33, 28]，我们使用子集正确率：仅当网络对 $K$ 条（独立）测试数据中的全部 $K$ 条都分类正确时记 1，否则记 0。在此定义下，随 $K$ 从 1 增到 5——尤其再叠加对模型规模的稀疏采样——该模型族表现出「涌现」的 MNIST 数字集合正确分类能力（图 10C）。这一卷积族的涌现分类能力在形态上与已发表涌现能力相符，例如 BIG-Bench Grounded Mappings 任务 [33]（图 10A）。