Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method

大语言模型预训练数据检测：一种基于散度的校准方法

Weichao Zhang Ruqing Zhang Jiafeng Guo † † thanks: Corresponding author Affiliation:  CAS Key Lab of Network Data Science and Technology, ICT, CAS, Beijing, China Affiliation:  University of Chinese Academy of Sciences, Beijing, China Affiliation:  Zhongguancun Laboratory, Beijing, China    Maarten de Rijke Yixing Fan Xueqi Cheng Affiliation:  CAS Key Lab of Network Data Science and Technology, ICT, CAS, Beijing, China Affiliation:  University of Chinese Academy of Sciences, Beijing, China Affiliation:  University of Amsterdam, Amsterdam, The Netherlands{zhangweichao22z, zhangruqing, guojiafeng, fanyixing, cxq}@ict.ac.cn, m.derijke@uva.nl

Abstract

摘要

As the scale of training corpora for large language models (LLMs) grows, model developers become increasingly reluctant to disclose details on their data. This lack of transparency poses challenges to scientific evaluation and ethical deployment. Recently, pretraining data detection approaches, which infer whether a given text was part of an LLM’s training data through black-box access, have been explored. The Min-K% Prob method, which has achieved state-of-the-art results, assumes that a non-training example tends to contain a few outlier words with low token probabilities. However, the effectiveness may be limited as it tends to misclassify non-training texts that contain many common words with high probabilities predicted by LLMs. To address this issue, we introduce a divergence-based calibration method, inspired by the divergence-from-randomness concept, to calibrate token probabilities for pretraining data detection. We compute the cross-entropy (i.e., the divergence) between the token probability distribution and the token frequency distribution to derive a detection score. We have developed a Chinese-language benchmark, PatentMIA, to assess the performance of detection approaches for LLMs on Chinese text. Experimental results on English-language benchmarks and PatentMIA demonstrate that our proposed method significantly outperforms existing methods. Our code and PatentMIA benchmark are available at https://github.com/zhang-wei-chao/DC-PDD.

随着大语言模型（LLM）训练语料规模不断扩大，模型开发者越来越不愿公开数据细节。这种不透明给科学评估与合乎伦理的部署带来挑战。近来有人探索预训练数据检测方法：在黑盒访问条件下，推断给定文本是否属于某 LLM 的训练数据。当前表现最好的 Min-K% Prob 假设：非训练样本往往含有少量低 token 概率的离群词。但其效果有限——它容易把「包含许多常见词、因而 LLM 给出高概率」的非训练文本误判为训练文本。为解决这一问题，我们引入受「相对随机性的散度」（divergence-from-randomness）启发的、基于散度的校准方法，用于校准预训练数据检测中的 token 概率。我们计算 token 概率分布与 token 频率分布之间的交叉熵（即散度），据此得到检测分数。我们还构建了中文基准 PatentMIA，以评估检测法在中文文本上的表现。英文基准与 PatentMIA 上的实验表明，所提方法显著优于现有方法。代码与 PatentMIA 见 https://github.com/zhang-wei-chao/DC-PDD。

1 Introduction

1 引言

A critical element contributing to the effectiveness of large language models (LLMs) is the large volume of data used for pretraining. In many cases, model developers are reluctant to disclose information about their training corpus Achiam et al. (2023); Touvron et al. (2023b); Brown et al. (2020); Yang et al. (2023); Bai et al. (2023). This lack of transparency complicates the assurance that all ethical and legal standards are met. The pretraining corpus may contain unauthorized private information or copyrighted content Mozes et al. (2023); Chang et al. (2023). Indeed, OpenAI and NVIDIA face lawsuits over copyright issues related to their training data Grynbaum and Mac (2023); Stempel (2024). Moreover, a lack of transparency around the pretraining data used prevents us from properly addressing the data contamination problem Dong et al. (2024); Cao et al. (2024) and, hence, from determining whether an LLM’s performance is due to genuine task understanding or to prior exposure to test data. We focus on the following key question: How can we detect if a black-box LLM was pretrained on a given text, considering that its training data is undisclosed?

促成大语言模型（LLM）有效性的关键要素之一，是用于预训练的海量数据。许多情况下，模型开发者不愿披露训练语料信息 Achiam et al. (2023); Touvron et al. (2023b); Brown et al. (2020); Yang et al. (2023); Bai et al. (2023)。这种不透明使人们难以确认是否满足全部伦理与法律标准。预训练语料可能含未经授权的隐私信息或受版权保护内容 Mozes et al. (2023); Chang et al. (2023)。事实上，OpenAI 与 NVIDIA 已因训练数据相关版权问题面临诉讼 Grynbaum and Mac (2023); Stempel (2024)。此外，预训练数据不透明也阻碍我们妥善处理数据污染问题 Dong et al. (2024); Cao et al. (2024)，从而难以判断 LLM 的表现究竟来自真正的任务理解，还是事先见过测试数据。我们聚焦如下关键问题：在训练数据未公开的前提下，如何检测一个黑盒 LLM 是否在给定文本上做过预训练？

Figure 1: A conceptual example: Let $x^{1}$ represent a non-training text and $x^{2}$ a training text. (a) Min-K% Prob directly selects the $k$% of tokens with the lowest probabilities for detection. (b) DC-PDD computes the divergence between the token probability distribution and the token frequency distribution for detection.

图 1：概念示例：设 $x^{1}$ 为非训练文本、$x^{2}$ 为训练文本。(a) Min-K% Prob 直接选取概率最低的 $k$% token 做检测。(b) DC-PDD 计算 token 概率分布与 token 频率分布之间的散度用于检测。

The pretraining data detection problem can be viewed as an instance of the membership inference attack (MIA) task Shokri et al. (2017), where the primary objective is to determine if a particular text was part of a target LLM’s training corpus. Prevailing methods to tackle this problem are based on the idea that a text’s token probability distribution can reveal its inclusion in the training set. E.g., the Min-K% Prob method Shi et al. (2024) is based on the hypothesis that non-training examples tend to have more tokens assigned lower probabilities than training examples do. Min-K% Prob relies on the assumption that data with higher probability is more likely to be training data. Language models trained with a cross-entropy loss function tend to favor high-frequency tokens when conducting next-token prediction, which will also lead to LLMs generally predicting higher probabilities for high-frequency tokens Jiang et al. (2019). As the conceptual example shown in figure 1, $x^{1}$ is a non-training text and $x^{2}$ is a training text. We can see that the lowest raw token probabilities for $x^{1}$ are higher than those for $x^{2}$, which may be because the words in $x^{1}$ (e.g., "boys", "great") are generally more common than the words in $x^{2}$ (e.g., "erudite", "conundrum"). Therefore, Min-k% Prob will calculate a detection score of -0.88 for $x^{1}$ and -2.94 for $x^{2}$, which means that $x^{1}$ is more likely to be considered a training text than $x^{2}$. This is contrary to the actual situation.

预训练数据检测可视为成员推断攻击（MIA）任务的一种实例 Shokri et al. (2017)，目标是判断某段文本是否属于目标 LLM 的训练语料。主流做法基于如下直觉：文本的 token 概率分布可以揭示其是否进入训练集。例如 Min-K% Prob 方法 Shi et al. (2024) 假设：相对训练样本，非训练样本往往有更多被赋予更低概率的 token。Min-K% Prob 依赖「概率更高更像训练数据」这一假定。用交叉熵损失训练的语言模型在下一 token 预测时偏向高频 token，也会导致 LLM 对高频 token 普遍给出更高概率 Jiang et al. (2019)。如图 1 概念示例，$x^{1}$ 是非训练文本、$x^{2}$ 是训练文本。可见 $x^{1}$ 最低的若干原始 token 概率反而高于 $x^{2}$，可能因为 $x^{1}$ 中的词（如 "boys"、"great"）总体上比 $x^{2}$ 中的词（如 "erudite"、"conundrum"）更常见。于是 Min-K% Prob 会给 $x^{1}$ 算得分 -0.88、给 $x^{2}$ 算 -2.94，意味着 $x^{1}$ 比 $x^{2}$ 更像训练文本——这与实际情况相反。

Inspired by the divergence-from-randomness theory Amati and van Rijsbergen (2002), we introduce a divergence-based calibration method, named DC-PDD, to calibrate the token probabilities for pretraining data detection. The basic idea underlying divergence-from-randomness is that the higher the divergence of the within-document term-frequency of a word in a document from its frequency within the collection, the more information the word carries. In our scenario, the within-document term-frequency can be interpreted as the target LLM’s predicted probability for each token with regard to the text to be detected, to which we refer as the token probability distribution. The frequency of a word within the collection refers to the frequency of each token in the target LLM’s pretraining corpus, to which we refer as the token frequency distribution. According to the divergence-from-randomness theory, the higher the divergence between these two distributions, the more informative the tokens are in indicating that the text was part of the model’s training corpus, rather than solely relying on token probabilities as the indicator for detection.

受 divergence-from-randomness 理论 Amati and van Rijsbergen (2002) 启发，我们提出名为 DC-PDD 的、基于散度的校准方法，用于校准预训练数据检测中的 token 概率。该理论的基本想法是：文档内词频相对集合内词频的散度越高，该词携带的信息量越大。在我们的场景中，文档内词频可解释为目标 LLM 对拟检测文本中各 token 的预测概率，称为 token 概率分布；集合内词频则指各 token 在目标 LLM 预训练语料中的频率，称为 token 频率分布。按 divergence-from-randomness，两分布散度越高，这些 token 越能指示文本曾进入训练语料，而不是仅凭 token 概率做检测。

Like prior works Shi et al. (2024); Duan et al. (2024), we assume that we only have access to the target LLM as a black box: we can compute token probabilities for the text to be detected but have no access to the internals of the LLM (e.g., weights and activations). We first obtain the token probability distribution by querying the LLM with the text. Next, we use a large-scale publicly available corpus as a reference corpus to obtain an estimation of the token frequency distribution since an LLM’s pretraining corpus is not accessible usually. We then calibrate the token probabilities by comparing the token probability distribution to the token frequency distribution. Based on the calibrated token probabilities, we derive a score for pretraining data detection. Finally, a predefined threshold is applied to the score to determine whether the text was included in the LLM’s pretraining corpus.

与先前工作 Shi et al. (2024); Duan et al. (2024) 一样，我们假定只能黑盒访问目标 LLM：可计算拟检测文本的 token 概率，但无法获得内部（如权重与激活）。我们先用文本查询 LLM 得到 token 概率分布；再用大规模公开语料作为参照语料，估计 token 频率分布（通常无法访问真实预训练语料）；接着通过比较两分布校准各 token 概率，并据此得到检测分数；最后用预设阈值判断文本是否属于预训练语料。

Figure 1(b) illustrates that DC-PDD assigns a score to text that better reflects whether it is training data or non-training data (i.e., a training text should have a higher score than a non-training text). In contrast to other calibration methods Carlini et al. (2021); Zhang et al. (2024), DC-PDD neither requires additional reference models nor extra access requirements on the target LLM.

图 1(b) 表明，DC-PDD 给文本的打分更能反映其是否为训练数据（训练文本分数应高于非训练文本）。相对其他校准方法 Carlini et al. (2021); Zhang et al. (2024)，DC-PDD 既不需要额外参照模型，也不对目标 LLM 提出额外访问要求。

Benchmark	Data source	Language	Text length	#Examples	Applicable models
WikiMIA Shi et al. (2024)	Wikipedia	English	32	776	Open-source LLMs released between 2017 and 2023
64	542
128	250
256	82
BookMIA Shi et al. (2024)	Books	English	512	9,870	OpenAI models released before 2023	OpenAI models released before 2023
OpenAI models released before 2023
PatentMIA (Ours)	GooglePatent	Chinese	512	10,000	Open-source Chinese LLMs released between January 1, 2023 and March 1, 2024	Open-source Chinese LLMs released between	January 1, 2023 and March 1, 2024
Open-source Chinese LLMs released between
January 1, 2023 and March 1, 2024

OpenAI models released before 2023

Open-source Chinese LLMs released between
January 1, 2023 and March 1, 2024

Table 1: Benchmark summary statistics: Each benchmark has an equal split of training and non-training examples. “Text Length” refers to the number of words contained in each text example of the benchmark. “#Examples” denotes the number of text examples in the benchmark.

基准	数据来源	语言	文本长度	#样例	适用模型
WikiMIA Shi et al. (2024)	Wikipedia	英文	32	776	2017–2023 发布的开源 LLM
			64	542
			128	250
			256	82
BookMIA Shi et al. (2024)	书籍	英文	512	9,870	2023 年前发布的 OpenAI 模型
PatentMIA（本文）	GooglePatent	中文	512	10,000	2023-01-01 至 2024-03-01 发布的开源中文 LLM

表 1：基准统计概览：各基准训练/非训练样例各半。「文本长度」指每条样例的词数。「#样例」为基准中的文本条数。

To facilitate this study and the evaluation of pretraining data detection for LLMs, we introduce a new benchmark named PatentMIA, specifically designed for Chinese-language pretraining data detection. PatentMIA is sourced from Google-Patents Google (2006) and constructed following Shi et al. (2024), who distinguish between training and non-training data based on cut-off dates of the target LLM, where training data precedes, and non-training data follows, the cut-off date.

为推动 LLM 预训练数据检测研究与评测，我们提出专为中文设计的新基准 PatentMIA。数据来自 Google-Patents Google (2006)，构建方式遵循 Shi et al. (2024)：按目标 LLM 的知识截止日期区分训练与非训练数据——截止日期之前为训练、之后为非训练。

We conduct experiments on two English-language benchmarks Shi et al. (2024) and on PatentMIA against a range of representative, state-of-the-art methods. Our experiments show that the proposed DC-PDD significantly outperforms prior methods. E.g., in the commonly used detection performance metrics, AUC and TPR@5%FPR, DC-PDD surpasses Min-K% Prob by $8.6$% and $13.3$%, respectively, on existing BookMIA benchmark.

我们在两个英文基准 Shi et al. (2024) 与 PatentMIA 上，对一系列代表性、当时最优方法做实验。结果表明所提 DC-PDD 显著优于先前方法。例如在常用指标 AUC 与 TPR@5%FPR 上，DC-PDD 在既有 BookMIA 基准上相对 Min-K% Prob 分别高出 $8.6$% 与 $13.3$%。

2 Problem Statement

2 问题陈述

2.1 Task Description

2.1 任务描述

Formally, given a piece of text $x$ and an LLM $\mathcal{M}$ with no knowledge of its pretraining corpus $\mathcal{D}$, the pretraining data detection task aims to design a method to determine if $x$ was included in $\mathcal{D}$. Thus, given $x$ and $\mathcal{M}$ as input, a method $\mathcal{A}$ for the pretraining data detection task returns $1$ if it predicts that $x$ is included in $\mathcal{D}$ and $0$ if it is not:

形式化地，给定一段文本 $x$ 以及对其预训练语料 $\mathcal{D}$ 一无所知的 LLM $\mathcal{M}$，预训练数据检测任务旨在设计方法判断 $x$ 是否属于 $\mathcal{D}$。因此，以 $x$ 与 $\mathcal{M}$ 为输入，检测方法 $\mathcal{A}$ 若预测 $x\in\mathcal{D}$ 则返回 $1$，否则返回 $0$：

$$\mathcal{A}(x,\mathcal{M})\rightarrow\{0,1\}. \tag{1}$$

Black-box setting. Like prior works Shi et al. (2024); Duan et al. (2024), we assume that we have access to $\mathcal{M}$ as a black-box, which means that we can compute token probabilities for $x$. The internals of the model, such as the weights and activations, are not available.

黑盒设定。 与先前工作 Shi et al. (2024); Duan et al. (2024) 一样，我们假定只能黑盒访问 $\mathcal{M}$，即可计算 $x$ 的 token 概率；模型内部（如权重与激活）不可得。

2.2 Benchmark Construction

2.2 基准构建

Unlike traditional membership inference attacks Yeom et al. (2018); Jagannatha et al. (2021); Carlini et al. (2022), which are conducted on locally trained models where the training and non-training data are explicitly known, the pretraining data detection for LLMs poses a new challenge as the pretraining corpus of LLMs is not disclosed. Here, we introduce existing benchmarks and our newly constructed benchmark that are specifically designed for LLMs. Table 1 shows their overall statistics.

与在本地训练模型上、训练/非训练数据明确可知的传统成员推断攻击 Yeom et al. (2018); Jagannatha et al. (2021); Carlini et al. (2022) 不同，面向 LLM 的预训练数据检测面临新挑战：预训练语料未公开。这里介绍既有基准以及我们新构建的基准。表 1 给出总体统计。

Pre-existing datasets. Shi et al. (2024) proposed a benchmark construction method by distinguishing between the training and non-training data based on the knowledge cut-off date of the target LLM, where training data precedes and non-training data follows the cut-off date. This method has been used to construct two English-language benchmarks: WikiMIA and BookMIA. In this paper, we conduct experiments on these benchmarks.

既有数据集。 Shi et al. (2024) 提出按目标 LLM 的知识截止日期区分训练与非训练数据的基准构建法，并据此构建了两个英文基准：WikiMIA 与 BookMIA。本文在这些基准上做实验。

A Chinese-language benchmark: PatentMIA. Existing benchmarks for the pretraining data detection task are exclusively in English. Other languages exhibit unique grammatical characteristics such as flexible spacing and case insensitivity compared to English, potentially influencing the effectiveness of methods for the detection task. These differences warrant specific benchmarks to assess the performance of detection methods in languages other than English. We propose a Chinese-language benchmark for that reason. Next, we detail the construction of the PatentMIA benchmark.

中文基准：PatentMIA。 现有预训练数据检测基准均为英文。其他语言相对英语有灵活分词、大小写不敏感等语法特点，可能影响检测方法效果，因而需要专门基准评估非英语场景。我们据此提出中文基准，下详 PatentMIA 的构建。

Data source. We collect data from Google-Patents Google (2006) as (i) it contains a large volume of high-quality, publicly available Chinese patent texts and some publicly available large-scale Chinese corpora like ChineseWebText Chen et al. (2023) explicitly incorporate data from this website, which indicates that existing LLMs are highly likely to have used such data for pretraining; and (ii) if the priority date of a patent is after the release date of the LLM, there is a guarantee that the patent text was not present during LLM’s pretraining.

数据来源。 我们从 Google-Patents Google (2006) 采集数据，原因是：(i) 含大量高质量、公开可得的中文专利文本，且 ChineseWebText Chen et al. (2023) 等公开大规模中文语料明确纳入该站数据，说明现有 LLM 极可能用其做预训练；(ii) 若专利优先权日晚于 LLM 发布日，可保证该专利文本不在预训练阶段出现。

Data collection. Based on Google-Patents, we construct a Chinese-language benchmark called PatentMIA as follows. (i) *Data crawling.*We randomly crawl 5,000 Chinese patent pages with a priority date after March 1, 2024 and 5,000 patent pages with a publication date before January 1, 2023 respectively. (ii) *Data preprocessing.*These pages then undergo several preprocessing and cleaning steps similar to those used in ChineseWebText to ensure the data format matches the pretraining data format of LLMs. (iii) *Snippet extraction.*For each page, we randomly extract a snippet of 512 words from the original content, creating a balanced set of 10,000 examples. We use jieba[^1] to segment Chinese texts into words.

数据采集。 基于 Google-Patents，我们按如下步骤构建中文基准 PatentMIA。(i) *爬取。*随机爬取优先权日 晚于 2024 年 3 月 1 日 的中文专利页 5,000 篇，以及公开日 早于 2023 年 1 月 1 日 的专利页 5,000 篇。(ii) *预处理。*参照 ChineseWebText 做清洗，使格式贴近 LLM 预训练数据。(iii) *片段抽取。*每页随机抽取 512 词片段，得到均衡的 10,000 条样例。中文分词使用 jieba[^1]。

3 Method

3 方法

3.1 Overview

3.1 概览

Given a piece of text $x=x_{1}x_{2}\ldots x_{n}$, where $x_{i}$ represent the tokens after tokenizing $x$, and a target LLM $\mathcal{M}$, we compute a detection score by measuring the divergence between the token probability distribution of $x$ and the token frequency distribution in pretraining corpus, without any model training processes. Our method consists of four steps: (i) Token probability distribution computation, by querying $\mathcal{M}$ with $x$ (Section 3.2). (ii) Token frequency distribution computation, by using a large-scale publicly available corpus $\mathcal{D^{\prime}}$ as a reference corpus to obtain an estimation of the token frequency distribution since $\mathcal{M}$’s pretraining corpus is not assumed to be accessible (Section 3.3). (iii) Score calculation via comparison, by comparing the above two distributions to calibrate the token probability for each token $x_{i}$ in $x$, and derive a score for pretraining data detection based on the calibrated token probabilities (Section 3.4). (iv) binary decision, by applying a predefined threshold to the score, we predict whether $x$ was included in $\mathcal{M}$’s pretraining corpus or not (Section 3.5).

给定文本 $x=x_{1}x_{2}\ldots x_{n}$（$x_{i}$ 为分词后的 token）与目标 LLM $\mathcal{M}$，我们通过度量 $x$ 的 token 概率分布与预训练语料中 token 频率分布之间的散度计算检测分数，无需任何模型训练。方法分四步：(i) 计算 token 概率分布：用 $x$ 查询 $\mathcal{M}$（第 3.2 节）。(ii) 计算 token 频率分布：用大规模公开语料 $\mathcal{D^{\prime}}$ 作参照估计频率分布（假定 $\mathcal{M}$ 的预训练语料不可得）（第 3.3 节）。(iii) 比较得分：比较两分布校准各 token $x_{i}$ 的概率，再由校准后概率得到检测分数（第 3.4 节）。(iv) 二值决策：对分数施加预设阈值，预测 $x$ 是否属于 $\mathcal{M}$ 的预训练语料（第 3.5 节）。

We summarize our method in Algorithm 1.

算法 1 总结了我们的方法。

0:  A text to be detected $x=x_{1}x_{2}\ldots x_{n}$, a target LLM $\mathcal{M}$, vocabulary of LLM $V=\{x_{i}\}_{i=1}^{\lvert V\rvert}$, reference corpus $\mathcal{D^{\prime}}$, decision threshold $\tau$
1:  Prepend a start-of-sentence token to $x$
2:  for $i=1$ to $n$ do
3:   Access the token probability $p(x_{i};\mathcal{M})$ from $\mathcal{M}$, w.r.t. Eq. (3)
4:  end for
5:  for $i=1$ to $\lvert V\rvert$ do
6:   Compute the token frequency $p(x_{i};\mathcal{D^{\prime}})$ based on $\mathcal{D^{\prime}}$, w.r.t. Eq. (5)
7:  end for
8:  for $i=1$ to $n$ do
9:   Compute $\alpha_{i}$ for $x_{i}$ based on $p(x_{i};\mathcal{M})$ and $p(x_{i};\mathcal{D^{\prime}})$, w.r.t. Eq. (6), (7)
10:  end for
11:  Select $\alpha_{i}$ corresponding to tokens with the first occurrence in $x$ to compute a score $\beta$, w.r.t. Eq. (8)
12:  if $\beta\geq\tau$ then
13:   1: $\mathcal{M}$ was pretrained on $x$
14:  else
15:   0: $\mathcal{M}$ was not pretrained on $x$
16:  end if

Algorithm 1 Our DC-PDD

0:  待检测文本 $x=x_{1}x_{2}\ldots x_{n}$，目标 LLM $\mathcal{M}$，词表 $V=\{x_{i}\}_{i=1}^{\lvert V\rvert}$，参照语料 $\mathcal{D^{\prime}}$，决策阈值 $\tau$
1:  在 $x$ 前拼接句首 token
2:  for $i=1$ to $n$ do
3:   从 $\mathcal{M}$ 取 token 概率 $p(x_{i};\mathcal{M})$，见式 (3)
4:  end for
5:  for $i=1$ to $\lvert V\rvert$ do
6:   基于 $\mathcal{D^{\prime}}$ 计算 token 频率 $p(x_{i};\mathcal{D^{\prime}})$，见式 (5)
7:  end for
8:  for $i=1$ to $n$ do
9:   由 $p(x_{i};\mathcal{M})$ 与 $p(x_{i};\mathcal{D^{\prime}})$ 计算 $\alpha_{i}$，见式 (6)、(7)
10:  end for
11:  选取 $x$ 中首次出现的 token 对应的 $\alpha_{i}$ 计算分数 $\beta$，见式 (8)
12:  if $\beta\geq\tau$ then
13:   1：$\mathcal{M}$ 在 $x$ 上做过预训练
14:  else
15:   0：$\mathcal{M}$ 未在 $x$ 上预训练
16:  end if

算法 1 我们的 DC-PDD

3.2 Token Probability Distribution Computation

3.2 计算 Token 概率分布

To obtain all the probabilities of $x_{i}$ in $x$ from $\mathcal{M}$, we first prepend a start-of-sentence token, denoted as $x_{0}$, to $x$, since the model does not return a prediction for the first token:

为从 $\mathcal{M}$ 得到 $x$ 中全部 $x_{i}$ 的概率，先在 $x$ 前拼接句首 token（记为 $x_{0}$），因为模型不对首 token 返回预测：

$$x^{\prime}=x_{0}x_{1}x_{2}\ldots x_{n}. \tag{2}$$

Subsequently, we feed $x^{\prime}$ into $\mathcal{M}$, resulting in a sequence of predicted probabilities corresponding to the true tokens:

随后将 $x^{\prime}$ 送入 $\mathcal{M}$，得到与真实 token 对应的预测概率序列：

$$\{p(x_{i}\mid x_{<i};\mathcal{M}):0<i\leq n\}. \tag{3}$$

Note that the probability of each token $x_{i}$ is predicted by $\mathcal{M}$ based on the preceding context $x_{<i}$ for $0<i\leq n$. For brevity in subsequent expressions, we simplify $p(x_{i}\mid x_{<i};\mathcal{M})$ to $p(x_{i};\mathcal{M})$.

注意每个 token $x_{i}$ 的概率由 $\mathcal{M}$ 基于前缀上下文 $x_{<i}$（$0<i\leq n$）预测。后文为简洁，将 $p(x_{i}\mid x_{<i};\mathcal{M})$ 简写为 $p(x_{i};\mathcal{M})$。

3.3 Token Frequency Distribution Computation

3.3 计算 Token 频率分布

According to the divergence-from-randomness theory, after obtaining the token probability distribution for $x$ from $\mathcal{M}$, we also need to calculate the frequency of $x_{i}$ appearing in the pretraining corpus $\mathcal{D}$ of $\mathcal{M}$ to get the token frequency distribution. However, since $\mathcal{D}$ is not accessible, we cannot directly calculate these terms. To address this, we use a large-scale publicly available corpus $\mathcal{D^{\prime}}$ to obtain an estimation of these terms:

按 divergence-from-randomness，在从 $\mathcal{M}$ 得到 $x$ 的 token 概率分布后，还需要 $x_{i}$ 在 $\mathcal{M}$ 预训练语料 $\mathcal{D}$ 中的频率以得到 token 频率分布。但 $\mathcal{D}$ 不可得，无法直接计算。为此我们用大规模公开语料 $\mathcal{D^{\prime}}$ 估计这些量：

$$p(x_{i};\mathcal{D^{\prime}})=\frac{\operatorname{count}(x_{i})}{N^{\prime}}, \tag{4}$$

where $\operatorname{count}(x_{i})$ is the number of occurrences of $x_{i}$ in $\mathcal{D^{\prime}}$, and $N^{\prime}$ is the total number of tokens in $\mathcal{D^{\prime}}$. We employ Laplace smoothing to address the zero probability problem when $x_{i}$ does not occur in $\mathcal{D^{\prime}}$ even once:

其中 $\operatorname{count}(x_{i})$ 为 $x_{i}$ 在 $\mathcal{D^{\prime}}$ 中的出现次数，$N^{\prime}$ 为 $\mathcal{D^{\prime}}$ 总 token 数。当 $x_{i}$ 一次也未出现时，用 Laplace 平滑处理零概率：

$$p(x_{i};\mathcal{D^{\prime}})=\frac{\operatorname{count}(x_{i})+1}{N^{\prime}+\lvert V\rvert}, \tag{5}$$

where $\lvert V\rvert$ represents the vocabulary size of $\mathcal{M}$, i.e., the number of categories of tokens.

其中 $\lvert V\rvert$ 为 $\mathcal{M}$ 的词表大小，即 token 类别数。

3.4 Score Calculation through Comparison

3.4 通过比较计算分数

We compute the cross-entropy (i.e., the divergence) between the token probability distribution $p(x_{i};\mathcal{M})$ and the token frequency distribution $p(x_{i};\mathcal{D}^{\prime})$ to obtain a score $\alpha_{i}$ for each token $x_{i}$:

我们计算 token 概率分布 $p(x_{i};\mathcal{M})$ 与 token 频率分布 $p(x_{i};\mathcal{D}^{\prime})$ 之间的交叉熵（即散度），得到每个 token $x_{i}$ 的分数 $\alpha_{i}$：

$$\alpha_{i}=-p(x_{i};\mathcal{M})\cdot\log p(x_{i};\mathcal{D}^{\prime}). \tag{6}$$

We set a hyperparameter $a$ to control the upper bound of $\alpha_{i}$, preventing the final score from being dominated by a few tokens:

设置超参数 $a$ 控制 $\alpha_{i}$ 上界，避免最终分数被少数 token 主导：

$$\alpha_{i}=\begin{cases}\alpha_{i},&\text{if }\alpha_{i}<a\
a,&\text{if }\alpha_{i}\geq a.\end{cases} \tag{7}$$

Typically, for a word that appears multiple times in a text, LLMs predict a higher probability for that word in subsequent occurrences since the model has seen the word earlier in the text. Therefore, we adopt a simple countermeasure that only uses $\alpha_{i}$ corresponding to the first occurrence of $x_{i}$ in $x$ to calculate the final score $\beta$:

通常，同一词在文本中多次出现时，LLM 会在后续出现处给出更高概率，因为模型已在前文见过该词。我们采取简单对策：只用 $x$ 中 $x_{i}$ 首次出现对应的 $\alpha_{i}$ 计算最终分数 $\beta$：

$$\beta=\frac{1}{\lvert\operatorname{FOS}(x)\rvert}\sum_{x_{j}\in\operatorname{FOS}(x)}{\alpha_{j}}, \tag{8}$$

where $\operatorname{FOS}(x)$ denotes the set of tokens with the first occurrence in $x$.

其中 $\operatorname{FOS}(x)$ 表示 $x$ 中首次出现的 token 集合。

3.5 Binary Decision

3.5 二值决策

After calculating the score $\beta$ for $x$ following the aforementioned three steps, we predict whether $x$ was included in $\mathcal{M}$’s pretraining corpus $\mathcal{D}$ by applying a predefined threshold $\tau$ to $\beta$:

按上述三步算得 $x$ 的分数 $\beta$ 后，对 $\beta$ 施加预设阈值 $\tau$，预测 $x$ 是否属于 $\mathcal{M}$ 的预训练语料 $\mathcal{D}$：

$$\operatorname{Decision}(x,\mathcal{M})=\begin{cases}0~(x\notin\mathcal{D}),&\text{if }\beta<\tau\
1~(x\in\mathcal{D}),&\text{if }\beta\geq\tau.\end{cases} \tag{9}$$

If $\beta$ is not less than $\tau$, we predict that $x$ was included in $\mathcal{D}$; otherwise, it was not.

若 $\beta$ 不小于 $\tau$，则预测 $x\in\mathcal{D}$；否则预测不属于。

4 Experimental Settings

4 实验设置

Benchmarks and models. To evaluate the performance of DC-PDD, we conduct experiments on three benchmarks mentioned in Table 1. Specifically, for WikiMIA, we consider OPT-6.7B Zhang et al. (2022), Pythia-6.9B Biderman et al. (2023), Llama-13B Touvron et al. (2023a), and GPT-NeoX-20B Black et al. (2022), since they were released after 2017 and before 2023, and are well-known for incorporating Wikipedia dumps into their pretraining data. For BookMIA, we consider GPT-3,[^2] since it’s an OpenAI model released before 2023. These settings are akin to Shi et al. (2024). For our benchmark PatentMIA, we select Baichuan-13B Yang et al. (2023) and Qwen1.5-14B Team (2024), since they are representative models in Chinese text generation and are released between January 1, 2023 and March 1, 2024.

基准与模型。 为评估 DC-PDD，我们在表 1 所列三个基准上实验。具体地，WikiMIA 上考虑 OPT-6.7B Zhang et al. (2022)、Pythia-6.9B Biderman et al. (2023)、Llama-13B Touvron et al. (2023a) 与 GPT-NeoX-20B Black et al. (2022)，因其发布于 2017–2023 之间，且以纳入 Wikipedia dump 著称。BookMIA 上考虑 GPT-3，[^2]因其为 2023 年前发布的 OpenAI 模型。设定与 Shi et al. (2024) 相近。对我们的 PatentMIA，选取 Baichuan-13B Yang et al. (2023) 与 Qwen1.5-14B Team (2024)，二者为中文生成代表性模型，且发布于 2023-01-01 至 2024-03-01。

Baselines. We consider the following methods as our baselines, each predicting whether an example was included in training set based on: (i) PPL: The perplexity of the example. (ii) Lowercase: The ratio of the example’s perplexity to that of the lowercased example. (iii) Zlib: The ratio of the example’s perplexity against its zlib entropy. (iv) Small Ref: The ratio of an example’s perplexity to the example’s perplexity under a smaller model pretrained on the same data. (v) Min-K% ProbShi et al. (2024): The average log-likelihood of the $k$% of tokens with the lowest probabilities. (vi) Min-K%++ ProbZhang et al. (2024): The average normalized log-likelihood of the k% of tokens with the lowest normalized probabilities, where the normalization is based on the statistics of the categorical distribution over the entire vocabulary. Note that the first four baselines were introduced in Carlini et al. (2021). For more details on our baselines, please refer to Appendix A.1.

基线。 我们考虑以下方法，各自依据不同信号预测样例是否属于训练集：(i) PPL：样例困惑度。(ii) Lowercase：样例困惑度相对其小写版本困惑度的比值。(iii) Zlib：样例困惑度相对其 zlib 熵的比值。(iv) Small Ref：样例在目标模型上的困惑度相对同数据上更小模型困惑度的比值。(v) Min-K% ProbShi et al. (2024)：概率最低的 $k$% token 的平均对数似然。(vi) Min-K%++ ProbZhang et al. (2024)：最低的 $k$% 归一化概率 token 的平均归一化对数似然，归一化基于全词表类别分布统计。前四种基线由 Carlini et al. (2021) 提出。更多细节见附录 A.1。

Evaluation metrics. Following most existing works Shi et al. (2024); Duan et al. (2024); Zhang et al. (2024), we use AUC score (area under ROC curve) and TPR (true positive rate) at a low FPR (false positive rate) (TPR@5%FPR) as our metrics. For more details on these metrics, please refer to Appendix A.2.

评测指标。 遵循多数既有工作 Shi et al. (2024); Duan et al. (2024); Zhang et al. (2024)，我们使用 AUC（ROC 曲线下面积）以及低 FPR 下的 TPR（TPR@5%FPR）。细节见附录 A.2。

Method	BookMIA	PatentMIA	WikiMIA
GPT-3	Baichuan-13B	Qwen1.5-14B	OPT-6.7B	Pythia-6.9B	Llama-13B	GPT-NeoX-20B
PPL	0.635	0.608	0.599	0.625	0.651	0.678	0.707
Lowercase	0.671	-	-	0.587	0.605	0.606	0.680
Zlib	0.537	0.634	0.618	0.644	0.676	0.697	0.723
Small Ref	-	0.657	0.565	0.654	0.660	0.658	0.714
Min-K% Prob	0.639	0.643	0.637	0.674	0.695	0.715	0.756
Min-K%++ Prob	-	0.625	0.630	0.692	0.697	0.838	0.754
DC-PDD	0.725*	0.699*	0.689*	0.677	0.698	0.697	0.766*

Table 2: AUC scores for detecting pretraining texts. Bold indicates the best performing method. Two-tailed t-tests show that DC-PDD significantly improves over Min-K% Prob ( * indicates $p\leq 0.05$).

方法	BookMIA	PatentMIA	WikiMIA
	GPT-3	Baichuan-13B	Qwen1.5-14B	OPT-6.7B	Pythia-6.9B	Llama-13B	GPT-NeoX-20B
PPL	0.635	0.608	0.599	0.625	0.651	0.678	0.707
Lowercase	0.671	-	-	0.587	0.605	0.606	0.680
Zlib	0.537	0.634	0.618	0.644	0.676	0.697	0.723
Small Ref	-	0.657	0.565	0.654	0.660	0.658	0.714
Min-K% Prob	0.639	0.643	0.637	0.674	0.695	0.715	0.756
Min-K%++ Prob	-	0.625	0.630	0.692	0.697	0.838	0.754
DC-PDD	0.725*	0.699*	0.689*	0.677	0.698	0.697	0.766*

表 2：检测预训练文本的 AUC。加粗为最优。双侧 t 检验显示 DC-PDD 相对 Min-K% Prob 显著提升（* 表示 $p\leq 0.05$）。

Method	BookMIA	PatentMIA	WikiMIA
GPT-3	Baichuan-13B	Qwen1.5-14B	OPT-6.7B	Pythia-6.9B	Llama-13B	GPT-NeoX-20B
PPL	0.224	0.166	0.159	0.130	0.144	0.216	0.180
Lowercase	0.240	-	-	0.094	0.130	0.158	0.130
Zlib	0.192	0.159	0.129	0.180	0.209	0.187	0.223
Small Ref	-	0.211	0.078	0.122	0.158	0.151	0.187
Min-K% Prob	0.203	0.170	0.163	0.166	0.180	0.201	0.216
Min-K%++ Prob	-	0.130	0.141	0.215	0.201	0.381	0.245
DC-PDD	0.336*	0.264*	0.271*	0.180*	0.245*	0.230*	0.317*

Table 3: TPR@5%FPR scores for detecting pretraining texts. Bold indicates the best performing method. Two-tailed t-tests show that DC-PDD significantly improves over Min-K% Prob ( * indicates $p\leq 0.05$).

方法	BookMIA	PatentMIA	WikiMIA
	GPT-3	Baichuan-13B	Qwen1.5-14B	OPT-6.7B	Pythia-6.9B	Llama-13B	GPT-NeoX-20B
PPL	0.224	0.166	0.159	0.130	0.144	0.216	0.180
Lowercase	0.240	-	-	0.094	0.130	0.158	0.130
Zlib	0.192	0.159	0.129	0.180	0.209	0.187	0.223
Small Ref	-	0.211	0.078	0.122	0.158	0.151	0.187
Min-K% Prob	0.203	0.170	0.163	0.166	0.180	0.201	0.216
Min-K%++ Prob	-	0.130	0.141	0.215	0.201	0.381	0.245
DC-PDD	0.336*	0.264*	0.271*	0.180*	0.245*	0.230*	0.317*

表 3：检测预训练文本的 TPR@5%FPR。加粗为最优。双侧 t 检验显示 DC-PDD 相对 Min-K% Prob 显著提升（* 表示 $p\leq 0.05$）。

Implementation details. For the start-of-sentence token $x_{0}$ to prepend, we use <|endoftext|> in Pythia, Qwen1.5, GPT-NeoX and GPT-3, in OPT and Llama, and in Baichuan. For the reference corpus $\mathcal{D^{\prime}}$ to compute the token frequency distribution, we take a subset of C4 Raffel et al. (2020) ($\approx 15$Gb) for English text detection and take a subset of ChineseWebText Chen et al. (2023) ($\approx 15$Gb) for Chinese text detection. For hyperparameter $a$ settings, we set it to $0.01$ for WikiMIA and PatentMIA detection tasks, and to $10$ for BookMIA. Since we take the AUC score as our evaluation metric, we do not need to determine a specific threshold $\tau$ in our method. For the baseline implementation, we set $k=20$ to achieve the optimal performance of Min-K% Prob following Shi et al. (2024). Correspondingly, the hyperparameter $k$ in Min-K%++ Prob is also set to $20$ for fair comparison. For the smaller reference model setting, we employ OPT-350M as the smaller model for OPT-6.7B, Pythia-70M for Pythia-6.9B, Llama-7B for Llama-13B, GPT-Neo-125M for GPT-NeoX-20B, Baichuan-7B for Baichuan-13B and Qwen1.5-7B for Qwen1.5-14B.

Implementation details. For the start-of-sentence token $x_{0}$ to prepend, we use <|endoftext|> in Pythia, Qwen1.5, GPT-NeoX and GPT-3, in OPT and Llama, and in Baichuan. For the reference corpus $\mathcal{D^{\prime}}$ to compute the token frequency distribution, we take a subset of C4 Raffel et al. (2020) ($\approx 15$Gb) for English text detection and take a subset of ChineseWebText Chen et al. (2023) ($\approx 15$Gb) for Chinese text detection. For hyperparameter $a$ settings, we set it to $0.01$ for WikiMIA and PatentMIA detection tasks, and to $10$ for BookMIA. Since we take the AUC score as our evaluation metric, we do not need to determine a specific threshold $\tau$ in our method. For the baseline implementation, we set $k=20$ to achieve the optimal performance of Min-K% Prob following Shi et al. (2024). Correspondingly, the hyperparameter $k$ in Min-K%++ Prob is also set to $20$ for fair comparison. For the smaller reference model setting, we employ OPT-350M as the smaller model for OPT-6.7B, Pythia-70M for Pythia-6.9B, Llama-7B for Llama-13B, GPT-Neo-125M for GPT-NeoX-20B, Baichuan-7B for Baichuan-13B and Qwen1.5-7B for Qwen1.5-14B.

5 Experimental Results

5 实验结果

Here, we report our main results, several ablation studies, and additional experiments investigating factors influencing detection performance.

这里报告主结果、若干消融，以及影响检测表现的额外因素实验。

5.1 Main Results

5.1 主结果

Our results can be found in Table 2 and 3. We observe that: (i) DC-PDD surpasses most baselines across three benchmarks and various target models. For instance, on existing BookMIA benchmark, DC-PDD exceeds the best baseline Lowercase $5.4$% and $9.6$% in terms of AUC and TPR@5%FPR. On our PatentMIA benchmark, DC-PDD exceeds the best baseline Min-K% Prob $5.4$% and $13.2$% in terms of AUC and TPR@5%FPR. (ii) Compared to Min-K% Prob, the AUC improvement of DC-PDD on the WikiMIA benchmark is less than that of Min-K%++ Prob, possibly because WikiMIA has only 250 examples, with fewer cases shown in Figure 1 we aim to optimize. While Min-K%++ Prob calibrates token probabilities from other points, which might suit these examples better. This indicates that token probabilities are impacted by various factors and are unreliable for detection. Hence, we plan to explore better detection signals in the future. (iii) The superior performance of DC-PDD is more agnostic to data and models, in comparison to other methods. For example, while Min-K% Prob and Min-K%++ Prob perform well on models using the WikiMIA benchmark, they do not do as well on models using the PatentMIA benchmark. A similar phenomenon can be observed with the Zlib method. (iv) Additionally, the Small Ref method are not applicable to GPT-3, as closed-source models lack corresponding smaller models in the same series. The Min-K%++ Prob is also not applicable to GPT-3 since GPT-3 do not provide the access to the next-token prediction probability distribution across the model’s entire vocabulary. The Lowercase method is unsuitable for detecting Chinese text, as Chinese characters do not have case distinctions. (v) By evaluating performance on the PatentMIA benchmark, except for the Lowercase method, it is evident that existing methods are still effective for Chinese-language pretraining data detection, with our method consistently achieving the best results.

结果见表 2 与表 3。我们观察到：(i) DC-PDD 在三个基准、多种目标模型上超过多数基线。例如在既有 BookMIA 上，相对最优基线 Lowercase，AUC 与 TPR@5%FPR 分别高出 $5.4$% 与 $9.6$%；在 PatentMIA 上相对最优基线 Min-K% Prob 分别高出 $5.4$% 与 $13.2$%。(ii) 相对 Min-K% Prob，DC-PDD 在 WikiMIA 上的 AUC 提升小于 Min-K%++ Prob，可能因 WikiMIA 仅 250 例，图 1 所示我们希望优化的情形较少；而 Min-K%++ Prob 从其他角度校准概率，或许更契合这些样例。这表明 token 概率受多因素影响、用于检测并不可靠，我们计划未来探索更好的检测信号。(iii) 相对其他方法，DC-PDD 的优势更不依赖具体数据与模型。例如 Min-K% Prob 与 Min-K%++ Prob 在 WikiMIA 上表现不错，在 PatentMIA 上则一般；Zlib 也有类似现象。(iv) 此外，Small Ref 不适用于 GPT-3，因闭源模型缺少同系列更小模型；Min-K%++ Prob 也不适用于 GPT-3，因其不提供全词表下一 token 预测分布；Lowercase 不适合中文检测，因汉字无大小写。(v) 在 PatentMIA 上，除 Lowercase 外，现有方法对中文预训练数据检测仍有效，而我们的方法始终最优。

Figure 2: Ablation studies of DC-PDD

图 2：DC-PDD 的消融实验

5.2 Ablation Studies

5.2 消融实验

DC-PDD employs two strategies before using the calibrated token probabilities to compute the score $\beta$ for $x$ for detection. They are (i) LUP: Limiting the UPper bound of each calibrated token probability, w.r.t. Eq. (7), and (ii) SFO: only Selecting the calibrated token probabilities corresponding to tokens with the First Occurrence in $x$ to compute $\beta$, w.r.t. Eq. (8). We conduct ablation studies to explore the effect of these strategies using the following three method variants:

在用校准后的 token 概率计算检测分数 $\beta$ 之前，DC-PDD 采用两项策略：(i) LUP：限制各校准 token 概率的上界，见式 (7)；(ii) SFO：仅选取 $x$ 中首次出现的 token 对应的校准概率来计算 $\beta$，见式 (8)。我们用以下三个变体做消融：

• • CLD: It serves as the initialization of DC-PDD by averaging all the CaLibrateD token probabilities to compute a score for detection.

• • CLD：作为 DC-PDD 的初始化，对全部校准后的 token 概率取平均得到检测分数。

• • +LUP: Based on ‘CLD’, it incorporates the LUP strategy to compute $\beta$.

• • +LUP：在「CLD」基础上加入 LUP 策略计算 $\beta$。

• • +SFO: Based on ‘+LUP’, it further incorporates the SFO strategy to compute $\beta$.

• • +SFO：在「+LUP」上再加入 SFO 策略计算 $\beta$。

Results are shown in Figure 2. For Baichuan-13B and Qwen1.5-14B, both strategies contribute to the effectiveness of DC-PDD. However, for GPT-3, we found that the LUP strategy did not result in a significant performance improvement. We speculate that this may be related to the setting of the hyperparameter $a$ involved in the LUP strategy. Therefore, we discuss the impact of $a$ on DC-PDD in detail in Section 5.3.

结果见图 2。对 Baichuan-13B 与 Qwen1.5-14B，两项策略均有助于 DC-PDD。但对 GPT-3，LUP 未带来显著提升。我们推测这可能与 LUP 中超参 $a$ 的设定有关，故在第 5.3 节详细讨论 $a$ 的影响。

(a) AUC score vs. model size.

(a) AUC 随模型规模变化。

5.3 Impact of Different Factors

5.3 不同因素的影响

This section explores several factors that may influence the performance of DC-PDD, including two method-independent factors (model size and text length) and two method-dependent factors (the reference corpus $\mathcal{D^{\prime}}$ and hyperparameter $a$).

本节探讨可能影响 DC-PDD 表现的若干因素，包括两个与方法无关的因素（模型规模与文本长度）以及两个与方法相关的因素（参照语料 $\mathcal{D^{\prime}}$ 与超参 $a$）。

Model size. To investigate the impact of model size on the performance of DC-PDD, we analyze the Qwen1.5 family with models of 1.8B, 4B, 7B, and 14B versions to determine if larger models demonstrate improved results. As illustrated in Figure 3(a), DC-PDD consistently achieves the best results across all model sizes, and like other methods, the AUC score increases as the model size grows, confirming findings from prior research Shi et al. (2024); Liu et al. (2024). The reason for this trend is probably because larger models, having more parameters, are better at memorizing the pre-training data.

模型规模。 为考察模型规模对 DC-PDD 的影响，我们在 Qwen1.5 系列的 1.8B、4B、7B、14B 版本上分析更大模型是否更好。如图 3(a) 所示，DC-PDD 在各规模上均最优，且与其他方法一样，AUC 随规模增大而提升，印证先前研究 Shi et al. (2024); Liu et al. (2024)。原因可能是更大模型参数更多、更能记忆预训练数据。

Text length. We further explore the potential impact of text length on the performance of DC-PDD. For this purpose, we perform assessments using four different length settings (64, 128, 256, 512) in our PatentMIA benchmark to determine whether short texts are more challenging than longer texts. Figure 3(b) illustrates that DC-PDD still consistently outperforms other baselines across all text length settings, and the AUC score also improves with increasing length in Chinese-language pretraining data detection. This trend may be due to the fact that longer texts carry more information that the target model has memorized, making them easier to differentiate from non-training texts.

文本长度。 我们进一步考察文本长度的影响。在 PatentMIA 上用 64、128、256、512 四种长度设定，看短文本是否更难。图 3(b) 显示 DC-PDD 在各长度上仍始终优于其他基线，且 AUC 随长度增加而提升。该趋势可能因更长文本携带更多目标模型已记忆的信息，更易与非训练文本区分。

Reference corpus $\mathcal{D^{\prime}}$. Recall that we use a reference corpus $\mathcal{D^{\prime}}$ to estimate the token frequency distribution of the LLM’s pretraining corpus, w.r.t. Eq. (4). To analyze the effect of different reference corpora on the efficacy of the method, we compare the performance of DC-PDD under various reference corpus settings across different scales and domains. Specifically, when detecting WikiMIA-128 from pythia-6.9B, we employee $\approx 1$Gb of C4 corpus, $\approx 10$Gb of C4 corpus, $\approx 1$Gb of Case-law corpus, and $\approx 10$Gb of Case-law corpus as the reference corpus respectively. Note that the Case-law Louis Brulé Naudet (2024) is a corpus in the legal domain. As shown in Table 4, We observe that the performance of DC-PDD does not exhibit significant differences across the various reference corpora, indicating that DC-PDD is not sensitive to the selection of a reference corpus. Notably, when the reference corpus is chosen as the $\approx 10$Gb of C4 corpus, the performance of DC-PDD is the best. This enhancement may be attributed to the greater diversity of the C4 corpus compared to the $\approx 10$Gb of Case-law corpus, as well as the richer data compared to the $\approx 1$Gb of C4 corpus, which allow for a more accurate estimation of the token frequency distribution in the LLM’s pretraining corpus, thereby resulting in better performance.

参照语料 $\mathcal{D^{\prime}}$。 回顾我们用参照语料 $\mathcal{D^{\prime}}$ 估计 LLM 预训练语料的 token 频率分布，见式 (4)。为分析不同参照语料的影响，我们在不同规模与领域设定下比较 DC-PDD。具体地，在用 pythia-6.9B 检测 WikiMIA-128 时，分别以约 1Gb C4、约 10Gb C4、约 1Gb Case-law、约 10Gb Case-law 为参照。Case-law Louis Brulé Naudet (2024) 为法律领域语料。如表 4，各参照语料下 DC-PDD 表现差异不大，说明对参照选择不敏感。其中约 10Gb C4 最好，可能因其相对约 10Gb Case-law 更多样，也比约 1Gb C4 更丰富，从而更准确估计预训练语料的 token 频率分布。

$\mathcal{D^{\prime}}$	C4	Case-law
$\approx 1$Gb	$\approx 10$Gb	$\approx 1$Gb	$\approx 10$Gb
Pythia-6.9B	0.688	0.698	0.687	0.688

Table 4: AUC scores of DC-PDD in different reference corpus settings.

$\mathcal{D^{\prime}}$	C4	Case-law
	$\approx 1$Gb	$\approx 10$Gb	$\approx 1$Gb	$\approx 10$Gb
Pythia-6.9B	0.688	0.698	0.687	0.688

表 4：不同参照语料设定下 DC-PDD 的 AUC。

Hyperparameter $a$. Recall that we set a hyperparameter $a$ to prevent the final score from being dominated by a few tokens, w.r.t. Eq. (7). We evaluate DC-PDD with different $a$ settings to investigate their impact on detection performance. As shown in Table 5, performance varies significantly with $a$ set to $0.001$, $0.01$, $0.1$, $1$, and $10$. Actually, if $a$ is set too high, it does not effectively limit the calibrated token probabilities. Conversely, if set too low, it will result in nearly equal calibrated token probabilities, causing scores for training and non-training text to be similar and thus, ineffective for detection. From the Table 5, we can see that the optimal $a$ setting varies across different target models and benchmarks. For instance, the optimal $a$ is $10$ in detecting BookMIA from GPT-3 while it is $0.01$ in detecting PatentMIA from Qwen1.5-14B. When $a$ is set to $0.01$, the overall performance for all models is optimal. Therefore, we recommend setting $a$ to $0.01$ when using DC-PDD for pretraining data detection in practical scenarios. In future work, we will explore more flexible methods for setting $a$ to achieve better performance of DC-PDD.

超参 $a$。 回顾我们设超参 $a$ 以防最终分数被少数 token 主导，见式 (7)。我们在不同 $a$ 下评估 DC-PDD。如表 5，在 $0.001$、$0.01$、$0.1$、$1$、$10$ 下表现差异显著。$a$ 过大则无法有效限制校准概率；$a$ 过小则校准概率近乎相等，训练与非训练文本分数相近，检测失效。表 5 显示最优 $a$ 随目标模型与基准而变：例如在 GPT-3 上检测 BookMIA 时最优为 $10$，在 Qwen1.5-14B 上检测 PatentMIA 时最优为 $0.01$。当 $a=0.01$ 时各模型总体最优。因此实践中推荐将 $a$ 设为 $0.01$。未来将探索更灵活的 $a$ 设定以进一步提升 DC-PDD。

$a$	0.001	0.01	0.1	1	10
PatentMIA:
Baichuan-13B	0.645	0.699	0.664	0.647	0.645
Qwen1.5-14B	0.640	0.689	0.652	0.623	0.619
BookMIA:
GPT-3	0.673	0.676	0.665	0.667	0.725

Table 5: AUC scores of DC-PDD in different $a$ settings.

$a$	0.001	0.01	0.1	1	10
PatentMIA：
Baichuan-13B	0.645	0.699	0.664	0.647	0.645
Qwen1.5-14B	0.640	0.689	0.652	0.623	0.619
BookMIA：
GPT-3	0.673	0.676	0.665	0.667	0.725

表 5：不同 $a$ 设定下 DC-PDD 的 AUC。

6 Related Work

6 相关工作

Membership inference attack (MIA). MIA is the de-facto threat model when evaluating privacy concerns in machine learning models. First introduced by Shokri et al. (2017), MIA’s objective is to ascertain whether a specific sample was part of a model’s training dataset. Prior MIA research has focused on traditional deep learning models Sablayrolles et al. (2019); Song and Shmatikov (2019) and fine-tuning language models Hisamoto et al. (2020); Jagannatha et al. (2021); Mattern et al. (2023). But recently, MIA on LLMs has attracted growing attention with various applications, including examination of training data memorization Nasr et al. (2023), data contamination Oren et al. (2023), and copyright infringement Meeus et al. (2023); Duarte et al. (2024). We consider a different type of MIA: pretraining data detection.

成员推断攻击（MIA）。 MIA 是评估机器学习模型隐私风险时事实上的威胁模型。Shokri et al. (2017) 首次提出，目标是判断特定样本是否属于模型训练集。先前 MIA 研究聚焦传统深度学习模型 Sablayrolles et al. (2019); Song and Shmatikov (2019) 以及微调语言模型 Hisamoto et al. (2020); Jagannatha et al. (2021); Mattern et al. (2023)。近来面向 LLM 的 MIA 受到更多关注，应用包括训练数据记忆检查 Nasr et al. (2023)、数据污染 Oren et al. (2023) 与版权侵权 Meeus et al. (2023); Duarte et al. (2024)。我们考虑另一类 MIA：预训练数据检测。

Pretraining data detection for LLMs. Here, the MIA problem centers on identifying whether a piece of text was used by an LLM for pretraining. According to the access conditions to LLMs, current pretraining data detection methods for LLMs can be divided into two categories: (i) The white-box setting: assuming one has access to internals of LLMs, such as weights and activations. (ii) The black-box setting: assuming one can only query LLMs to compute token probabilities for the text.

面向 LLM 的预训练数据检测。 此处 MIA 问题聚焦判断某段文本是否被 LLM 用于预训练。按对 LLM 的访问条件，现有方法可分两类：(i) 白盒：可访问权重与激活等内部；(ii) 黑盒：只能查询 LLM 以计算文本的 token 概率。

There is limited research on the white-box setting since the internals of LLMs are typically not disclosed, rendering detection methods in white-box scenarios impractical. Liu et al. (2024) propose to use the probing technique for pretraining data detection, based on the assumption that texts encountered during the LLM’s pretraining phase are represented differently in its internal activations compared to unseen texts.

白盒研究有限，因 LLM 内部通常不公开，白盒检测不实用。Liu et al. (2024) 提出用探测技术做预训练数据检测，假定预训练阶段见过的文本与未见文本在内部激活中表示不同。

Most research focuses on the black-box setting, assuming that the token probability distribution of a text can provide crucial information about whether the text was included in the training set. Carlini et al. (2021) considered the model’s perplexity for a text as an indicator to detect pretraining data from GPT-2 Radford et al. (2019). They further introduced three methods, Zlib, Lowercase, and Smaller Ref, that take into account the intrinsic complexity of the target text. More recently, Shi et al. (2024) have proposed a straightforward yet well-performing method called Min-K% Prob. Min-K% Prob tends to classify a non-training text composed of common words as training data. A concurrent study Min-K%++ Prob Zhang et al. (2024) improves Min-K% Prob by normalizing token probabilities, but requires access to the next-token prediction probability distribution across the LLM’s entire vocabulary, which is unavailable in closed-source LLMs like GPT-3 Brown et al. (2020).

多数研究聚焦黑盒，假定文本的 token 概率分布能提供其是否进入训练集的关键信息。Carlini et al. (2021) 将模型对文本的困惑度作为从 GPT-2 Radford et al. (2019) 检测预训练数据的指标，并进一步引入考虑目标文本固有复杂度的 Zlib、Lowercase 与 Smaller Ref。近来 Shi et al. (2024) 提出简洁且表现良好的 Min-K% Prob：它倾向于把由常见词组成的非训练文本判为训练数据。同期工作 Min-K%++ Prob Zhang et al. (2024) 通过归一化 token 概率改进 Min-K% Prob，但需要全词表下一 token 预测分布，闭源 LLM（如 GPT-3 Brown et al. (2020)）通常不可得。

We consider the black-box setting and calibrate the token probabilities before using them for detection. What distinguishes our approach is that it neither requires additional reference models (unlike Small Ref) nor does it have extra access requirements on the LLM (unlike Min-K%++ Prob).

我们考虑黑盒设定，并在用于检测前校准 token 概率。与其他方法的区别在于：既不需要额外参照模型（不同于 Small Ref），也不对 LLM 提出额外访问要求（不同于 Min-K%++ Prob）。

7 Conclusion

7 结论

In this work, we proposed DC-PDD to improve methods that directly rely on token probabilities for pretraining data detection, which tend to misclassify non-training texts containing many common words as training texts. The key idea of DC-PDD is to calibrate the token probabilities and thereby make them more informative signals for detection. The calibration process is achieved by computing the cross-entropy (i.e., the divergence) between the token probability distribution and the token frequency distribution. Experiments demonstrate the superior performances of DC-PDD compared to various baselines. In future work, we want to detect whether an LLM was pretrained on a given corpus (corpus-level detection), rather than just on a piece of text (sample-level detection).

本文提出 DC-PDD，改进那些直接依赖 token 概率做预训练数据检测、因而容易把含大量常见词的非训练文本误判为训练文本的方法。核心想法是校准 token 概率，使其成为更有信息量的检测信号。校准通过计算 token 概率分布与 token 频率分布之间的交叉熵（即散度）完成。实验表明 DC-PDD 相对多种基线表现更优。未来我们希望检测 LLM 是否在给定语料（语料级）而非仅在单段文本（样本级）上做过预训练。

Limitations

局限

DC-PDD, while showing promising results in pre-training data detection from LLMs, has several limitations. (i) DC-PDD utilizes a reference corpus to calculate the token frequency distribution to estimate that of the training corpus. Although working, the similarity between these two distributions remains uncertain. Additionally, the language of reference corpus should be the same as that of text to be detected. (ii) Secondly, an important hyperparameter in DC-PDD is the upper bound of calibrated token probabilities. We have demonstrated its significant impact on method performance, but not how the optimal value should be set. We leave this issue to future work. (iii) Thirdly, DC-PDD is specific to textual data. While some detection methods can be applied universally across different data modalities by relying on sample-level loss values obtained from models, our method is based on token-level probability. This specificity hinders its direct application to other types of data, such as images. (iv) Fourthly, DC-PDD requires access to token probabilities, and therefore is not applicable to some closed-source models. In the future, we will explore detection methods based solely on model output to design more generalizable detection methods. (v) Lastly, except for the closed-source model GPT-3 Brown et al. (2020), our research primarily focused on models with up to 20 billion parameters due to hardware constraints. Further studies replicating our work using larger-scale models will be essential to validate the effectiveness of DC-PDD in scenarios involving larger models.

尽管 DC-PDD 在从 LLM 检测预训练数据上表现可观，仍有若干局限。(i) 它用参照语料计算 token 频率分布以估计训练语料频率；虽可用，两分布相似程度仍不确定，且参照语料语言须与待检测文本一致。(ii) 关键超参是校准 token 概率的上界；我们已证明其对表现影响显著，但尚未说明最优值应如何设定，留待未来。(iii) DC-PDD 专用于文本数据。一些检测方法可凭样本级损失跨模态通用，而我们基于 token 级概率，难以直接用于图像等其他模态。(iv) 需要访问 token 概率，因而对部分闭源模型不适用。未来将探索仅依赖模型输出的更通用检测法。(v) 除闭源 GPT-3 Brown et al. (2020) 外，受硬件限制，研究主要聚焦至多约 200 亿参数的模型。在更大模型上复现以验证 DC-PDD 将至关重要。

Ethical Considerations

伦理考量

Although DC-PDD aims to address issues such as copyright infringement or data contamination through pretraining data detection, it can also be used to compromise the privacy of individuals whose data has been used to train models, as pretraining data detection problem is an instance of Membership Inference Attacks (MIAs). Recognizing the potential risks associated with MIAs, we are extremely cautious with the data we use to ensure there is limited risk of any exposure of confidential data. For example, the PatentMIA benchmark is collected from the publicly available Google-Patents website and does not involve personal privacy data. Additionally, the other benchmarks we use have also been employed in prior research and do not pose any privacy risks.

尽管 DC-PDD 旨在通过预训练数据检测应对版权侵权或数据污染等问题，它也可被用于损害其数据被用于训练的个体的隐私，因为预训练数据检测是成员推断攻击（MIA）的一种。认识到 MIA 相关风险后，我们在数据使用上极为谨慎，以限制机密数据暴露。例如 PatentMIA 采集自公开的 Google-Patents，不涉及个人隐私；我们使用的其他基准也已在先前研究中采用，不构成隐私风险。

Acknowledgements

致谢

This work was funded by the National Natural Science Foundation of China (NSFC) under Grants No. 62472408 and 62372431, the Strategic Priority Research Program of the CAS under Grants No. XDB0680102 and XDB0680301, the National Key Research and Development Program of China under Grants No. 2023YFA1011602 and 2021QY1701, the Youth Innovation Promotion Association CAS under Grants No. 2021100, the Lenovo-CAS Joint Lab Youth Scientist Project, and the project under Grants No. JCKY2022130C039. This work was also (partially) funded by the Dutch Research Council (NWO), under project numbers 024.004.022, NWA.1389.20.183, and KICH3.LTP.20.006, and the European Union’s Horizon Europe program under grant agreement No 101070212.All content represents the opinion of the authors, which is not necessarily shared or endorsed by their respective employers and/or sponsors.

本工作获国家自然科学基金（NSFC）项目 62472408 与 62372431、中国科学院战略性先导科技专项 XDB0680102 与 XDB0680301、国家重点研发计划 2023YFA1011602 与 2021QY1701、中国科学院青年创新促进会 2021100、联想—中科院联合实验室青年科学家项目，以及 JCKY2022130C039 等资助。亦部分获荷兰科学研究组织（NWO）项目 024.004.022、NWA.1389.20.183 与 KICH3.LTP.20.006，以及欧盟地平线欧洲计划资助协议 101070212 支持。文中观点仅为作者个人，未必代表各自雇主与/或资助方。

References

参考文献

Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774 .

Amati and van Rijsbergen (2002) Gianni Amati and Cornelis Joost van Rijsbergen. 2002. Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Transactions on Information Systems (TOIS) , 20(4):357–389.

Bai et al. (2023) Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609 .

Biderman et al. (2023) Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. 2023. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning , pages 2397–2430. PMLR.

Black et al. (2022) Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, et al. 2022. GPT-NeoX-20B: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745 .

Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems , 33:1877–1901.

Cao et al. (2024) Jialun Cao, Wuqi Zhang, and Shing-Chi Cheung. 2024. Concerned with data contamination? Assessing countermeasures in code language model. arXiv preprint arXiv:2403.16898 .

Carlini et al. (2022) Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. 2022. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP) , pages 1897–1914. IEEE.

Carlini et al. (2021) Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2633–2650.

Chang et al. (2023) Kent K. Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. 2023. Speak, memory: An archaeology of books known to ChatGPT/GPT-4. arXiv preprint arXiv:2305.00118 .

Chen et al. (2023) Jianghao Chen, Pu Jian, Tengxiao Xi, Yidong Yi, Chenglin Ding, Qianlong Du, Guibo Zhu, Chengqing Zong, Jinqiao Wang, and Jiajun Zhang. 2023. Chinesewebtext: Large-scale high-quality chinese web text extracted with effective evaluation model. arXiv preprint arXiv:2311.01149 .

Dong et al. (2024) Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, and Ge Li. 2024. Generalization or memorization: Data contamination and trustworthy evaluation for large language models. arXiv preprint arXiv:2402.15938 .

Duan et al. (2024) Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. 2024. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841 .

Duarte et al. (2024) André V Duarte, Xuandong Zhao, Arlindo L Oliveira, and Lei Li. 2024. De-cop: Detecting copyrighted content in language models training data. arXiv preprint arXiv:2402.09910 .

Google (2006) Google. 2006. Google Patents. https://patents.google. com/ .

Grynbaum and Mac (2023) Michael M. Grynbaum and Ryan Mac. 2023. The Times sues OpenAI and Microsoft over A.I. use of copyrighted work. .

Hisamoto et al. (2020) Sorami Hisamoto, Matt Post, and Kevin Duh. 2020. Membership inference attacks on sequence-to-sequence models: Is my data in your machine translation system? Transactions of the Association for Computational Linguistics , 8:49–63.

Jagannatha et al. (2021) Abhyuday Jagannatha, Bhanu Pratap Singh Rawat, and Hong Yu. 2021. Membership inference attack susceptibility of clinical language models. arXiv preprint arXiv:2104.08305 .

Jiang et al. (2019) Shaojie Jiang, Pengjie Ren, Christof Monz, and Maarten de Rijke. 2019. Improving neural response diversity with frequency-aware cross-entropy loss. In The World Wide Web Conference , pages 2879–2885.

Liu et al. (2024) Zhenhua Liu, Tong Zhu, Chuanyuan Tan, Haonan Lu, Bing Liu, and Wenliang Chen. 2024. Probing language models for pre-training data detection. arXiv preprint arXiv:2406.01333 .

Louis Brulé Naudet (2024) Timothy Dolan Louis Brulé Naudet. 2024. The case-law, centralizing legal decisions for better use.

Mattern et al. (2023) Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick. 2023. Membership inference attacks against language models via neighbourhood comparison. arXiv preprint arXiv:2305.18462 .

Meeus et al. (2023) Matthieu Meeus, Shubham Jain, Marek Rei, and Yves-Alexandre de Montjoye. 2023. Did the neurons read your book? Document-level membership inference for large language models. arXiv preprint arXiv:2310.15007 .

Mozes et al. (2023) Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis D. Griffin. 2023. Use of LLMs for illicit purposes: Threats, prevention measures, and vulnerabilities. arXiv preprint arXiv:2308.12833 .

Nasr et al. (2023) Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. 2023. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311.17035 .

Oren et al. (2023) Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, and Tatsunori B Hashimoto. 2023. Proving test set contamination in black box language models. arXiv preprint arXiv:2310.17623 .

Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9.

Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140):1–67.

Sablayrolles et al. (2019) Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Hervé Jégou. 2019. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning , pages 5558–5567. PMLR.

Shi et al. (2024) Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2024. Detecting pretraining data from large language models . In The Twelfth International Conference on Learning Representations .

Shokri et al. (2017) Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) , pages 3–18. IEEE.

Song and Shmatikov (2019) Congzheng Song and Vitaly Shmatikov. 2019. Auditing data provenance in text-generation models. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 196–206.

Stempel (2024) Jonathan Stempel. 2024. Nvidia is sued by authors over AI use of copyrighted works. .

Team (2024) Qwen Team. 2024. Introducing qwen1.5 .

Touvron et al. (2023a) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 .

Touvron et al. (2023b) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 .

Watson et al. (2021) Lauren Watson, Chuan Guo, Graham Cormode, and Alex Sablayrolles. 2021. On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2111.08440 .

Yang et al. (2023) Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305 .

Yeom et al. (2018) Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. 2018. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st Computer Security Foundations Symposium (CSF) , pages 268–282. IEEE.

Zhang et al. (2024) Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Yang, and Hai Li. 2024. Min-K%++: Improved baseline for detecting pre-training data from large language models. arXiv preprint arXiv:2404.02936 .

Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. OPT: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 .

Appendix A Appendix

附录 A 附录

A.1 Baseline details

A.1 基线细节

The baselines are all based on a detection score to determine a text $x$ whether was included in the per-training corpus of an LLM $\mathcal{M}$. Followings are the details of how they calculate the detection score.

各基线均基于一个检测分数判断文本 $x$ 是否属于 LLM $\mathcal{M}$ 的预训练语料。以下说明分数如何计算。

PPL. Carlini et al. (2021) This is an instance of Loss Attack proposed by Yeom et al. (2018). In the context of LLMs, this loss corresponds to perplexity. Thus, the detection score is the perplexity of $x$. A low score suggests that $x$ was likely part of the pretraining data.

PPL。 Carlini et al. (2021) 这是 Yeom et al. (2018) 提出的损失攻击在 LLM 语境下的实例，损失对应困惑度。因此检测分数即 $x$ 的困惑度。分数低暗示 $x$ 更可能属于预训练数据。

Small Ref. Carlini et al. (2021) This method exactly follows the approach described by Watson et al. (2021), which assumes access to a reference model, $\mathcal{M}_{ref}$, trained on a disjoint set of training data drawn from a similar distribution and posits that the intrinsic complexity of $x$ can be quantified as $\mathcal{M}_{ref}$’s perplexity for $x$. Since the assumption is impractical, the Small Ref method employs a smaller model from the same family of $\mathcal{M}$ as a substitute for $\mathcal{M}_{ref}$, and then calibrate $\mathcal{M}$’s perplexity for $x$ using a difficulty estimate through the smaller model’s perplexity for $x$. Consequently, the detection score is calculated as the ratio of $x$’s perplexity under $\mathcal{M}$ to $x$’s perplexity under a smaller model pre-trained on the same data. A low score suggests that $x$ was likely part of the pretraining data.

Small Ref。 Carlini et al. (2021) 完全遵循 Watson et al. (2021)：假定可访问在相似分布、不相交训练数据上训练的参照模型 $\mathcal{M}_{ref}$，并用其对 $x$ 的困惑度量化 $x$ 的固有复杂度。该假定不现实，故 Small Ref 用与 $\mathcal{M}$ 同系列的更小模型代替 $\mathcal{M}_{ref}$，再以更小模型困惑度校准 $\mathcal{M}$ 对 $x$ 的困惑度。检测分数为 $x$ 在 $\mathcal{M}$ 上的困惑度与同数据预训练更小模型上困惑度之比。分数低暗示更可能属于预训练数据。

Zlib. Carlini et al. (2021) Similar to the Small Ref method, but uses the zlib entropy of $x$ in place of the smaller model’s perplexity for $x$. The zlib entropy is the entropy in bits when the sequence is compressed using zlib.[^3] The detection score is then determined by the ratio of $\mathcal{M}$’s perplexity for $x$ to the zlib entropy of $x$. A low score suggests that $x$ was likely part of the pretraining data.

Zlib。 Carlini et al. (2021) 类似 Small Ref，但用 $x$ 的 zlib 熵代替更小模型困惑度。zlib 熵是用 zlib 压缩该序列时的比特熵。[^3]检测分数为 $\mathcal{M}$ 对 $x$ 的困惑度与 zlib 熵之比。分数低暗示更可能属于预训练数据。

Lowercase. Carlini et al. (2021) Similarly to the Small Ref method, but uses $\mathcal{M}$’s perplexity for the lowercase of $x$ to replace the smaller model’s perplexity for $x$. The detection score is then determined by the ratio of $\mathcal{M}$’s perplexity for $x$ to $\mathcal{M}$’s perplexity for the lowercase of $x$. A low score suggests that $x$ was likely part of the pretraining data.

Lowercase。 Carlini et al. (2021) 同样类似 Small Ref，但用 $\mathcal{M}$ 对 $x$ 小写版本的困惑度代替更小模型困惑度。检测分数为 $\mathcal{M}$ 对 $x$ 的困惑度与对小写 $x$ 困惑度之比。分数低暗示更可能属于预训练数据。

Min-K% Prob. Shi et al. (2024) Min-K% Prob is based on the intuition that non-member examples tend to have more tokens assigned lower probabilities than member examples do. Thus, it begins by calculating the probability of each token in $x$, then selects the k% of tokens with the lowest probabilities to compute their average log-likelihood as the detection score. A high score suggests that $x$ was likely part of the pretraining data.

Min-K% Prob。 Shi et al. (2024) 基于非成员样例往往有更多被赋予更低概率的 token 这一直觉。先计算 $x$ 中各 token 概率，再取概率最低的 k% token，以其平均对数似然为检测分数。分数高暗示更可能属于预训练数据。

Min-K%++ Prob. Zhang et al. (2024) The underlying idea of Min-K%++ Prob is that if the probability of the current input token surpasses the probabilities of other tokens in the vocabulary, it is probable that the input has been seen during training, irrespective of the actual probability value of the input token. Therefore, it first calculates the probability of each token in $x$, then normalizes the token probability using the statistics of the categorical distribution over the entire vocabulary, and finally selects the $k$% of tokens with the lowest normalized probabilities to compute their average as the detection score. A high score suggests that $x$ was likely part of the pretraining data.

Min-K%++ Prob。 Zhang et al. (2024) 核心想法是：若当前输入 token 的概率超过词表中其他 token，则不论其绝对概率值大小，该输入都更可能在训练中见过。因此先计算各 token 概率，再用全词表类别分布统计做归一化，最后取归一化概率最低的 $k$% token 的平均作为检测分数。分数高暗示更可能属于预训练数据。

A.2 Metrics

A.2 指标

Area Under the ROC Curve (AUC). The AUC score quantifies the overall performance of a classification method. To calculate the AUC score for a method, we need to compute the True Positive Rates (TPRs) and False Positive Rates (FPRs) at all classification thresholds and plot a TPR vs. FPR curve, known as the ROC curve. The AUC is then defined as the Area Under the ROC curve, providing an aggregate measure of the effect of all possible classification thresholds. Therefore, AUC provides a comprehensive, threshold-independent score that reflects the method’s ability to distinguish between positive and negative cases effectively.

ROC 曲线下面积（AUC）。 AUC 量化分类方法的总体表现。计算时需在所有分类阈值下得到真阳性率（TPR）与假阳性率（FPR），绘制 TPR–FPR 曲线（即 ROC 曲线）；AUC 定义为该曲线下面积，是对所有可能阈值效果的综合度量，因而提供与阈值无关的、反映区分正负例能力的分数。

TPR (true positive rate) at a low FPR (false positive rate). We report TPR at a low FPR by adjusting the threshold value, Specifically, we choose $5$% as our target FPR value, and report the corresponding TPR value.

低 FPR 下的 TPR。 我们通过调整阈值报告低 FPR 下的 TPR。具体取目标 FPR 为 $5$%，报告对应的 TPR。