The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models

PRISM Alignment 数据集：参与式、有代表性且个体化的人类反馈，如何揭示大语言模型在主观与多元文化对齐上的面貌

Hannah Rose Kirk Thanks:  {hannah.kirk,scott.hale}@oii.ox.ac.uk $†$Joint last authors; $‡$Work done at University of Sheffield Affiliation:  University of Oxford    Alexander Whitefield Affiliation:  University of Pennsylvania    Paul Röttger Affiliation:  Bocconi University    Andrew Bean Affiliation:  University of Oxford Affiliation:  AWS AI Labs    Katerina Margatina    Juan Ciro Affiliation:  ML Commons Affiliation:  Contextual AI    Rafael Mosquera Affiliation:  ML Commons Affiliation:  Factored AI    Max Bartolo Affiliation:  UCL Affiliation:  Cohere    Adina Williams Affiliation:  MetaAI    He He Affiliation:  New York University    Bertie Vidgen Affiliation:  University of Oxford    Scott A. Hale Affiliation:  University of Oxford Affiliation:  Meedan

Abstract

摘要

Human feedback is central to the alignment of Large Language Models (LLMs). However, open questions remain about methods (how), domains (where), people (who) and objectives (to what end) of feedback processes. To navigate these questions, we introduce Prism, a dataset that maps the sociodemographics and stated preferences of 1,500 diverse participants from 75 countries, to their contextual preferences and fine-grained feedback in 8,011 live conversations with 21 LLMs. With Prism, we contribute (i) wider geographic and demographic participation in feedback; (ii) census-representative samples for two countries (UK, US); and (iii) individualised ratings that link to detailed participant profiles, permitting personalisation and attribution of sample artefacts. We target subjective and multicultural perspectives on value-laden and controversial issues, where we expect interpersonal and cross-cultural disagreement. We use Prism in three case studies to demonstrate the need for careful consideration of which humans provide what alignment data.   Data & Code: github.com/HannahKirk/prism-alignment

人类反馈是大语言模型（LLM）对齐的核心。不过，关于反馈过程的方法（如何做）、领域（在何处）、人群（向谁问）和目标（为了什么）仍有未决问题。为了应对这些问题，我们推出 Prism：它把来自 75 个国家的 1,500 名多样化参与者的社会人口统计与陈述偏好，映射到他们与 21 个 LLM 进行的 8,011 场实时对话中的情境偏好和细粒度反馈。借助 Prism，我们贡献了 (i) 更广的地理与人口参与；(ii) 两个国家（英国、美国）的普查代表性样本；以及 (iii) 与详细参与者档案相连接的个体化评分，从而支持个性化，也能归因样本伪迹。我们聚焦价值负载和争议议题上的主观与多元文化视角，这些地方最容易出现人际和跨文化分歧。我们用 Prism 做了三项案例研究，说明必须仔细考虑：是哪些人提供了什么样的对齐数据。 数据与代码： github.com/HannahKirk/prism-alignment

Data & Dataset Card: huggingface.co/datasets/HannahRoseKirk/prism-alignment

数据与 Dataset Card： huggingface.co/datasets/HannahRoseKirk/prism-alignment

1 Introduction

1 引言

Human feedback serves a direct role for the alignment of large language models (LLMs), defined as the steering of AI behaviour towards a set of preferences or values. This increased emphasis on human feedback raises unresolved questions: how we collect human feedback when designing methodologies that rely on ordinal or cardinal scales, broad or fine-grained desiderata, and explicit or implicit signals; where we focus human labour when selecting domains, topics or tasks to collect feedback over; who we ask for feedback when recruiting participants to voice their idiosyncratic preferences, values, or beliefs [1]; and to what end when specifying an objective to pursue personalised alignment [2, 3, 4] or to aggregate individual preferences into collective outcomes favourable for societies at large [5, 6, 7, 8, 9].

人类反馈在大语言模型（LLM）的对齐中起着直接作用。对齐被定义为把 AI 行为引导向某一组偏好或价值。这种对人类反馈的重视提出了尚未解决的问题：我们如何收集人类反馈——在设计方法时要在序数或基数量表、宽泛或细粒度的期望、显式或隐式信号之间做出选择；我们把人类劳动投向何处——在选择收集反馈的领域、话题或任务时；我们向谁征求反馈——在招募参与者表达其独特偏好、价值或信念时 [1]；以及为了什么目的——在指定目标时，是追求个性化对齐 [2, 3, 4]，还是把个体偏好汇总成对社会整体有利的集体结果 [5, 6, 7, 8, 9]。

Despite the success of human feedback learning [10, 11], answering these questions is constrained by gaps in existing datasets, such as (i) over-reliance on binary A/B comparisons, without fine-grained ratings or explanations [12]; (ii) small or biased samples recruited from narrow crowdwork or tech communities [10, 13] (iii) limited sample information (annotator IDs or sociodemographics) [14]; and (iv) scarce documentation for how values are operationalised [15, 16]. Most datasets rely only on revealed or contextual preferences [1],[^1] and much attention is devoted to technical or statistical issues in feedback learning [18, 19, 20], rather than data-centric human factors. Relying on ‘generic’ human data teaches behaviours which are reductionist because values are relational and non-separable from the person, community or operating context [21, 22, 23]; and non-generalisable because the indiscriminate aggregation of data subsumes hidden annotator contexts as universalities [24, 25, 26, 27, 28].

尽管从人类反馈中学习已经取得成功 [10, 11]，回答这些问题仍受现有数据集缺口的制约，例如 (i) 过度依赖二元 A/B 比较，缺少细粒度评分或解释 [12]；(ii) 样本偏小或有偏，主要来自狭窄的众包或技术社区 [10, 13]；(iii) 样本信息有限（标注者 ID 或社会人口统计）[14]；以及 (iv) 对价值如何被操作化缺少充分记录 [15, 16]。大多数数据集只依赖揭示偏好或情境偏好 [1]，[^1] 而大量注意力放在反馈学习的技术或统计问题上 [18, 19, 20]，而不是以数据为中心的人的因素。依赖“通用”人类数据会教出还原论的行为，因为价值是关系性的，无法与个人、社区或运作语境分离 [21, 22, 23]；也会教出不可推广的行为，因为不加区分地汇总数据，会把隐藏的标注者语境当成普遍性 [24, 25, 26, 27, 28]。

Human Participants					Large Language Models
Survey Participants	1,500		Conversations		Total Providers	6
With Conversations	1,396 (93.1%)	$\Rightarrow$	Total Conversations	8,011	$\Leftarrow$	Total Models	21
Birth Countries	75		Unguided	3,113 (38.9%)		Commercial API	12 (57.1%)
Reside Countries	38		Controversy guided	2,438 (30.4%)		Open Access	9 (42.9%)
Conversations / participant	5.7$\pm$1.0		Values guided	2,460 (30.7%)		Conversations / model	1430.9$\pm$171.1
Models seen / participant	13.9$\pm$2.5		Turns / conversation	3.4$\pm$1.6		Unique raters / model	924.3$\pm$94.4
			Total Interactions	27,172
			Total Utterances	68,371

Figure 1: The Prism dataset. In Stage 1, 1,500 participants fill in the Survey detailing their background, familiarity with LLMs and stated preferences over behaviours (Section 2.1). Demographic and geographic breakdowns are in Table 5 and Table 8). Participants then progress to Stage 2, where they converse with LLMs on topics of their choosing, rate the responses on a cardinal scale, and give fine-grained feedback (Section 2.2). In the first turn, four models respond to the opening prompt (; , , , ). In subsequent turns, the conversation continues with two responses sampled from the highest-rated model at a non-deterministic temperature (; ). There are 8,011 Conversations between participants () and LLMs (), forming 27,172 Interactions (human message with a set of model responses), and 68,371 Utterances (triples of {human message, model response, score}).

人类参与者					大语言模型
问卷参与者	1,500		对话		提供方总数	6
参与了对话	1,396 (93.1%)	$\Rightarrow$	对话总数	8,011	$\Leftarrow$	模型总数	21
出生国家	75		无引导	3,113 (38.9%)		商业 API	12 (57.1%)
居住国家	38		争议引导	2,438 (30.4%)		开放获取	9 (42.9%)
对话数 / 参与者	5.7$\pm$1.0		价值引导	2,460 (30.7%)		对话数 / 模型	1430.9$\pm$171.1
见到的模型数 / 参与者	13.9$\pm$2.5		轮次 / 对话	3.4$\pm$1.6		独特评分者 / 模型	924.3$\pm$94.4
			交互总数	27,172
			语句总数	68,371

图 1：Prism 数据集。 在第一阶段，1,500 名参与者填写问卷，说明自己的背景、对 LLM 的熟悉程度以及对行为的陈述偏好（第 2.1 节）。人口与地理分布见 Table 5 和 Table 8。参与者随后进入第二阶段，按自己选择的话题与 LLM 对话，用基数量表给回复打分，并给出细粒度反馈（第 2.2 节）。第一轮中，四个模型回应开场提示（; , , , ）。随后各轮，对话在非确定性温度下从评分最高的模型中采样两条回复继续（; ）。参与者（）与 LLM（）之间共有 8,011 场对话，形成 27,172 次交互（一条人类消息配上一组模型回复），以及 68,371 条语句（{人类消息, 模型回复, 分数} 三元组）。

We introduce Prism, a new resource for navigating empirical questions of human feedback. We employ both the ask and observe principles of social science by mapping detailed survey responses of humans around the world onto their live conversations with LLMs (Fig. 1). This setup permits alignment methods relying on either contextual preference comparisons typical for RLHF [29, 30, 31], or stated preferences and principles like constitutional AI [6, 32]. In addition to pairing stated and contextual preferences, Prism has the following features. Participatory: To ensure wider active participation in alignment data [25, 33], we recruit 1,500 English-speaking crowdworkers from diverse geographies and demographics; Representative: As units for preference aggregation, we include two census-representative samples (UK, US); and Individualised: To expose hidden human context and permit personalised preferences, each rating links to a pseudonymous ID and detailed participant profile. We source Subjective and Multicultural perspectives to avoid value-monism and cultural homogenisation in the opinions that LLMs represent [34, 35, 36] and operate in the descriptive paradigm without guidelines that characterise ‘good’ responses [15, 16]. Opinion diversity varies along the objective–subjective spectrum (e.g. what is the capital of France? vs. is abortion wrong?), so we prime participants for values and controversy guided dialogues but also collect neutral unguided dialogues as a baseline. To our knowledge, Prism is the first human feedback dataset to target cross-cultural controversies and value-laden prompts, where interpersonal disagreement is rife. After introducing Prism (Section 2), we demonstrate its value via three case studies (Section 3): (1) Do different people initiate different discussions with LLMs? (2) Do people prefer differently aligned models, and (3) How do sampling decisions affect welfare outcomes? Prism provides many more research avenues such as engineers targeting personalised alignment [2] or consensus across opinion distributions [5, 37]; social scientists examining how exposure to LLMs affects public attitudes; or policymakers seeking democratic input on AI-citizen interactions on topics like immigration, abortion or euthanasia. Alignment cannot be neatly bifurcated into technical and normative components [38]. Prism assists in navigating these complexities with more human voices adjudicating alignment norms.

我们推出 Prism，作为应对人类反馈经验问题的新资源。我们同时采用社会科学中的询问与观察原则，把世界各地人类的详细问卷回答映射到他们与 LLM 的实时对话上（图 1）。这一设置既支持依赖情境偏好比较的对齐方法（RLHF 中的典型做法 [29, 30, 31]），也支持像 constitutional AI 那样依赖陈述偏好与原则的方法 [6, 32]。除了把陈述偏好与情境偏好配对之外，Prism 还有如下特点。参与式： 为了让更广泛的人主动参与对齐数据 [25, 33]，我们招募了 1,500 名来自多样地理与人口背景的英语众包工作者；有代表性： 作为偏好汇总的单元，我们纳入两个普查代表性样本（英国、美国）；个体化： 为了揭示隐藏的人类语境并支持个性化偏好，每一条评分都连接到一个化名 ID 和详细的参与者档案。我们采集主观与多元文化视角，以避免 LLM 所代表的观点陷入价值一元论和文化同质化 [34, 35, 36]，并在描述性范式下运作，不提供刻画“好回复”的指南 [15, 16]。意见多样性沿客观–主观谱系变化（例如“法国的首都是哪里？”对比“堕胎是否错误？”），因此我们引导参与者进行价值和争议导向的对话，同时也收集中性的无引导对话作为基线。据我们所知，Prism 是首个专门针对跨文化争议和价值负载提示——也就是人际分歧最为密集之处——的人类反馈数据集。介绍完 Prism（第 2 节）之后，我们通过三项案例研究展示其价值（第 3 节）：(1) 不同的人会不会向 LLM 发起不同的讨论？ (2) 人们是否偏好对齐方式不同的模型，以及 (3) 抽样决策如何影响福利结果？ Prism 还提供许多其他研究路径，例如工程师追求个性化对齐 [2] 或跨越意见分布的共识 [5, 37]；社会科学家考察接触 LLM 如何影响公众态度；或政策制定者寻求关于移民、堕胎或安乐死等议题上 AI 与公民互动的民主输入。对齐无法干净地拆成技术部分与规范部分 [38]。Prism 帮助我们在这些复杂之处引入更多人类声音，来裁定对齐规范。

2 The Prism Alignment Dataset

2 Prism Alignment 数据集

Prism maps the characteristics and preferences of diverse humans onto their real-time interactions with LLMs (Fig. 1). Participants complete a Survey (Section 2.1) with questions about their demographics and stated preferences, then proceed to the Conversations with LLMs (Section 2.2), where they input prompts, rate responses and give fine-grained feedback in a series of multi-turn interactions. With the two-stage setup: (i) we avoid over-generalising from a “generic human” by matching ratings to detailed participant characteristics; (ii) we track how contextual preferences (in local conversations) depart from stated preferences (in survey); and (iii) we give participants autonomy to communicate in their own words what is important and why [39, 25]. Both stages received ethics board approval and ran with informed consent (Appendix D). Participants were paid £9/hour and the task took 70 minutes on average. Data collection ran from 22nd November to 22nd December 2023.[^2]We provide a data statement in Appendix B, data clause in Appendix C, and full codebooks detailing each variable in Appendix V.

Prism 把多样化人类的特征与偏好映射到他们与 LLM 的实时交互上（图 1）。参与者先完成一份关于人口统计与陈述偏好的问卷（第 2.1 节），再进入与 LLM 的对话（第 2.2 节）：他们输入提示、给回复打分，并在一系列多轮交互中给出细粒度反馈。借助这一两阶段设置：(i) 我们避免从“通用人类”过度推广，而是把评分与详细的参与者特征相匹配；(ii) 我们追踪情境偏好（在局部对话中）如何偏离陈述偏好（在问卷中）；(iii) 我们让参与者用自己的话说明什么重要、为什么重要 [39, 25]。两个阶段都获得了伦理委员会批准，并在知情同意下进行（附录 D）。参与者时薪为 9 英镑，任务平均耗时 70 分钟。数据收集从 2023 年 11 月 22 日持续到 12 月 22 日。[^2]我们在附录 B 提供数据声明，在附录 C 提供数据条款，并在附录 V 给出详述每个变量的完整编码手册。

2.1 The Survey

Prior to starting the survey, we ensure that all participants are over 18, obtain their informed consent, give a brief primer on LLMs (or AI language models), and dissuade LLM-written responses. The survey constructs a participant profile containing five features:

2.1 问卷

在开始问卷之前，我们确认所有参与者年满 18 岁，获得他们的知情同意，简要介绍 LLM（或 AI 语言模型），并劝阻由 LLM 代写的回答。问卷构造的参与者档案包含五个部分：

LLM familiarity and usage We ask about participants’ familiarity with LLMs (61% are somewhat familiar, 28% very familiar and 10% not familiar at all) and whether to their knowledge they have used them indirectly (in products like LinkedIn post-writing tool); or directly (via a specialised interface like ChatGPT). Individuals that have used LLMs directly or indirectly (84%) are branched to questions on frequency of use (7% every day, 21% every week, and 20% every month) and purpose of use (the most popular tasks are research overviews selected by 49%, professional work by 37%, creative writing by 31% and programming help by 27%). Full results in Appendix I.

对 LLM 的熟悉程度与使用情况 我们询问参与者对 LLM 的熟悉程度（61% 有些熟悉，28% 非常熟悉，10% 完全不熟悉），以及据他们所知是否间接使用过（例如在 LinkedIn 的发帖写作工具等产品中），或直接使用过（通过 ChatGPT 这类专门界面）。直接或间接使用过 LLM 的人（84%）会被引导到使用频率（7% 每天、21% 每周、20% 每月）和使用目的（最常见的任务是研究综述，49% 选择；专业工作 37%；创意写作 31%；编程帮助 27%）的问题上。完整结果见附录 I。

Self-written system string (“constitution”) System strings can guide LLM behaviours as a high-level global instruction prompts prepended to all subsequent interactions [40, 41], and have been analogised as “constitutions” or governing principles for AI [32]. Factuality, professionalism, humanness and harmlessness all emerged as key principles (Section M.1) from the following instruction:

自写系统字符串（“宪章”） 系统字符串可以作为前置到随后所有交互上的高层全局指令，用来引导 LLM 行为 [40, 41]，也被类比为 AI 的“宪章”或治理原则 [32]。事实性、专业性、类人性和无害性都作为关键原则出现（第 M.1 节），来自如下指令：

Figure 2: Schematic of fine-grained attribute ratings. The same attributes appear in three places in our task: A is asked once in the survey; B and C are asked per conversation. For performance attributes, we ask participants to consider only the highest-rated model in the first conversation turn; for choice attributes, we ask them to consider this highest-rated model relative to other models in the first turn.

图 2：细粒度属性评分示意图。 同一组属性在任务中出现三次：A 在问卷中只问一次；B 和 C 在每场对话中都会问。对表现属性，我们请参与者只考虑第一轮对话中评分最高的模型；对选择属性，我们请他们把这一最高分模型相对于第一轮中的其他模型来考虑。

Stated preferences for LLM behaviours In contrast to this open-ended preference elicitation, we collect structured ratings on fine-grained behaviour attributes. Participants score the importance of each attribute on a visual analog scale [42] (Fig. 2). A statement like “It is important that an AI language model produces factual and informative responses” maps (0,100) where the ends of scale are (Strongly disagree, Strongly agree). Numeric scores are recorded, but not shown to participants to avoid anchoring and dependency biases. We only collect responses to these statements once before participants interact with LLMs but the same attributes appear in the Conversations stage; so, we can track how stated ‘abstract’ preferences relate to contextual ‘in-situ’ preferences.[^3] Overall, we find clusters of subjective attributes (values, creativity and diversity) versus objective attributes (factuality, fluency and helpfulness; Section N.1). While the majority of participants agree that these more objective attributes are important (highly-skewed positive distribution, $\mu\in[86,89],\sigma\in[14,16]$), there is little agreement on the meta-importance of subjective attributes (Section N.2). In fact, responses for whether value alignment itself is important follow an almost normal distribution ($\mu=54,\sigma=26$).

对 LLM 行为的陈述偏好 与这种开放式的偏好诱发相对，我们收集关于细粒度行为属性的结构化评分。参与者在视觉模拟量表 [42] 上给每个属性的重要性打分（图 2）。例如“重要的是，一个 AI 语言模型能给出符合事实且信息丰富的回复”映射到 (0,100)，量表两端为（强烈不同意，强烈同意）。我们记录数字分数，但不向参与者显示，以避免锚定和依赖偏差。这些陈述只在参与者与 LLM 交互之前收集一次，但同一组属性也会出现在对话阶段；因此我们可以追踪陈述的“抽象”偏好如何关联到情境中的“当场”偏好。[^3] 总体来看，我们看到主观属性（价值、创造性与多样性）与客观属性（事实性、流畅性与有用性；第 N.1 节）分成两簇。大多数参与者同意这些更客观的属性很重要（高度右偏的正分布，$\mu\in[86,89],\sigma\in[14,16]$），但在主观属性的元重要性上几乎没有共识（第 N.2 节）。事实上，价值对齐本身是否重要的回答接近正态分布（$\mu=54,\sigma=26$）。

Self-written description Values and preferences are subjective and personal. We ascribe participants autonomy to communicate salient aspects of their identity in a short profile, beyond essentialising associations with structured demographics alone. Honesty, hard work and empathy emerged as common values (Section M.2) from the following instruction:

自写描述 价值与偏好是主观而个人的。我们赋予参与者自主权，让他们在一段短档案中说明自己身份中突出的方面，而不只是被结构化人口统计所本质化。诚实、勤奋和共情作为常见价值出现（第 M.2 节），来自如下指令：

Basic demographics We ask standard demographics: age, gender, employment status, martial status, educational attainment, ethnicity, religious affiliation, English proficiency, country of birth, and country of residence. There is always a “Prefer not to say” option. For gender, participants can select Male, Female, Non-Binary, or self-describe. We collect self-described ethnicity and religion because no pre-set groups exhaust how individuals may self-identify across cultures and global regions. We provide a manual annotation of these strings into aggregated categorisations for statistical analysis (Appendix F). Because of how we recruit participants (Section 2.3), our sample covers diverse demographics (Appendix G) and geographies (Appendix H), with representation from people born in 75 countries. However, the sample still skews White, Western and educated, and only contains English-language speakers.

基本人口统计 我们询问标准人口统计：年龄、性别、就业状况、婚姻状况、教育程度、族裔、宗教信仰、英语水平、出生国和居住国。始终提供“不愿透露”选项。对性别，参与者可以选择男性、女性、非二元，或自行描述。我们收集自行描述的族裔和宗教，因为没有任何预设分组能穷尽个体在不同文化和全球地区中的自我认同方式。我们对这些字符串做了人工标注，归入用于统计分析的汇总类别（附录 F）。由于招募方式（第 2.3 节），我们的样本覆盖多样的人口统计（附录 G）和地理分布（附录 H），包含出生于 75 个国家的人。不过样本仍偏向白人、西方和受过教育的人群，并且只包含英语使用者。

2.2 The Conversations

After completing the survey, participants move to the second stage, consisting of real-time conversations with LLMs via a custom-built interface on the Dynabench platform [43, 44].

2.2 对话

完成问卷后，参与者进入第二阶段，通过 Dynabench 平台 [43, 44] 上定制的界面与 LLM 进行实时对话。

Selecting conversation type We prime participants to diversify their prompts along the objective-subjective spectrum by asking them to complete two conversations across three conditions or conversation types (six in total).[^4] They select the type before inputting their opening prompt:

选择对话类型 为了让提示沿着客观–主观谱系多样化，我们请参与者在三种条件或对话类型下各完成两场对话（共六场）。[^4] 他们在输入开场提示之前先选择类型：

Opening the conversation Participants construct a free-text prompt of their choosing and receive up to four responses from different LLMs.[^5] The participants then rate each response on a visual analogue scale (VAS) [42, 45] from ‘‘Terrible’’ to ‘‘Perfect’’. We record the slider position as a score from 1--100 but do not show participants the number to avoid anchoring or conditional dependence of scores across conversations. We opt for this cardinal feedback for three reasons: (i) it encourages subjectivity; (ii) it permits studying the relative merit of cardinality versus ordinality for reward modelling because ratings can be converted to rankings but not vice versa; (iii) it allows expression of preference intensity above and beyond chosen:rejected pairs.[^6] However, we acknowledge that the cardinal scale introduces some intrapersonal measurement noise from a more cognitively demanding task and carries less interpersonal comparability than ordinal preferences, see Limitations (Section 5).

开启对话 参与者构造一条自己选择的自由文本提示，并收到最多四个不同 LLM 的回复。[^5] 然后他们在从“糟糕”到“完美”的视觉模拟量表（VAS）[42, 45] 上给每条回复打分。我们把滑块位置记录为 1–100 的分数，但不向参与者显示数字，以免跨对话出现锚定或条件依赖。我们选择这种基数反馈有三个原因：(i) 它鼓励主观性；(ii) 它允许研究基数相对序数对奖励建模的相对优劣，因为评分可以转成排序，反之则不行；(iii) 它允许表达偏好强度，而不只是 chosen:rejected 对。[^6] 不过我们承认，基数量表会因认知负担更高而引入一些个人内部的测量噪声，人际可比性也弱于序数偏好，见局限性（第 5 节）。

Continuing the conversation The highest-scoring LLM from the opening turn is locked into subsequent turns, with random tie-breaks in the case of identical scores. Participants must continue the conversation for at least another turn, but are asked to vary their conversations between 2 and 10 turns to avoid introducing a dataset artefact. We encourage some variation in conversation length ($\mu_{T}=3.4,\sigma_{T}=1.6$) but there is a strong drop off after the second turn (Appendix O). Participants then rate two responses on a VAS like before, but both are now sampled from the selected model with a non-deterministic temperature. These within-model responses are more similar in style and content than across-model responses (in the first turn), and score deviations are narrower (Appendix O).

继续对话 开场轮得分最高的 LLM 会被锁定到随后各轮；分数相同则随机打破平局。参与者必须至少再继续一轮，但我们请他们把对话长度变化控制在 2 到 10 轮之间，以免引入数据集伪迹。我们鼓励对话长度有一定变化（$\mu_{T}=3.4,\sigma_{T}=1.6$），但第二轮之后有明显下降（附录 O）。参与者随后像之前一样在 VAS 上给两条回复打分，但这两条现在都来自被选中的模型，并使用非确定性温度采样。这些模型内回复在风格和内容上比（第一轮的）跨模型回复更相近，分数偏差也更窄（附录 O）。

Collecting fine-grained feedback After the conversation ends, participants first rate statements about the performance of their highest-rated model like “The response was well-written” on a VAS from Performed very poorly to Performed very well, or select N/A if the statement is irrelevant for the context. We then ask participants to consider why they chose this model, rating statements like “I chose this response because it was well-written” on a VAS from Very unimportant to Very important (or select N/A). Attributes are shared with the Survey (Fig. 2). We find strong correlations between performance attributes and choice attributes (except safety) but weak correlations of these pairs to stated preferences given in the Survey, perhaps due to conversational, model or task-design confounders (Section N.1). In general, the distribution of scores over performance and choice attributes is narrower and more positively skewed (bunched to 100) compared to stated preferences (Section N.2). Finally, we collect open-ended natural language feedback on the whole conversation. Participants contributed both content and stylistic feedback ($\mu=29$ words, $\sigma=19$, Section M.3).

收集细粒度反馈 对话结束后，参与者先就其最高分模型的表现给诸如“回复写得很好”这类陈述打分，VAS 从表现非常差到表现非常好，若该陈述与语境无关可选 N/A。然后我们请参与者思考自己为什么选择这个模型，给诸如“我选择这条回复是因为它写得很好”的陈述打分，VAS 从非常不重要到非常重要（或选 N/A）。属性与问卷共享（图 2）。我们发现表现属性与选择属性之间相关性很强（安全除外），但这些配对与问卷中的陈述偏好相关性较弱，这可能来自对话、模型或任务设计的混杂（第 N.1 节）。总体来看，表现和选择属性上的分数分布比陈述偏好更窄、更向正向偏斜（聚集在 100 附近）（第 N.2 节）。最后，我们收集关于整场对话的开放式自然语言反馈。参与者同时贡献了内容和风格方面的反馈（$\mu=29$ 词，$\sigma=19$，第 M.3 节）。

2.3 The Sample

Our sampling aims were depth in the demographics represented within countries and breadth across global regions. We recruit English-speaking participants from Prolific in two distinct paths:

2.3 样本

我们的抽样目标是：在各国内部获得所代表人口统计上的深度，并在全球地区之间获得广度。我们通过两条不同路径从 Prolific 招募英语参与者：

Census-representative sample (UK, US) Samples matched to simplified census data (age, ethnicity, gender) were only available for the UK and US. The minimum pool size for a statistical guarantee of representativeness was 300, which set a lower bound for participant quota. After collecting data, we observed some skew in our ‘representative’ samples between observed and expected distributions in recent census data, which we partially correct for (Appendix L). These samples permit future studies on more representative populations that can be replicated across two countries; however their inclusion biases Prism as a whole towards two Western nations already over-represented in AI research.

普查代表性样本（英国、美国） 与简化普查数据（年龄、族裔、性别）匹配的样本只对英国和美国可用。为保证代表性统计效力的最小池规模是 300，这设定了参与者配额的下限。收集数据后，我们观察到这些“代表性”样本在近期普查数据的观察分布与期望分布之间存在一些偏斜，我们对此做了部分校正（附录 L）。这些样本允许未来在两个国家之间复制、针对更具代表性人群的研究；不过把它们纳入也会让 Prism 整体偏向两个在 AI 研究中本已过度代表的西方国家。

Balanced samples (rest of world) The distribution of Prolific workers outside the US and the UK skews strongly to Europe and Northern America, and some countries dominate continental counts (Appendix J). To avoid more active workforces biasing the sample, we set up 33 country-specific studies where there is $>1$ eligible worker, and allocate sample quotas so that each global region is similarly represented.[^7] We balance each national sample by gender where possible (Table 10).

平衡样本（世界其余地区） 美国和英国以外的 Prolific 工作者分布强烈偏向欧洲和北美洲，一些国家主导了所在大洲的计数（附录 J）。为避免更活跃的劳动力使样本偏向，我们在有超过 1 名合格工作者的地方设立了 33 项国别研究，并分配样本配额，使各全球地区的代表程度相近。[^7] 在可能的情况下，我们按性别平衡每个国家样本（Table 10）。

Included models The rapidly evolving landscape necessitates a model-agnostic approach to avoid data staleness. We include 21 different LLMs (9 open-access, 12 commercial-API) from various model families and parameter sizes, which diversifies the training data, capabilities, and degree of existing safeguards or alignment biases. To avoid text length confounding preferences [46] and to reduce participant fatigue, we include system prompts instructing models to limit their responses to $\leq 50$ words. We show the full list of models, decoding parameters and generation details in Appendix P.

纳入的模型 快速变化的格局要求一种与具体模型无关的做法，以免数据迅速过时。我们纳入来自不同模型家族和参数规模的 21 个 LLM（9 个开放获取，12 个商业 API），从而多样化训练数据、能力，以及已有护栏或对齐偏差的程度。为避免文本长度混淆偏好 [46]，并减轻参与者疲劳，我们加入系统提示，要求模型把回复限制在 $\leq 50$ 词。完整的模型列表、解码参数和生成细节见附录 P。

Figure 3: Topic prevalence by conversation types and participant identity. We show total prompts clustered into topics (bars), and total members in each group (top panels). Per group and topic, we plot the over-representation factor of observed vs. expected group proportions and show significant regression coefficients (base category indicated by $\dagger$). All coefficients are in Fig. 23, topic-group counts in Fig. 27 and centroid prompts in Table 21. Location is by birth region (with UK and US split out), but most regions have few countries (Appendix H). Key results (Section 3.1): Priming participants to select a conversation type (unguided, values or controversy guided) significantly influenced diversity of prompts. Identity factors have some significant interactions with prompt choice but each topic contains prompts authored by intersectionally-diverse participants.

图 3：按对话类型与参与者身份看话题流行度。 我们展示聚成各话题的提示总数（柱），以及每组的成员总数（顶部面板）。对每个组别和话题，我们画出观察比例相对期望比例的过度代表因子，并标出显著的回归系数（基准类别用 $\dagger$ 标出）。全部系数见图 23，话题–组别计数见图 27，质心提示见 Table 21。地点按出生地区划分（英国和美国单独列出），但多数地区只包含少数国家（附录 H）。主要结果（第 3.1 节）： 引导参与者选择对话类型（无引导、价值引导或争议引导）显著影响了提示的多样性。身份因素与提示选择有一些显著交互，但每个话题都包含由交叉多样的参与者撰写的提示。

3 Experiments with Prism

3 用 Prism 做实验

3.1 Case Study I: Do Different People Initiate Different Discussions with LLMs?

Methods We use a pre-trained sentence transformer (all-mpnet-base-v2) to embed each opening prompt in 768-D, then apply UMAP to reduce to 20-D, before clustering with HDBScan [47]. 70% of prompts are assigned to 22 topic clusters and 30% remain as outliers. We name each cluster by prompting gpt-4-turbo with the top n-grams extracted with TF-IDF and closest texts to the cluster centroid. We define an over-representation factor as $\frac{N_{g,t}/N_{t}}{b_{g}}$, to compute observed versus expected topic prevalence per identity group. For the partial contribution of identity attributes, we estimate an OLS regression for each topic $y^{t}$ ($t\in 1\ldots 22$) and cluster standard errors at the individual level: $y_{i,c}^{j}=\alpha^{t}+\text{gender}_{i}^{\prime}\beta_{1}^{t}+\text{age}_{i}^{\prime}\beta_{2}^{t}+\text{birth\_region}_{i}^{\prime}\beta_{3}^{t}+\text{ethnicity}_{i}^{\prime}\beta_{4}^{t}+\text{religion}_{i}^{\prime}\beta_{5}^{t}+\text{prompt}_{i}^{\prime}\beta_{6}^{t}+\varepsilon_{i,c}$, where $y^{t}_{i,c}=1$ if the prompt of participant $i$ in conversation $c$ is categorised as topic $t$. The identity vectors (e.g. gender) represent sets of variables, with a base category removed (indicated in Fig. 3). The coefficients of interest are contained in vectors $\{\beta_{d}^{t}\}_{d=1}^{6}$, where component $g$ of $\beta_{d}^{t}$ is interpreted as the increase in probability of a participant choosing topic $t$ if they are in the group indexed by $g$ (e.g. Female) compared to the base group (e.g. Male). See Appendix R for extended methods.

3.1 案例研究 I：不同的人会不会向 LLM 发起不同的讨论？

方法 我们用预训练的句向量模型（all-mpnet-base-v2）把每条开场提示嵌入到 768 维，再用 UMAP 降到 20 维，随后用 HDBScan 聚类 [47]。70% 的提示被分到 22 个话题簇，30% 仍是离群点。我们用 TF-IDF 提取的高频 n-gram 以及最接近簇质心的文本去提示 gpt-4-turbo，为每个簇命名。我们把过度代表因子定义为 $\frac{N_{g,t}/N_{t}}{b_{g}}$，用来计算每个身份组别上观察话题流行度相对期望值的比值。为估计身份属性的偏贡献，我们对每个话题 $y^{t}$（$t\in 1\ldots 22$）估计一个 OLS 回归，并在个体层面聚类标准误：$y_{i,c}^{j}=\alpha^{t}+\text{gender}_{i}^{\prime}\beta_{1}^{t}+\text{age}_{i}^{\prime}\beta_{2}^{t}+\text{birth\_region}_{i}^{\prime}\beta_{3}^{t}+\text{ethnicity}_{i}^{\prime}\beta_{4}^{t}+\text{religion}_{i}^{\prime}\beta_{5}^{t}+\text{prompt}_{i}^{\prime}\beta_{6}^{t}+\varepsilon_{i,c}$，其中若参与者 $i$ 在对话 $c$ 中的提示被归为话题 $t$，则 $y^{t}_{i,c}=1$。身份向量（例如 gender）表示一组变量，并去掉一个基准类别（见图 3）。我们关心的系数包含在向量 $\{\beta_{d}^{t}\}_{d=1}^{6}$ 中，其中 $\beta_{d}^{t}$ 的分量 $g$ 解释为：参与者若属于由 $g$ 索引的组别（例如女性），相对于基准组别（例如男性），选择话题 $t$ 的概率增量。扩展方法见附录 R。

Results Our instructions had a significant priming effect, resulting in a high density of controversial and value-laden topics (Fig. 3). Topics significantly correlated with controversy guidance are Gender & LGBTQ+ Identity, Israel–Palestine Conflict, and Discussions on Abortion, while topics significantly correlated with the values guidance are Managing Relationships, Job Search, and Religion & Spirituality. In contrast, the ‘unguided’ condition correlates with task-oriented and neutral topics like Popular Culture, Recipes & Cooking and Travel Recommendations. Only Climate Change is not significantly correlated to conversation type. Controlling for conversation type, 11% of coefficients are significant ($\alpha=99\%$); so, identity factors have some predictive power on topic prevalence. Significant relationships include: women and non-binary people discuss gender and LGBTQ+ issues more than men; older people discuss elections and travel more than younger people; Black participants discuss climate change less than White participants, and all regions question LLMs about abortion less often than US participants. When we examine granular regions in embedding space using a single-link hierarchical clustering algorithm (Appendix S), local prompt neighbourhoods tend to be intersectionally-diverse: 84% of them meet or exceed entropy across intersectional demographics that would be expected under random sampling. During this local exploration, we retrieve regions of semantically-identical prompts rated by multiple diverse individuals (e.g. one neighbourhood “Does God exist?” has 7 religious and 7 irreligious participants), finding that interpersonal differences in contextual preferences persist even when dialogue context is fixed (Section S.4). So, despite Prism containing semantically-diverse prompts, people from different backgrounds occupy common discussion spaces, providing an anchor to examine diverse perspectives to shared issues.

结果 我们的说明有显著的启动效应，带来了高密度的争议性和价值负载话题（图 3）。与争议引导显著相关的话题是性别与 LGBTQ+ 身份、以色列–巴勒斯坦冲突和关于堕胎的讨论；与价值引导显著相关的话题是处理人际关系、求职和宗教与灵性。相反，“无引导”条件与面向任务、较为中性的话题相关，例如流行文化、食谱与烹饪和旅行推荐。只有气候变化与对话类型没有显著相关。在控制对话类型之后，11% 的系数显著（$\alpha=99\%$）；因此，身份因素对话题流行度有一定预测力。显著关系包括：女性和非二元人群比男性更多讨论性别和 LGBTQ+ 议题；年长者比年轻人更多讨论选举和旅行；黑人参与者比白人参与者更少讨论气候变化；所有地区向 LLM 询问堕胎的频率都低于美国参与者。当我们用单链接层次聚类算法考察嵌入空间中的细粒度区域时（附录 S），局部提示邻域往往在交叉身份上是多样的：其中 84% 达到或超过随机抽样下交叉人口统计所期望的熵。在这种局部探索中，我们找回了一些由多名多样化个体评分、语义上相同的提示区域（例如一个“上帝存在吗？”邻域中有 7 名有宗教信仰和 7 名无宗教信仰的参与者），并发现即便对话语境固定，情境偏好上的人际差异仍然存在（第 S.4 节）。因此，尽管 Prism 包含语义多样的提示，来自不同背景的人仍占据共同的讨论空间，这为考察人们对共同议题的多样视角提供了锚点。

3.2 Case Study II: Do Different People Prefer Differently-Aligned Models?

Methods Observed preference differences at the model-level are confounded by interactions of topic prevalence and model aptitude (e.g. men ask more about aliens and gpt-4 is poor on extraterrestrial knowledge). Evidence of shared dialogue spaces (Section 3.1) and group-topic score differences (Section T.2) mitigate some concern, but to further control for context, we use opening prompts from the balanced subset of participants (n=1,246) with equal conversations per type (n=6,669). The mean participant rates 14/21 LLMs but unseen ratings are missing at random. Our aggregation (social choice) function over participant ratings is derived from Pairwise Rank Centrality ($\mathcal{P}$) [48] and Convergence Voting [49], both inspired by PageRank [50]. Each model is a node in a graph and transition probabilities between nodes are calculated by the proportion of pairwise battle wins. This process simulates a random walk on a Markov chain, leading to a stationary distribution of scores that reflect the collective preference intensity across models. Here, we compute $\mathcal{P}$ over subsamples using a regularisation parameter of 1 and tie threshold of 5, but present extended methods and robustness checks in Appendix T.

3.2 案例研究 II：不同的人是否偏好对齐方式不同的模型？

方法 在模型层面观察到的偏好差异，会被话题流行度与模型能力的交互所混淆（例如男性更多询问外星人，而 gpt-4 在地外知识上较差）。共享对话空间的证据（第 3.1 节）以及组别–话题分数差异（第 T.2 节）缓解了部分担忧，但为进一步控制语境，我们使用平衡子集参与者（n=1,246）的开场提示，每人每种类型的对话数相同（n=6,669）。平均每位参与者给 14/21 个 LLM 打分，未见过的评分是随机缺失的。我们对参与者评分的汇总（社会选择）函数来自 Pairwise Rank Centrality（$\mathcal{P}$）[48] 和 Convergence Voting [49]，两者都受 PageRank [50] 启发。每个模型是图中的一个节点，节点之间的转移概率由成对对战的胜率计算。这一过程模拟马尔可夫链上的随机游走，得到反映各模型上集体偏好强度的平稳分布分数。这里我们对子样本计算 $\mathcal{P}$，正则化参数取 1，平局阈值取 5，扩展方法和稳健性检查见附录 T。

Results We find rankings are sensitive to idiosyncratic, contextual, and group-wise variance. Samples of 100 people introduce significant noise, resulting in a fairly even distribution of collective preference among the top 10 models (Fig. 4). Rankings are sensitive to what participants talk about: zephyr-7b performs highly on controversy but not in unguided domains, while claude-2 has the opposite trend; and where they are from: relative to overall rank, palm-2 drops 4 places for participants in the US, llama-7b drops 7 places in Asia, while mistral-7b gains 7 places in Africa. We further observe that Prism produces surprising ranks relative to other leaderboards. We apply our method to ChatbotArena data [51], finding gpt models fare significantly worse in Prism, while open models like zephyr-7b do significantly better (95% CI over 1,000 bootstraps, Section T.9). This may be due to domain shift (task-orientated/coding prompts vs. controversial/cultural prompts), sample diversity or task incentives. To identify drivers of score differences, we generate hypotheses by qualitatively examining battles between command and gpt-4/-turbo, then test these with an OLS regression on all model responses (Section T.8). We find that formatting and refusals partially explain score differences with significant positive effects from additional characters, ending in a question mark (“Would you like to know more?”) and enumeration, but significant negative effect of line breaks. De-anthropomorphic phrases (“As an AI, I don’t have personal opinions.”) significantly reduce score but not as substantially as refusals (“Sorry I cannot engage.”). The proportion of explained variance in score by these factors is low ($R^{2}=0.06$), so we encourage more sophisticated methods in future work for partialling out the effect of style versus content, or participant, model and conversation fixed-effects, as determinants of score.

结果 我们发现排名对特异、情境和组别差异都很敏感。100 人的样本会引入显著噪声，使前 10 个模型的集体偏好分布相当均匀（图 4）。排名对参与者谈论什么敏感：zephyr-7b 在争议领域表现很高，但在无引导领域并非如此，而 claude-2 趋势相反；也对参与者来自哪里敏感：相对总体排名，palm-2 在美国参与者中下降 4 位，llama-7b 在亚洲下降 7 位，而 mistral-7b 在非洲上升 7 位。我们还观察到，相对其他排行榜，Prism 产生了出人意料的名次。我们把同一方法用到 ChatbotArena 数据 [51] 上，发现 gpt 系列模型在 Prism 中明显更差，而 zephyr-7b 这类开放模型明显更好（1,000 次自助法的 95% 置信区间，第 T.9 节）。这可能来自领域偏移（面向任务/编程的提示 vs. 争议/文化提示）、样本多样性或任务激励。为识别分数差异的驱动因素，我们通过定性考察 command 与 gpt-4/-turbo 之间的对战来生成假设，再用针对所有模型回复的 OLS 回归加以检验（第 T.8 节）。我们发现格式和拒答部分解释了分数差异：额外字符、以问号结尾（“Would you like to know more?”）以及枚举有显著正向效应，但换行有显著负向效应。去拟人化短语（“As an AI, I don’t have personal opinions.”）会显著降低分数，但幅度不如拒答（“Sorry I cannot engage.”）。这些因素解释的分数方差比例很低（$R^{2}=0.06$），因此我们鼓励未来工作用更精细的方法，把风格与内容的效应，或参与者、模型和对话的固定效应，作为分数的决定因素分离出来。

Figure 4: Sources of variation in model preferences. Panel A shows idiosyncratic variance in distributions of Pairwise Rank Centrality scores for 100 randomly-drawn participants (over 1,000 bootstraps). For Panels B and C, we show conversational context variation and group-wise variation. We show overall rank based on Pairwise Rank Centrality over n=6,669 balanced conversations (numbered circles). We then trace how rank changes by sampling the group on $x$ (e.g. filtering to only values guided conversations, or only US participants). Across these subsamples, we show most spots climbed () and spots fallen () by each model relative to overall rank. Key results (Section 3.2): Rankings are sensitive to sample composition, varying with which participants are sampled (Panel A,C) and what they talk about (B). Rankings differ from other leaderboards, explained by Prism’s characteristics (sample diversity, domain shifts) as well as response characteristics (length, formatting, refusals).

图 4：模型偏好差异的来源。 面板 A 展示 100 名随机抽取的参与者（经 1,000 次自助法）在 Pairwise Rank Centrality 分数分布上的特异差异。对面板 B 和 C，我们展示对话语境差异和组别差异。我们给出基于 n=6,669 场平衡对话的 Pairwise Rank Centrality 总体排名（带编号的圆圈）。然后追踪排名如何随在 $x$ 上抽样该组而变化（例如只筛选价值引导对话，或只筛选美国参与者）。在这些子样本中，我们展示每个模型相对总体排名上升最多（）和下降最多（）的名次。主要结果（第 3.2 节）： 排名对样本构成敏感，随抽到哪些参与者（面板 A、C）以及他们谈论什么（B）而变化。排名与其他排行榜不同，可以由 Prism 的特征（样本多样性、领域偏移）以及回复特征（长度、格式、拒答）来解释。

3.3 Case Study III: How do Sampling Decisions Affect Welfare Outcomes?

Methods We use ‘welfare’ to capture the extent to which a chosen LLM aligns with the preferences of a user population. We consider two welfare measures: average model rating (meanRating), and average likelihood that a model is chosen (rated highest in the opening turn, meanChoice). Previous experiments indicate dialogue and preference diversity across people, suggesting that the welfare of downstream LLM users may depend on who provides feedback. To test this, we first randomly generate seven sub-samples of individuals ‘in the seat of power’ to select their favourite LLM (based on mean rating). Four sampling schemes randomly draw $N$ individuals from a representative sample ($N\in\{10,20,50,100\}$). Three schemes randomly draw 100 individuals from specific low-diversity sub-populations (male, white, and $\geq$45 years old). For each draw, we then measure the distribution of welfare from this LLM being imposed on different stakeholder populations [9]: the entire population, non-male individuals, non-white individuals, and individuals $<$45 years old. We report the distribution of average welfare outcomes across random draws from each sampling scheme. We conduct this experiment for the UK and US representative samples. Extended methods are in Appendix U.

3.3 案例研究 III：抽样决策如何影响福利结果？

方法 我们用“福利”来刻画所选 LLM 与用户群体偏好的对齐程度。我们考虑两种福利度量：平均模型评分（meanRating），以及某模型被选中的平均可能性（在开场轮评分最高，meanChoice）。此前实验表明，人们在对话和偏好上存在多样性，这意味着下游 LLM 用户的福利可能取决于谁提供反馈。为检验这一点，我们首先随机生成七个子样本，让其中的个体“坐在权力座位上”按其平均评分选出最喜欢的 LLM。四种抽样方案从代表性样本中随机抽取 $N$ 个个体（$N\in\{10,20,50,100\}$）。三种方案从特定的低多样性子群体（男性、白人，以及 45 岁及以上）中随机抽取 100 个个体。对每一次抽取，我们再衡量把这一 LLM 施加到不同利益相关群体 [9] 上的福利分布：全体人口、非男性个体、非白人个体，以及 45 岁以下个体。我们报告每种抽样方案下、多次随机抽取的平均福利结果分布。我们对英国和美国的代表性样本都做了这一实验。扩展方法见附录 U。

Figure 5: Welfare distributions for the US. The distribution of mean welfare for four subpopulations in the US (welfare pop) induced by seven sampling schemes (in the seat of power). The $y$ axis is the sampled subpopulation (e.g. Rep is a ‘representative’ sample of the population) and sample size in brackets (e.g (100)). Each violin shows the distribution of mean welfare for the panel’s subpopulation induced by a sampling scheme. The top four Rating comparisons use the meanRating welfare measure and the bottom Choice comparisons use the meanChoice measure. The red distributions are FOSD by Rep (100) in blue (i.e. less optimal scheme). Key results (Section 3.3): Large representative samples mostly outperform smaller or demographically-restricted samples and sampling exclusively from a specific group tends to reduce the welfare of out-group participants (male vs. non-male, white vs. non-white). No single model achieves majority preference (max 45% meanChoice).

图 5：美国样本的福利分布。 七种抽样方案（坐在权力座位上）给美国四个子群体（福利群体）带来的平均福利分布。$y$ 轴是被抽样的子群体（例如 Rep 是人口的“代表性”样本）以及括号中的样本量（例如 (100)）。每把小提琴展示某种抽样方案给该面板子群体带来的平均福利分布。上边四组 Rating 比较使用 meanRating 福利度量，下边的 Choice 比较使用 meanChoice 度量。红色分布被蓝色的 Rep (100) 一阶随机占优（FOSD）（也就是较差的方案）。主要结果（第 3.3 节）： 较大的代表性样本大多优于较小或人口受限的样本；只从特定群体抽样往往会降低外群体参与者的福利（男性 vs. 非男性，白人 vs. 非白人）。没有单一模型能获得多数偏好（最高 45% 的 meanChoice）。

Results We find as sample size falls, the probability of choosing a LLM with worse mean welfare rises. Larger samples from the target sub-population appear to first order stochastically dominate[^8] (FOSD) smaller samples from the target sub-population. Sampling exclusively from a specific group tends to reduce the welfare of out-group individuals. For example, when consider the welfare of the representative US sample (Fig. 5), sampling from US males is FOSD by sampling from the full US sample. Furthermore, average measures can conceal the welfare of minority groups: sampling 100 white individuals appears to FOSD sampling 100 representative individuals when assessing welfare of the population at large, but minority stakeholders (non-white population) are worse off under this scheme. Finally, regardless of the model chosen, a large proportion of participants prefer a different model. For the US, the model that maximises meanChoice only achieves a probability of $45\%$. If a participant is shown the winning model, and three other models at random, the probability that they will choose the winning model is $<50\%$. The probability they will pick the winning model over all other 20 LLMs can only be lower. This suggests that we should not expect a single LLM to satisfy everyone’s preferences in a given population. We repeat the welfare analysis for the UK sample and conduct robustness checks with imputed missing data in Appendix U.

结果 我们发现随着样本量下降，选出平均福利更差的 LLM 的概率会上升。来自目标子群体的较大样本，似乎对来自同一目标子群体的较小样本构成一阶随机占优[^8]（FOSD）。只从特定群体抽样往往会降低外群体个体的福利。例如，在考虑美国代表性样本的福利时（图 5），从美国男性中抽样会被从全体美国样本中抽样所 FOSD。此外，平均度量可能掩盖少数群体的福利：在评估全体人口的福利时，抽取 100 名白人个体似乎对抽取 100 名代表性个体构成 FOSD，但少数利益相关者（非白人群体）在这一方案下处境更差。最后，无论选出哪个模型，都有很大一部分参与者更偏好另一个模型。对美国而言，最大化 meanChoice 的模型也只达到 $45\%$ 的概率。如果向一名参与者展示胜出模型以及另外三个随机模型，他们选出胜出模型的概率仍低于 $50\%$。他们在全部其余 20 个 LLM 中选出胜出模型的概率只会更低。这说明我们不应期望单一 LLM 能满足给定群体中每个人的偏好。我们在附录 U 中对英国样本重复了福利分析，并用填补缺失数据做了稳健性检查。

4 Related Work

4 相关工作

Participation & Representation in Science & Technology There is a long history of technologies failing diverse users who lack consultation during design [52, 53, 54]. Conscious participation can be intrinsically valuable as an act of justice [55, 56]. However, in internet-harvested pre-training data, participation is involuntary or cooptative [55, 33], and unequal representation risks cultural homogenisation and minority stereotyping [57, 58, 59, 60, 61, 62]. Labelling data or giving feedback is active procedural participation [53] but often relies on narrow specifications from technology providers of what counts as high-quality language or preferable outputs [15, 16, 63, 64]. In ML or NLP data, variability in subjective experience is commonly collapsed into majority votes [27, 65, 66, 67, 68], without sufficient documentation of annotator artefacts or disagreements [69, 70, 71, 72, 73], despite evidence that sociodemographics affect labels [74, 75, 76, 77, 78, 79]. Multiple scientific fields are guilty of over-generalising conclusions from the ‘generic human’ drawn from ‘WEIRD’ societies [80, 81]. Prism releases participant IDs and characteristics to spotlight sample diversity while acknowledging sample specificity [82].

科学技术中的参与与代表 技术未能服务在设计阶段缺少咨询的多样化用户，有着漫长的历史 [52, 53, 54]。有意识的参与本身可以作为正义行动而具有内在价值 [55, 56]。不过，在从互联网采集的预训练数据中，参与是非自愿或被收编的 [55, 33]，而不平等的代表会带来文化同质化和少数群体刻板印象的风险 [57, 58, 59, 60, 61, 62]。标注数据或给出反馈是主动的程序性参与 [53]，但往往依赖于技术提供方对何谓高质量语言或更可取输出的狭窄规定 [15, 16, 63, 64]。在机器学习或自然语言处理数据中，主观经验的变异通常被压成多数票 [27, 65, 66, 67, 68]，却没有充分记录标注者伪迹或分歧 [69, 70, 71, 72, 73]，尽管已有证据表明社会人口统计会影响标签 [74, 75, 76, 77, 78, 79]。多个科学领域都犯过从“WEIRD”社会中抽取的“通用人类”过度推广结论的错误 [80, 81]。Prism 发布参与者 ID 和特征，既凸显样本多样性，也承认样本的特殊性 [82]。

Learning from Human Feedback Using human feedback to condition the loss function for training LLMs overcomes challenges of specifying rewards [83, 84, 85]. Combining human feedback, reinforcement learning and natural language generation has a history in machine translation [86, 87, 88] and dialogue [89, 90, 91, 92, 93, 94]. RLHF pipelines rely on binary comparisons [29, 30, 31, 85], principles or rules [32, 95], fine-grained feedback [12], or natural language [96], to reward dimensions like helpfulness, honesty and harmlessness [97, 30]. Reward models then update LLMs via algorithms like PPO [98] or Reinforce [99, 100]; but reward model free techniques are competitive, e.g. DPO [18], supervised fine-tuning [101] and rejection sampling [102, 5, 103]. There is rising demand for high-quality human feedback [104, 105], but the complexity and cost of collecting data incentivises scraping preferences, e.g. on Reddit [29, 106] or StackOverflow [107], or simulating humans with LLMs [108, 109, 110]. Similar to Prism, ChatbotArena [51], Lmsys-1m [111] and WildChat [112] feature user-rated model interactions, but for narrow communities (HuggingFace Spaces) and domains (coding, task-orientated). Unlike these datasets, OpenConvos [113] collect optional contributor demographics, and Dices [79] provide demographics for multiple raters per conversation. Other datasets target specific behaviours [30, 114], or multilingual coverage [115]. Surveys on attitudes towards AI [116, 117] and community assemblies [6, 118, 119] offer another lens on public priorities. To our knowledge, Prism is the first to link preference ratings and detailed survey responses.

从人类反馈中学习 用人类反馈来约束训练 LLM 的损失函数，克服了指定奖励的困难 [83, 84, 85]。把人类反馈、强化学习和自然语言生成结合起来，在机器翻译 [86, 87, 88] 和对话 [89, 90, 91, 92, 93, 94] 中已有历史。RLHF 管线依赖二元比较 [29, 30, 31, 85]、原则或规则 [32, 95]、细粒度反馈 [12] 或自然语言 [96]，以奖励有用性、诚实性和无害性等维度 [97, 30]。奖励模型随后通过 PPO [98] 或 Reinforce [99, 100] 这类算法更新 LLM；但无需奖励模型的技术也很有竞争力，例如 DPO [18]、监督微调 [101] 和拒绝采样 [102, 5, 103]。对高质量人类反馈的需求正在上升 [104, 105]，但收集数据的复杂性和成本促使人们去抓取偏好，例如在 Reddit [29, 106] 或 StackOverflow [107] 上，或用 LLM 模拟人类 [108, 109, 110]。与 Prism 类似，ChatbotArena [51]、Lmsys-1m [111] 和 WildChat [112] 也包含用户评分的模型交互，但面向狭窄社区（HuggingFace Spaces）和领域（编程、面向任务）。与这些数据集不同，OpenConvos [113] 收集可选的贡献者人口统计，Dices [79] 为每场对话的多名评分者提供人口统计。其他数据集针对特定行为 [30, 114]，或多语覆盖 [115]。关于对 AI 态度的调查 [116, 117] 以及社区集会 [6, 118, 119] 提供了观察公众优先事项的另一视角。据我们所知，Prism 是首个把偏好评分与详细问卷回答连接起来的数据集。

5 Limitations, Discussions and Conclusions

5 局限性、讨论与结论

Ethical Considerations and Limitations We collect informed consent, pseudononymise IDs, check for PII (Appendix E) and disallow deanonymisation in our terms (Appendix C), but privacy risks remain, especially given the sensitive nature of conversations. Asking participants to engage with controversies expands human preference data to discursive areas with the greatest expected degree of interpersonal disagreement, but risks encouraging hateful, bigoted, biased or otherwise harmful content. Prism is less toxic than previous datasets (0.06%, Appendix E). We do not moderate prior to release to permit conversational safety research. There are many sources of variance in Prism and alternative divisions of the data may yield different outcomes [120]. Granting free choice of dialogue, using cardinal feedback scales and focusing on many kinds of models and participants introduces diversity and subjective freedom but complicates controlled experiments and limits statistical power. Prism is still biased towards English-speaking crowdworkers whose task-specific incentives may not align with wider populations. We expand on ethical risks and limitations in our data statement (Appendix B).

伦理考虑与局限性 我们收集知情同意，对 ID 做假名化处理，检查个人身份信息（附录 E），并在条款中禁止去匿名化（附录 C），但隐私风险仍然存在，尤其考虑到对话内容的敏感性质。请参与者接触争议，能把人类偏好数据扩展到预期人际分歧最大的话语领域，但也有鼓励仇恨、偏见、偏颇或其他有害内容的风险。Prism 的毒性低于此前数据集（0.06%，附录 E）。我们在发布前不做审核，以便支持对话安全研究。Prism 中有许多差异来源，对数据的其他划分可能产生不同结果 [120]。给予对话的自由选择、使用基数反馈量表，并关注多种模型和参与者，会引入多样性和主观自由，但也使受控实验更复杂，并限制统计效力。Prism 仍然偏向英语众包工作者，他们的任务特定激励未必与更广泛人群一致。我们在数据声明（附录 B）中进一步讨论伦理风险和局限性。

We raise three discussion points on the boundaries of where we collect preferences, for what end and with what lasting impact. First, aligning LLMs via ‘preference-based utilitarianism’ [121] may not be synonymous with individual or societal well-being, prompting the question of whether there are limits for “legitimate” human feedback. Preferences may be (i) at odds with self-interest due to myopia or information asymmetries (e.g. participants who want anthropomorphic LLMs despite evidenced harms [122, 123, 124, 125, 126]) or (ii) incompatible with others’ interest (e.g. participants who prefer ‘anti-woke’ LLMs that argue in a debate vs. those who favour neutrality). Relying on decontextualized preference observations carries the risk of silently reinforcing biases from those in power [61, 65]; so we recommend transparency surrounding individual disagreements before aggregation decisions [9, 127], especially if participant positionality affects their epistemic legitimacy to define harm [59, 128, 129]. Second, irreconcilable personal preferences and morals matter more when the ‘unit of alignment’ is operationalised as a group, culture or even species, rather than an individual. Prism permits personalised or steerable alignment using participant profiles and specific ratings [2, 3, 4, 37] as well as collective alignment via opinion consensus or distribution of rewards [5, 6, 7, 8, 28]; though group deliberation in groups may yield different outcomes than gathering data from one person at a time [6, 118, 119]. With growing use of synthetic alignment data, Prism can assist in calibrating LLM-as-judge protocols to more diverse rater pools [51, 130]. Finally, Prism was motivated by participation as justice via inclusionary alignment practices that, relative to passive roles in annotation tasks or pre-training data, prioritise active input from local citizens with specialised knowledge of their own and communities’ needs [55]. However, participation remains thin because the humans crucial to the success of RLHF do not typically share in downstream benefits or profits [33, 131]. Ultimately, the impact of our work depends on those developing, researching and regulating LLMs because effective participation requires being asked and being heard [53].

我们就在何处收集偏好、为了什么目的以及会留下何种持久影响，提出三点讨论。第一，通过“基于偏好的功利主义” [121] 来对齐 LLM，未必等同于个人或社会的福祉，这引出一个问题：“正当”的人类反馈是否有边界。 偏好可能 (i) 因短视或信息不对称而与自身利益相悖（例如尽管已有危害证据 [122, 123, 124, 125, 126]，参与者仍希望 LLM 拟人），或 (ii) 与他人利益不相容（例如有人偏好在辩论中持“反觉醒”立场的 LLM，有人则倾向中立）。依赖去语境化的偏好观察，有可能在不知不觉中强化掌权者的偏见 [61, 65]；因此我们建议在做出汇总决策之前，先透明地呈现个体分歧 [9, 127]，尤其是当参与者的立场会影响他们定义伤害的认识合法性时 [59, 128, 129]。第二，当“对齐单元”被操作化为一个群体、一种文化甚至一个物种，而不是一个个体时，难以调和的个人偏好与道德就更加要紧。Prism 既支持利用参与者档案和具体评分的个性化或可引导对齐 [2, 3, 4, 37]，也支持通过意见共识或奖励分布进行的集体对齐 [5, 6, 7, 8, 28]；不过群体审议的结果可能不同于一次从一个人那里收集数据 [6, 118, 119]。随着合成对齐数据的使用增加，Prism 可以帮助把 LLM-as-judge 协议校准到更多样的评分者池 [51, 130]。最后，Prism 的动机是把参与当作正义：相对标注任务或预训练数据中的被动角色，包容性的对齐实践优先吸收本地公民的主动输入，因为他们对自身及社区需求有专门知识 [55]。不过参与仍然很薄，因为 RLHF 成功所依赖的那些人通常并不能分享下游收益或利润 [33, 131]。归根结底，我们工作的影响取决于那些开发、研究和监管 LLM 的人，因为有效的参与既需要被询问，也需要被听见 [53]。

In their early demonstrations of aligning AI systems to human feedback, Bai et al. 2022a discuss alignment data as a public good. We echo this sentiment with Prism—a new feedback dataset from 1,500 diverse humans, motivated by the need for inclusive, participatory and open scientific research into the pressing question of what it means to align LLMs to human preferences in a pluralistic world.

在对齐 AI 系统与人类反馈的早期演示中，Bai 等人 2022a 把对齐数据当作一种公共品来讨论。我们用 Prism 呼应这一看法——这是来自 1,500 名多样化人类的新反馈数据集，其动机是：在一个多元世界里，要对齐 LLM 与人类偏好究竟意味着什么，这一紧迫问题需要包容、参与式和开放的科学研究。

Acknowledgments and Disclosure of Funding

致谢与资助披露

This project was awarded the MetaAI Dynabench Grant “Optimising feedback between humans-and-models-in-the-loop”. For additional compute support, the project was awarded the Microsoft Azure Accelerating Foundation Model Research Grant. For additional annotation support, we received funding from the OpenPhil grant and NSF grant (IIS-2340345) via New York University. We are grateful for support received in the form of research access or credits from OpenAI, Anthropic, Aleph Alpha, Google, HuggingFace and Cohere. Hannah Rose Kirk’s PhD is supported by the Economic and Social Research Council grant ES/P000649/1. Paul Röttger is a member of the Data and Marketing Insights research unit of the Bocconi Institute for Data Science and Analysis, and is supported by a MUR FARE 2020 initiative under grant agreement Prot. R20YSMBZ8S (INDOMITA). Andrew Bean’s PhD is supported by the Clarendon Fund Scholarships at the University of Oxford. We are particularly grateful to Maximilian Kasy for his valuable input and advice on the welfare experiments. We are indebted to the incredible effort and time that our Prolific annotators put into our task, as well as the expert advice from Prolific consultant Andrew Gordon. We also thank any Beta testers, including friends, family and colleagues at Oxford and New York University, for their help in piloting (and debugging!) our task. Lastly, we thank Jakob Mökander, Nathan Lambert, Natasha Jacques, Felix Simon, Nino Scherrer, Maximilian Kroner Dale, Saffron Huang, Amanda Curtis and Joanna Rivera-Carlisle for their feedback on the paper in its various eras. We use scientific colour maps in our figures [132].

本项目获得 MetaAI Dynabench 资助“Optimising feedback between humans-and-models-in-the-loop”。在额外算力支持方面，项目获得 Microsoft Azure Accelerating Foundation Model Research Grant。在额外标注支持方面，我们通过纽约大学获得 OpenPhil 资助以及 NSF 资助（IIS-2340345）。我们感谢 OpenAI、Anthropic、Aleph Alpha、Google、HuggingFace 和 Cohere 以研究访问或额度形式给予的支持。Hannah Rose Kirk 的博士研究得到 Economic and Social Research Council 资助 ES/P000649/1 的支持。Paul Röttger 是 Bocconi Institute for Data Science and Analysis 的 Data and Marketing Insights 研究单元成员，并得到 MUR FARE 2020 计划资助协议 Prot. R20YSMBZ8S（INDOMITA）的支持。Andrew Bean 的博士研究得到牛津大学 Clarendon Fund Scholarships 的支持。我们特别感谢 Maximilian Kasy 对福利实验的宝贵意见和建议。我们感谢 Prolific 标注者投入到任务中的巨大努力和时间，以及 Prolific 顾问 Andrew Gordon 的专业建议。我们也感谢所有 Beta 测试者，包括牛津和纽约大学的朋友、家人和同事，帮助试运行（并调试！）我们的任务。最后，我们感谢 Jakob Mökander、Nathan Lambert、Natasha Jacques、Felix Simon、Nino Scherrer、Maximilian Kroner Dale、Saffron Huang、Amanda Curtis 和 Joanna Rivera-Carlisle 在论文各个阶段给出的反馈。我们在图中使用了科学配色 [132]。

Author Contribution Statement

作者贡献声明

References

参考文献

Gabriel [2020] Iason Gabriel. Artificial Intelligence, Values and Alignment. Minds and Machines , 30(3):411–437, September 2020. ISSN 0924-6495, 1572-8641. doi: 10.1007/s11023-020-09539-2 .

Kirk et al. [2024] Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A. Hale. The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence , pages 1–10, April 2024. ISSN 2522-5839. doi: 10.1038/s42256-024-00820-y . URL https://www.nature.com/articles/s42256-024-00820-y . Publisher: Nature Publishing Group.

Jang et al. [2023] Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, and Prithviraj Ammanabrolu. Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging, October 2023. URL http://arxiv.org/abs/2310.11564 . arXiv:2310.11564 [cs].

Li et al. [2024a] Xinyu Li, Zachary C. Lipton, and Liu Leqi. Personalized Language Modeling from Personalized Human Feedback, February 2024a. URL http://arxiv.org/abs/2402.05133 . arXiv:2402.05133 [cs].

Bakker et al. [2022] Michiel A. Bakker, Martin J. Chadwick, Hannah R. Sheahan, Michael Henry Tessler, Lucy Campbell-Gillingham, Jan Balaguer, Nat McAleese, Amelia Glaese, John Aslanides, Matthew M. Botvinick, and Christopher Summerfield. Fine-tuning language models to find agreement among humans with diverse preferences. In Advances in neural information processing systems , volume 35, pages 38176–38189. Curran Associates, Inc., November 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/f978c8f3b5f399cae464e85f72e28503-Paper-Conference.pdf . _eprint: 2211.15006v1.

Anthropic [2023] Anthropic. Collective Constitutional AI: Aligning a Language Model with Public Input. Technical report, 2023. URL https://www.anthropic.com/news/collective-constitutional-ai-aligning-a-language-model-with-public-input .

Chakraborty et al. [2024] Souradip Chakraborty, Jiahao Qiu, Hui Yuan, Alec Koppel, Furong Huang, Dinesh Manocha, Amrit Singh Bedi, and Mengdi Wang. MaxMin-RLHF: Towards Equitable Alignment of Large Language Models with Diverse Human Preferences, February 2024. URL http://arxiv.org/abs/2402.08925 . arXiv:2402.08925 [cs].

Li et al. [2024b] Dexun Li, Cong Zhang, Kuicai Dong, Derrick Goh Xin Deik, Ruiming Tang, and Yong Liu. Aligning Crowd Feedback via Distributional Preference Reward Modeling, February 2024b. URL http://arxiv.org/abs/2402.09764 . arXiv:2402.09764 [cs].

Conitzer et al. [2024] Vincent Conitzer, Rachel Freedman, Jobst Heitzig, Wesley H. Holliday, Bob M. Jacobs, Nathan Lambert, Milan Mossé, Eric Pacuit, Stuart Russell, Hailey Schoelkopf, Emanuel Tewolde, and William S. Zwicker. Social Choice for AI Alignment: Dealing with Diverse Human Feedback, April 2024. URL http://arxiv.org/abs/2404.10271 . arXiv:2404.10271 [cs].

Kirk et al. [2023a] Hannah Kirk, Andrew Bean, Bertie Vidgen, Paul Rottger, and Scott Hale. The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages 2409–2430, Singapore, December 2023a. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.148 . URL https://aclanthology.org/2023.emnlp-main.148 .

Lambert et al. [2023a] Nathan Lambert, Thomas Krendl Gilbert, and Tom Zick. The History and Risks of Reinforcement Learning and Human Feedback, November 2023a. URL http://arxiv.org/abs/2310.13595 . arXiv:2310.13595 [cs].

Wu et al. [2023] Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, and Hannaneh Hajishirzi. Fine-Grained Human Feedback Gives Better Rewards for Language Model Training, June 2023. URL http://arxiv.org/abs/2306.01693 . arXiv:2306.01693 [cs].

Casper et al. [2023] Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Wang, Samuel Marks, Charbel-Raphaël Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman Anwar, Anand Siththaranjan, Max Nadeau, Eric J. Michaud, Jacob Pfau, Dmitrii Krasheninnikov, Xin Chen, Lauro Langosco, Peter Hase, Erdem Bıyık, Anca Dragan, David Krueger, Dorsa Sadigh, and Dylan Hadfield-Menell. Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback, September 2023. URL http://arxiv.org/abs/2307.15217 . arXiv:2307.15217 [cs].

Prabhakaran et al. [2021] Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, and Mark Diaz. On Releasing Annotator-Level Labels and Information in Datasets. In Proceedings of the Joint 15th Linguistic Annotation Workshop (LAW) and 3rd Designing Meaning Representations (DMR) Workshop , pages 133–138, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.law-1.14 . URL https://aclanthology.org/2021.law-1.14 .

Rottger et al. [2022] Paul Rottger, Bertie Vidgen, Dirk Hovy, and Janet Pierrehumbert. Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 175–190, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.13 . URL https://aclanthology.org/2022.naacl-main.13 .

Kirk et al. [2023b] Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A. Hale. The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models. In Socially Responsible Language Modelling Research (SoLaR) . arXiv, November 2023b. doi: 10.48550/arXiv.2310.02457 . URL http://arxiv.org/abs/2310.02457 .

Mas-Colell et al. [1995] Andreu Mas-Colell, Michael Dennis Whinston, Jerry R Green, et al. Microeconomic theory , volume 1. Oxford university press, New York, 1995.

Rafailov et al. [2024] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In Advances in Neural Information Processing Systems , volume 36, February 2024. URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html .

Zhu et al. [2024a] Banghua Zhu, Jiantao Jiao, and Michael I. Jordan. Principled Reinforcement Learning with Human Feedback from Pairwise or $K$-wise Comparisons, February 2024a. URL http://arxiv.org/abs/2301.11270 . arXiv:2301.11270 [cs, math, stat].

Zhu et al. [2024b] Banghua Zhu, Michael I. Jordan, and Jiantao Jiao. Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF, January 2024b. URL http://arxiv.org/abs/2401.16335 . arXiv:2401.16335 [cs, stat].

Turchin [2019] Alexey Turchin. AI Alignment Problem: "Human Values" Don’t Actually Exist. PhilArchive , 2019. URL https://philarchive.org/rec/TURAAP .

Earp et al. [2021] Brian D. Earp, Killian L. McLoughlin, Joshua T. Monrad, Margaret S. Clark, and Molly J. Crockett. How social relationships shape moral wrongness judgments. Nature Communications , 12(1):5776, October 2021. ISSN 2041-1723. doi: 10.1038/s41467-021-26067-4 . URL https://www.nature.com/articles/s41467-021-26067-4 . Publisher: Nature Publishing Group.

Mascolo et al. [2021] Michael F Mascolo, Allison DiBianca Fasoli, and David Greenway. A Relational Approach to Moral Development in Societies, Organizations and Individuals. Integral Review , 17(1), 2021.

Butler et al. [2000] Judith Butler, Ernesto Laclau, and Slavoj Žižek. Contingency, hegemony, universality: contemporary dialogues on the left . Phronesis. Verso, London, 2000. ISBN 978-1-85984-757-2 978-1-85984-278-2. OCLC: ocm44780799.

Sloane [2024] Mona Sloane. Controversies, contradiction, and “participation” in AI. Big Data & Society , 11(1):20539517241235862, March 2024. ISSN 2053-9517. doi: 10.1177/20539517241235862 . URL https://doi.org/10.1177/20539517241235862 . Publisher: SAGE Publications Ltd.

Talat et al. [2022a] Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. On the machine learning of ethical judgments from natural language. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 769–779, Seattle, United States, July 2022a. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.56 . URL https://aclanthology.org/2022.naacl-main.56 .

Aroyo and Welty [2015] Lora Aroyo and Chris Welty. Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation. AI Magazine , 36(1):15–24, March 2015. ISSN 2371-9621. doi: 10.1609/aimag.v36i1.2564 . URL https://ojs.aaai.org/index.php/aimagazine/article/view/2564 . Number: 1.

Siththaranjan et al. [2023] Anand Siththaranjan, Cassidy Laidlaw, and Dylan Hadfield-Menell. Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF, December 2023. URL http://arxiv.org/abs/2312.08358 . arXiv:2312.08358 [cs, stat].

Stiennon et al. [2020] Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems , volume 33, pages 3008–3021. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/1f89885d556929e98d3ef9b86448f951-Abstract.html .

Bai et al. [2022a] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback, April 2022a. URL http://arxiv.org/abs/2204.05862 . arXiv:2204.05862 [cs].

Ouyang et al. [2022] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems , volume 35, pages 27730–27744, December 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html .

Bai et al. [2022b] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. Constitutional AI: Harmlessness from AI Feedback, December 2022b. URL http://arxiv.org/abs/2212.08073 . arXiv:2212.08073 [cs].

Birhane et al. [2022] Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. Power to the People? Opportunities and Challenges for Participatory AI. In Equity and Access in Algorithms, Mechanisms, and Optimization , EAAMO ’22, pages 1–8, New York, NY, USA, October 2022. Association for Computing Machinery. ISBN 978-1-4503-9477-2. doi: 10.1145/3551624.3555290 . URL https://doi.org/10.1145/3551624.3555290 .

Durmus et al. [2023] Esin Durmus, Karina Nyugen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. Towards Measuring the Representation of Subjective Global Opinions in Language Models, June 2023. URL http://arxiv.org/abs/2306.16388 . arXiv:2306.16388 [cs].

AlKhamissi et al. [2024] Badr AlKhamissi, Muhammad ElNokrashy, Mai AlKhamissi, and Mona Diab. Investigating Cultural Alignment of Large Language Models, February 2024. URL http://arxiv.org/abs/2402.13231 . arXiv:2402.13231 [cs].

Ryan et al. [2024] Michael J. Ryan, William Held, and Diyi Yang. Unintended Impacts of LLM Alignment on Global Representation, February 2024. URL http://arxiv.org/abs/2402.15018 . arXiv:2402.15018 [cs].

Sorensen et al. [2024] Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. A Roadmap to Pluralistic Alignment, February 2024. URL http://arxiv.org/abs/2402.05070 . arXiv:2402.05070 [cs].

Gabriel and Ghazavi [2021] Iason Gabriel and Vafa Ghazavi. The Challenge of Value Alignment: from Fairer Algorithms to AI Safety, January 2021. URL http://arxiv.org/abs/2101.06060 . arXiv:2101.06060 [cs].

Stray [2020] Jonathan Stray. Aligning AI Optimization to Community Well-Being. International Journal of Community Well-Being , 3(4):443–463, December 2020. ISSN 2524-5309. doi: 10.1007/s42413-020-00086-3 . URL https://doi.org/10.1007/s42413-020-00086-3 .

Touvron et al. [2023] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open Foundation and Fine-Tuned Chat Models, July 2023. URL http://arxiv.org/abs/2307.09288 . arXiv:2307.09288 [cs].

Jiang et al. [2023] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7B, October 2023. URL http://arxiv.org/abs/2310.06825 . arXiv:2310.06825 [cs].

Gift [1989] Audrey G. Gift. Visual Analogue Scales: Measurement of Subjective Phenomena. Nursing Research , 38(5):286, October 1989. ISSN 0029-6562. URL https://journals.lww.com/nursingresearchonline/citation/1989/09000/visual_analogue_scales__measurement_of_subjective.6.aspx?casa_token=aO_mhu6sQyEAAAAA:y06v3LLFR-ZeutMmvlWTDebC4T_Je8nE_dS4M_qu96DJ6C_gR8Ro37l58bzqwrw5zSexya6bpnQspOJLfY8UXSrf .

Kiela et al. [2021] Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams. Dynabench: Rethinking Benchmarking in NLP. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 4110–4124, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.naacl-main.324 . URL https://aclanthology.org/2021.naacl-main.324 .

Thrush et al. [2022] Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, and Douwe Kiela. Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks. In Valerio Basile, Zornitsa Kozareva, and Sanja Stajner, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations , pages 174–181, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-demo.17 . URL https://aclanthology.org/2022.acl-demo.17 .

Aitken [1969] R. C. Aitken. Measurement of feelings using visual analogue scales. Proceedings of the Royal Society of Medicine , 62(10):989–993, October 1969. ISSN 0035-9157. URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1810824/ .

Singhal et al. [2023] Prasann Singhal, Tanya Goyal, Jiacheng Xu, and Greg Durrett. A Long Way to Go: Investigating Length Correlations in RLHF, October 2023. URL http://arxiv.org/abs/2310.03716 . arXiv:2310.03716 [cs].

Campello et al. [2013] Ricardo J. G. B. Campello, Davoud Moulavi, and Joerg Sander. Density-Based Clustering Based on Hierarchical Density Estimates. In Jian Pei, Vincent S. Tseng, Longbing Cao, Hiroshi Motoda, and Guandong Xu, editors, Advances in Knowledge Discovery and Data Mining , Lecture Notes in Computer Science, pages 160–172, Berlin, Heidelberg, 2013. Springer. ISBN 978-3-642-37456-2. doi: 10.1007/978-3-642-37456-2_14 .

Negahban et al. [2012] Sahand Negahban, Sewoong Oh, and Devavrat Shah. Iterative ranking from pair-wise comparisons. In Advances in Neural Information Processing Systems , volume 25. Curran Associates, Inc., 2012. URL https://papers.nips.cc/paper_files/paper/2012/hash/9adeb82fffb5444e81fa0ce8ad8afe7a-Abstract.html .

Bana et al. [2021] Gergei Bana, Wojciech Jamroga, David Naccache, and Peter Y. A. Ryan. Convergence Voting: From Pairwise Comparisons to Consensus, March 2021. URL http://arxiv.org/abs/2102.01995 . arXiv:2102.01995 [cs].

Page et al. [1999] Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The PageRank Citation Ranking: Bringing Order to the Web., November 1999. URL http://ilpubs.stanford.edu:8090/422/?doi=10.1.1.31.1768 . Type: Techreport.

Zheng et al. [2023] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems , 36:46595–46623, December 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html .

Noble [2018] Safiya Umoja Noble. Algorithms of oppression: how search engines reinforce racism . New York University Press, New York, 2018. ISBN 978-1-4798-4994-9 978-1-4798-3724-3.

Kelty [2019] Christopher M. Kelty. The Participant – A Century of Participation in Four Stories . The University of Chicago press, Chicago (Ill.) London, 2019. ISBN 978-0-226-66662-4 978-0-226-66676-1.

Criado-Perez [2019] Caroline Criado-Perez. Invisible women: exposing data bias in a world designed for men . Chatto & Windus, London, 2019. ISBN 978-1-78474-172-3 978-1-78474-292-8.

Sloane et al. [2022] Mona Sloane, Emanuel Moss, Olaitan Awomolo, and Laura Forlano. Participation Is not a Design Fix for Machine Learning. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization , EAAMO ’22, pages 1–6, New York, NY, USA, October 2022. Association for Computing Machinery. ISBN 978-1-4503-9477-2. doi: 10.1145/3551624.3555285 . URL https://dl.acm.org/doi/10.1145/3551624.3555285 .

Greene et al. [2023] Travis Greene, Copenhagen Business School, Galit Shmueli, National Tsing Hua University, Soumya Ray, and National Tsing Hua University. Taking the Person Seriously: Ethically Aware IS Research in the Era of Reinforcement Learning-Based Personalization. Journal of the Association for Information Systems , 24(6):1527–1561, 2023. ISSN 15369323. doi: 10.17705/1jais.00800 . URL https://aisel.aisnet.org/jais/vol24/iss6/6/ .

Nakano et al. [2021] Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted question-answering with human feedback. December 2021. URL http://arxiv.org/abs/2112.09332v3 .

Birhane et al. [2021] Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: misogyny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963 , 2021.

Benjamin [2019] Ruha Benjamin. Race After Technology: Abolitionist Tools for the New Jim Code . John Wiley & Sons, July 2019. ISBN 978-1-5095-2643-7.

Blodgett et al. [2020] Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. Language (Technology) is Power: A Critical Survey of “Bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 5454–5476, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.485 . URL https://aclanthology.org/2020.acl-main.485 .

Hershcovich et al. [2022] Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. Challenges and Strategies in Cross-Cultural NLP. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 6997–7013, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.482 . URL https://aclanthology.org/2022.acl-long.482 .

Wang et al. [2024] Wenxuan Wang, Wenxiang Jiao, Jingyuan Huang, Ruyi Dai, Jen-tse Huang, Zhaopeng Tu, and Michael R. Lyu. Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models, February 2024. URL http://arxiv.org/abs/2310.12481 . arXiv:2310.12481 [cs] version: 2.

Gururangan et al. [2022] Suchin Gururangan, Dallas Card, Sarah K. Dreier, Emily K. Gade, Leroy Z. Wang, Zeyu Wang, Luke Zettlemoyer, and Noah A. Smith. Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection, January 2022. URL http://arxiv.org/abs/2201.10474 . arXiv:2201.10474 [cs].

Dzieza [2023] Josh Dzieza. Inside the AI Factory, June 2023. URL https://www.theverge.com/features/23764584/ai-artificial-intelligence-data-notation-labor-scale-surge-remotasks-openai-chatbots .

Mohamed et al. [2020] Shakir Mohamed, Marie-Therese Png, and William Isaac. Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence. Philosophy & Technology , 33(4):659–684, December 2020. ISSN 2210-5441. doi: 10.1007/s13347-020-00405-8 . URL https://doi.org/10.1007/s13347-020-00405-8 .

Airoldi [2022] Massimo Airoldi. Machine habitus: toward a sociology of algorithms . Polity Press, Cambridge ; Medford, MA, 2022. ISBN 978-1-5095-4327-4 978-1-5095-4328-1. OCLC: on1247827618.

Talat et al. [2022b] Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. On the Machine Learning of Ethical Judgments from Natural Language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 769–779, Seattle, United States, July 2022b. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.56 . URL https://aclanthology.org/2022.naacl-main.56 .

Díaz et al. [2022] Mark Díaz, Ian Kivlichan, Rachel Rosen, Dylan Baker, Razvan Amironesei, Vinodkumar Prabhakaran, and Emily Denton. CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation. In 2022 ACM Conference on Fairness, Accountability, and Transparency , FAccT ’22, pages 2342–2351, New York, NY, USA, June 2022. Association for Computing Machinery. ISBN 978-1-4503-9352-2. doi: 10.1145/3531146.3534647 . URL https://doi.org/10.1145/3531146.3534647 .

Bender and Friedman [2018] Emily M. Bender and Batya Friedman. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics , 6:587–604, 2018. doi: 10.1162/tacl_a_00041 . URL https://aclanthology.org/Q18-1041 . Place: Cambridge, MA Publisher: MIT Press.

Mitchell et al. [2019] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT* ’19 , pages 220–229, 2019. doi: 10.1145/3287560.3287596 . URL http://arxiv.org/abs/1810.03993 . arXiv: 1810.03993.

Gebru et al. [2021] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. Datasheets for datasets. Communications of the ACM , 64(12):86–92, December 2021. ISSN 15577317. doi: 10.1145/3458723 . Publisher: Association for Computing Machinery.

Davani et al. [2022] Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations. Transactions of the Association for Computational Linguistics , 10:92–110, January 2022. ISSN 2307-387X. doi: 10.1162/tacl_a_00449 . URL https://doi.org/10.1162/tacl_a_00449 .

Gordon et al. [2022] Mitchell L. Gordon, Michelle S. Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael S. Bernstein. Jury Learning: Integrating Dissenting Voices into Machine Learning Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems , CHI ’22, pages 1–19, New York, NY, USA, April 2022. Association for Computing Machinery. ISBN 978-1-4503-9157-3. doi: 10.1145/3491102.3502004 . URL https://doi.org/10.1145/3491102.3502004 .

Plank et al. [2014] Barbara Plank, Dirk Hovy, and Anders Søgaard. Learning part-of-speech taggers with inter-annotator agreement loss. In Shuly Wintner, Sharon Goldwater, and Stefan Riezler, editors, Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics , pages 742–751, Gothenburg, Sweden, April 2014. Association for Computational Linguistics. doi: 10.3115/v1/E14-1078 . URL https://aclanthology.org/E14-1078 .

Nie et al. [2020] Yixin Nie, Xiang Zhou, and Mohit Bansal. What Can We Learn from Collective Human Opinions on Natural Language Inference Data? In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 9131–9143, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.734 . URL https://aclanthology.org/2020.emnlp-main.734 .

Wich et al. [2021] Maximilian Wich, Christian Widmer, Gerhard Hagerer, and Georg Groh. Investigating Annotator Bias in Abusive Language Datasets. In Ruslan Mitkov and Galia Angelova, editors, Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021) , pages 1515–1525, Held Online, September 2021. INCOMA Ltd. URL https://aclanthology.org/2021.ranlp-1.170 .

Sap et al. [2022] Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 5884–5906, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.431 . URL https://aclanthology.org/2022.naacl-main.431 .

Goyal et al. [2022] Nitesh Goyal, Ian D. Kivlichan, Rachel Rosen, and Lucy Vasserman. Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation. Proceedings of the ACM on Human-Computer Interaction , 6(CSCW2):363:1–363:28, November 2022. doi: 10.1145/3555088 . URL https://dl.acm.org/doi/10.1145/3555088 .

Aroyo et al. [2023] Lora Aroyo, Alex S. Taylor, Mark Diaz, Christopher M. Homan, Alicia Parrish, Greg Serapio-Garcia, Vinodkumar Prabhakaran, and Ding Wang. DICES Dataset: Diversity in Conversational AI Evaluation for Safety, June 2023. URL http://arxiv.org/abs/2306.11247 . arXiv:2306.11247 [cs].

Henrich et al. [2010] Joseph Henrich, Steven J. Heine, and Ara Norenzayan. Most people are not WEIRD. Nature , 466(7302):29–29, July 2010. ISSN 1476-4687. doi: 10.1038/466029a . URL https://www.nature.com/articles/466029a . Number: 7302 Publisher: Nature Publishing Group.

Urbina and Ruiz-Villaverde [2019] Dante A. Urbina and Alberto Ruiz-Villaverde. A Critical Review of Homo Economicus from Five Approaches. The American Journal of Economics and Sociology , 78(1):63–93, 2019. ISSN 1536-7150. doi: 10.1111/ajes.12258 . URL https://onlinelibrary.wiley.com/doi/abs/10.1111/ajes.12258 . _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/ajes.12258.

Apicella et al. [2020] Coren Apicella, Ara Norenzayan, and Joseph Henrich. Beyond WEIRD: A review of the last decade and a look ahead to the global laboratory of the future. Evolution and Human Behavior , 41(5):319–329, September 2020. ISSN 1090-5138. doi: 10.1016/j.evolhumbehav.2020.07.015 . URL https://www.sciencedirect.com/science/article/pii/S1090513820300957 .

Ng and Russell [2000] Andrew Ng and Stuart J. Russell. Algorithms for Inverse Reinforcement Learning. 2000.

Christiano et al. [2017] Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep Reinforcement Learning from Human Preferences. In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/hash/d5e2c0adad503c91f91df240d0cd4e49-Abstract.html .

Ziegler et al. [2019] Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-Tuning Language Models from Human Preferences. September 2019. URL http://arxiv.org/abs/1909.08593v2 .

Mirkin and Meunier [2015] Shachar Mirkin and Jean-Luc Meunier. Personalized machine translation: Predicting translational preferences. In Proceedings of the 2015 conference on empirical methods in natural language processing , pages 2019–2025, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: 10.18653/v1/D15-1238 . URL https://aclanthology.org/D15-1238 .

Nguyen et al. [2017] Khanh Nguyen, Hal Daumé III, and Jordan Boyd-Graber. Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pages 1464–1474, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18653/v1/D17-1153 . URL https://aclanthology.org/D17-1153 .

Kreutzer et al. [2017] Julia Kreutzer, Artem Sokolov, and Stefan Riezler. Bandit Structured Prediction for Neural Sequence-to-Sequence Learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 1503–1513, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10.18653/v1/P17-1138 . URL https://aclanthology.org/P17-1138 .

Walker [2000] Marilyn A Walker. An application of reinforcement learning to dialogue strategy selection in a spoken dialogue system for email. Journal of Artificial Intelligence Research , 12:387–416, 2000.

Schatzmann et al. [2006] Jost Schatzmann, Karl Weilhammer, Matt Stuttle, and Steve Young. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. The knowledge engineering review , 21(2):97–126, 2006.

Su et al. [2016] Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Maria Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve J. Young. Continuously learning neural dialogue management. abs/1606.02689, 2016. URL http://arxiv.org/abs/1606.02689 .

Li et al. [2017] Jiwei Li, Alexander H. Miller, Sumit Chopra, Marc’Aurelio Ranzato, and Jason Weston. Dialogue learning with human-in-the-loop. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings . OpenReview.net, 2017. URL https://openreview.net/forum?id=HJgXCV9xx .

Jaques et al. [2019] Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, and Rosalind Picard. Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog, July 2019. URL http://arxiv.org/abs/1907.00456 . arXiv:1907.00456 [cs, stat].

Jaques et al. [2020] Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, and Rosalind Picard. Human-centric dialog training via offline reinforcement learning. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , pages 3985–4003, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.327 . URL https://aclanthology.org/2020.emnlp-main.327 .

Glaese et al. [2022] Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soňa Mokrá, Nicholas Fernando, Boxi Wu, Rachel Foley, Susannah Young, Iason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, and Geoffrey Irving. Improving alignment of dialogue agents via targeted human judgements. September 2022. URL http://arxiv.org/abs/2209.14375v1 .

Scheurer et al. [2022] Jérémy Scheurer, Jon Ander Campos, Jun Shern Chan, Angelica Chen, Kyunghyun Cho, and Ethan Perez. Training Language Models with Language Feedback, November 2022. URL http://arxiv.org/abs/2204.14146 . arXiv:2204.14146 [cs].

Askell et al. [2021] Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, and Jared Kaplan. A General Language Assistant as a Laboratory for Alignment, December 2021. URL http://arxiv.org/abs/2112.00861 . arXiv:2112.00861 [cs].

Schulman et al. [2017] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, August 2017. URL http://arxiv.org/abs/1707.06347 . arXiv:1707.06347 [cs].

Williams [1992] Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning , 8(3):229–256, May 1992. ISSN 1573-0565. doi: 10.1007/BF00992696 . URL https://doi.org/10.1007/BF00992696 .

Ahmadian et al. [2024] Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs, February 2024. URL http://arxiv.org/abs/2402.14740 . arXiv:2402.14740 [cs] version: 1.

Zhou et al. [2023] Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: Less Is More for Alignment, May 2023. URL http://arxiv.org/abs/2305.11206 . arXiv:2305.11206 [cs].

Menick et al. [2022] Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, and Nat McAleese. Teaching language models to support answers with verified quotes, March 2022. URL http://arxiv.org/abs/2203.11147 . arXiv:2203.11147 [cs].

Thoppilan et al. [2022] Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. LaMDA: Language Models for Dialog Applications, February 2022. URL http://arxiv.org/abs/2201.08239 . arXiv:2201.08239 [cs].

Lambert et al. [2024] Nathan Lambert, Valentina Pyatkin, Jacob Morrison, L. J. Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. RewardBench: Evaluating Reward Models for Language Modeling, March 2024. URL http://arxiv.org/abs/2403.13787 . arXiv:2403.13787 [cs].

Ethayarajh et al. [2024] Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. KTO: Model Alignment as Prospect Theoretic Optimization, February 2024. URL http://arxiv.org/abs/2402.01306 . arXiv:2402.01306 [cs].

StanfordNLP [2023] StanfordNLP. Stanford Human Preferences Dataset, September 2023. URL https://huggingface.co/datasets/stanfordnlp/SHP .

Lambert et al. [2023b] Nathan Lambert, Lewis Tunstall, Nazneen Rajani, and Tristan Thrush. HuggingFace H4 Stack Exchange Preference Dataset, 2023b. URL https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences .

Agnew et al. [2024] William Agnew, A. Stevie Bergman, Jennifer Chien, Mark Díaz, Seliem El-Sayed, Jaylen Pittman, Shakir Mohamed, and Kevin R. McKee. The illusion of artificial inclusion, February 2024. URL http://arxiv.org/abs/2401.08572 . arXiv:2401.08572 [cs].

Dubois et al. [2024] Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback, January 2024. URL http://arxiv.org/abs/2305.14387 . arXiv:2305.14387 [cs].

Guo et al. [2024] Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Jiexin Wang, Huimin Chen, Bowen Sun, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, and Maosong Sun. Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment, February 2024. URL http://arxiv.org/abs/2402.19085 . arXiv:2402.19085 [cs, eess].

Zheng et al. [2024] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset, March 2024. URL http://arxiv.org/abs/2309.11998 . arXiv:2309.11998 [cs].

Zhao et al. [2023] Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. (InThe)WildChat: 570K ChatGPT Interaction Logs In The Wild. October 2023. URL https://openreview.net/forum?id=Bl8u7ZRlbM .

Köpf et al. [2023] Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. OpenAssistant Conversations – Democratizing Large Language Model Alignment, October 2023. URL http://arxiv.org/abs/2304.07327 . arXiv:2304.07327 [cs].

Ganguli et al. [2022] Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec, Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph, Sam McCandlish, Chris Olah, Jared Kaplan, and Jack Clark. Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned, November 2022. URL http://arxiv.org/abs/2209.07858 . arXiv:2209.07858 [cs].

Singh et al. [2024] Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergün, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Minh Chien, Sebastian Ruder, Surya Guthikonda, Emad A. Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, and Sara Hooker. Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning, February 2024. URL http://arxiv.org/abs/2402.06619 . arXiv:2402.06619 [cs].

Institute and Institute [2023] The Alan Turing Institute and The Ada Lovelace Institute. How do people feel about AI? A nationally representative survey of public attitudes to artificial intelligence in Britain. Technical report, 2023. URL https://attitudestoai.uk/assets/documents/Ada-Lovelace-Institute-The-Alan-Turing-Institute-How-do-people-feel-about-AI.pdf .

Mun et al. [2024] Jimin Mun, Liwei Jiang, Jenny Liang, Inyoung Cheong, Nicole DeCario, Yejin Choi, Tadayoshi Kohno, and Maarten Sap. Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits, March 2024. URL http://arxiv.org/abs/2403.14791 . arXiv:2403.14791 [cs].

Chang et al. [2024] Samuel Chang, Estelle Ciesla, Michael Finch, James Fishkin, Lodewijk Gelauff, Ashish Goel, Ricky Hernandez Marquez, Shoaib Mohammed, and Alice Siu. Meta Community Forum: Results Analysis. Technical report, Deliberative Democracy Lab, Stanford University, April 2024.

Bergman et al. [2024] Stevie Bergman, Nahema Marchal, John Mellor, Shakir Mohamed, Iason Gabriel, and William Isaac. STELA: a community-centred approach to norm elicitation for AI alignment. Scientific Reports , 14(1):6616, March 2024. ISSN 2045-2322. doi: 10.1038/s41598-024-56648-4 . URL https://www.nature.com/articles/s41598-024-56648-4 . Publisher: Nature Publishing Group.

Silberzahn et al. [2018] R. Silberzahn, E. L. Uhlmann, D. P. Martin, P. Anselmi, F. Aust, E. Awtrey, Š. Bahník, F. Bai, C. Bannard, E. Bonnier, R. Carlsson, F. Cheung, G. Christensen, R. Clay, M. A. Craig, A. Dalla Rosa, L. Dam, M. H. Evans, I. Flores Cervantes, N. Fong, M. Gamez-Djokic, A. Glenz, S. Gordon-McKeon, T. J. Heaton, K. Hederos, M. Heene, A. J. Hofelich Mohr, F. Högden, K. Hui, M. Johannesson, J. Kalodimos, E. Kaszubowski, D. M. Kennedy, R. Lei, T. A. Lindsay, S. Liverani, C. R. Madan, D. Molden, E. Molleman, R. D. Morey, L. B. Mulder, B. R. Nijstad, N. G. Pope, B. Pope, J. M. Prenoveau, F. Rink, E. Robusto, H. Roderique, A. Sandberg, E. Schlüter, F. D. Schönbrodt, M. F. Sherman, S. A. Sommer, K. Sotak, S. Spain, C. Spörlein, T. Stafford, L. Stefanutti, S. Tauber, J. Ullrich, M. Vianello, E.-J. Wagenmakers, M. Witkowiak, S. Yoon, and B. A. Nosek. Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science , 1(3):337–356, September 2018. ISSN 2515-2459. doi: 10.1177/2515245917747646 . URL https://doi.org/10.1177/2515245917747646 . Publisher: SAGE Publications Inc.

Tasioulas [2022] John Tasioulas. Artificial Intelligence, Humanistic Ethics. Daedalus , 151(2):232–243, May 2022. ISSN 0011-5266. doi: 10.1162/daed_a_01912 . URL https://doi.org/10.1162/daed_a_01912 .

Proudfoot [2011] Diane Proudfoot. Anthropomorphism and AI: Turing’s much misunderstood imitation game. Artificial Intelligence , 175(5):950–957, April 2011. ISSN 0004-3702. doi: 10.1016/j.artint.2011.01.006 . URL https://www.sciencedirect.com/science/article/pii/S000437021100018X .

Watson [2019] David Watson. The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence. Minds and Machines , 29(3):417–440, September 2019. ISSN 1572-8641. doi: 10.1007/s11023-019-09506-6 . URL https://doi.org/10.1007/s11023-019-09506-6 .

Weidinger et al. [2022] Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. Taxonomy of Risks posed by Language Models. In 2022 ACM Conference on Fairness, Accountability, and Transparency , pages 214–229, Seoul Republic of Korea, June 2022. ACM. ISBN 978-1-4503-9352-2. doi: 10.1145/3531146.3533088 . URL https://dl.acm.org/doi/10.1145/3531146.3533088 .

Abercrombie et al. [2023] Gavin Abercrombie, Amanda Cercas Curry, Tanvi Dinkar, Verena Rieser, and Zeerak Talat. Mirages. On Anthropomorphism in Dialogue Systems. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages 4776–4790, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.290 . URL https://aclanthology.org/2023.emnlp-main.290 .

Cheng et al. [2024] Myra Cheng, Kristina Gligoric, Tiziano Piccardi, and Dan Jurafsky. AnthroScore: A Computational Linguistic Measure of Anthropomorphism, February 2024. URL http://arxiv.org/abs/2402.02056 . arXiv:2402.02056 [cs].

Gooding and Mansoor [2023] Sian Gooding and Hassan Mansoor. The Impact of Preference Agreement in Reinforcement Learning from Human Feedback: A Case Study in Summarization, November 2023. URL http://arxiv.org/abs/2311.04919 . arXiv:2311.04919 [cs].

D’Ignazio and Klein [2020] Catherine D’Ignazio and Lauren F. Klein. Data Feminism . The MIT Press, March 2020. ISBN 978-0-262-35852-1. doi: 10.7551/mitpress/11805.001.0001 . URL https://direct.mit.edu/books/book/4660/Data-Feminism .

Birhane [2021] Abeba Birhane. Algorithmic injustice: a relational ethics approach. Patterns , 2(2):100205, February 2021. ISSN 26663899. doi: 10.1016/j.patter.2021.100205 . URL https://linkinghub.elsevier.com/retrieve/pii/S2666389921000155 .

Dong et al. [2024] Yijiang River Dong, Tiancheng Hu, and Nigel Collier. Can LLM be a Personalized Judge?, June 2024. URL http://arxiv.org/abs/2406.11657 . arXiv:2406.11657 [cs].

Perrigo [2024] Billy Perrigo. Inside OpenAI’s Plan to Make AI More ’Democratic’, February 2024. URL https://time.com/6684266/openai-democracy-artificial-intelligence/ .

Crameri et al. [2020] Fabio Crameri, Grace E. Shephard, and Philip J. Heron. The misuse of colour in science communication. Nature Communications , 11(1):5444, October 2020. ISSN 2041-1723. doi: 10.1038/s41467-020-19160-7 . URL https://www.nature.com/articles/s41467-020-19160-7 . Number: 1 Publisher: Nature Publishing Group.

Tomasev et al. [2021] Nenad Tomasev, Kevin R. McKee, Jackie Kay, and Shakir Mohamed. Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society , AIES ’21, pages 254–265, New York, NY, USA, July 2021. Association for Computing Machinery. ISBN 978-1-4503-8473-5. doi: 10.1145/3461702.3462540 . URL https://doi.org/10.1145/3461702.3462540 .

Hosking et al. [2024] Tom Hosking, Phil Blunsom, and Max Bartolo. Human Feedback is not Gold Standard, January 2024. URL http://arxiv.org/abs/2309.16349 . arXiv:2309.16349 [cs].

Tversky and Simonson [1993] Amos Tversky and Itamar Simonson. Context-Dependent Preferences. Management Science , 39(10):1179–1189, October 1993. ISSN 0025-1909. doi: 10.1287/mnsc.39.10.1179 . URL https://pubsonline.informs.org/doi/abs/10.1287/mnsc.39.10.1179 . Publisher: INFORMS.

Shmueli et al. [2021] Boaz Shmueli, Jan Fell, Soumya Ray, and Lun-Wei Ku. Beyond fair pay: Ethical implications of NLP crowdsourcing. In Association for Computational Linguistics (ACL) , pages 3758–3769, April 2021. URL https://arxiv.org/abs/2104.10097v1 . arXiv: 2104.10097 Publisher: tex.arxivid: 2104.10097.

Posch et al. [2022] Lisa Posch, Arnim Bleier, Fabian Flöck, Clemens M. Lechner, Katharina Kinder-Kurlanda, Denis Helic, and Markus Strohmaier. Characterizing the Global Crowd Workforce: A Cross-Country Comparison of Crowdworker Demographics. Human Computation , 9(1), August 2022. ISSN 2330-8001. doi: 10.15346/hc.v9i1.106 . URL http://arxiv.org/abs/1812.05948 . arXiv:1812.05948 [cs].

Albert and Smilek [2023] Derek A. Albert and Daniel Smilek. Comparing attentional disengagement between Prolific and MTurk samples. Scientific Reports , 13(1):20574, November 2023. ISSN 2045-2322. doi: 10.1038/s41598-023-46048-5 . URL https://www.nature.com/articles/s41598-023-46048-5 . Publisher: Nature Publishing Group.

Team [2023] Gemini Team. Gemini: A Family of Highly Capable Multimodal Models, December 2023. URL http://arxiv.org/abs/2312.11805 . arXiv:2312.11805 [cs].

Jiang et al. [2024] Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mixtral of Experts, January 2024. URL http://arxiv.org/abs/2401.04088 . arXiv:2401.04088 [cs].

Anthropic [2024] Anthropic. Introducing the next generation of Claude, April 2024. URL https://www.anthropic.com/news/claude-3-family .

Cohere [2024] Cohere. Command R, April 2024. URL https://docs.cohere.com/docs/command-r .

MetaAI [2024] MetaAI. Introducing Meta Llama 3: The most capable openly available LLM to date, April 2024. URL https://ai.meta.com/blog/meta-llama-3/ .

Santurkar et al. [2023] Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose Opinions Do Language Models Reflect?, March 2023. URL http://arxiv.org/abs/2303.17548 . arXiv:2303.17548 [cs].

Barth [1998] Fredrik Barth. Ethnic Groups and Boundaries: The Social Organization of Culture Difference . Waveland Press, March 1998. ISBN 978-1-4786-0795-3. Google-Books-ID: QaAQAAAAQBAJ.

Hamer et al. [2020] Katarzyna Hamer, Sam McFarland, Barbara Czarnecka, Agnieszka Golińska, Liliana Manrique Cadena, Magdalena Łużniak Piecha, and Tomasz Jułkowski. What Is an “Ethnic Group” in Ordinary People’s Eyes? Different Ways of Understanding It Among American, British, Mexican, and Polish Respondents. Cross-Cultural Research , 54(1):28–72, February 2020. ISSN 1069-3971. doi: 10.1177/1069397118816939 . URL https://doi.org/10.1177/1069397118816939 . Publisher: SAGE Publications Inc.

Suyemoto et al. [2020] Karen L. Suyemoto, Micaela Curley, and Shruti Mukkamala. What Do We Mean by “Ethnicity” and “Race”? A Consensual Qualitative Research Investigation of Colloquial Understandings. Genealogy , 4(3):81, September 2020. ISSN 2313-5778. doi: 10.3390/genealogy4030081 . URL https://www.mdpi.com/2313-5778/4/3/81 . Number: 3 Publisher: Multidisciplinary Digital Publishing Institute.

Iannaccone [1998] Laurence R. Iannaccone. Introduction to the Economics of Religion. Journal of Economic Literature , 36(3):1465–1495, 1998. ISSN 0022-0515. URL https://www.jstor.org/stable/2564806 . Publisher: American Economic Association.

Levy and Razin [2012] Gilat Levy and Ronny Razin. Religious Beliefs, Religious Participation, and Cooperation. American Economic Journal: Microeconomics , 4(3):121–151, August 2012. ISSN 1945-7669, 1945-7685. doi: 10.1257/mic.4.3.121 . URL https://pubs.aeaweb.org/doi/10.1257/mic.4.3.121 .

Dingemans and Ingen [2015] Ellen Dingemans and Erik Van Ingen. Does Religion Breed Trust? A Cross-National Study of the Effects of Religious Involvement, Religious Faith, and Religious Context on Social Trust. Journal for the Scientific Study of Religion , 54(4):739–755, 2015. ISSN 0021-8294. URL https://www.jstor.org/stable/26651394 . Publisher: [Society for the Scientific Study of Religion, Wiley].

Zhang et al. [2019] Hansong Zhang, Joshua N. Hook, Jennifer E. Farrell, David K. Mosher, Laura E. Captari, Steven P. Coomes, Daryl R. Van Tongeren, and Don E. Davis. Exploring Social Belonging and Meaning in Religious Groups. Journal of Psychology and Theology , 47(1):3–19, March 2019. ISSN 0091-6471. doi: 10.1177/0091647118806345 . URL https://doi.org/10.1177/0091647118806345 . Publisher: SAGE Publications Ltd.

Saroglou et al. [2020] Vassilis Saroglou, Magali Clobert, Adam B. Cohen, Kathryn A. Johnson, Kevin L. Ladd, Matthieu Van Pachterbeke, Lucia Adamovova, Joanna Blogowska, Pierre-Yves Brandt, Cem Safak Çukur, Kwang-Kuo Hwang, Anna Miglietta, Frosso Motti-Stefanidi, Antonio Muñoz-García, Sebastian Murken, Nicolas Roussiau, and Javier Tapia Valladares. Believing, Bonding, Behaving, and Belonging: The Cognitive, Emotional, Moral, and Social Dimensions of Religiousness across Cultures. Journal of Cross-Cultural Psychology , 51(7-8):551–575, September 2020. ISSN 0022-0221. doi: 10.1177/0022022120946488 . URL https://doi.org/10.1177/0022022120946488 . Publisher: SAGE Publications Inc.

Prolific [2024] Prolific. Representative samples, February 2024. URL https://researcher-help.prolific.com/hc/en-gb/articles/360019236753-Representative-samples .

Reimers and Gurevych [2019] Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks, August 2019. URL http://arxiv.org/abs/1908.10084 . arXiv:1908.10084 [cs].

McInnes et al. [2020] Leland McInnes, John Healy, and James Melville. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction, September 2020. URL http://arxiv.org/abs/1802.03426 . arXiv:1802.03426 [cs, stat].

Kazemi et al. [2022] Ashkan Kazemi, Kiran Garimella, Gautam Kishore Shahi, Devin Gaffney, and Scott A. Hale. Research note: Tiplines to uncover misinformation on encrypted platforms: A case study of the 2019 Indian general election on WhatsApp. Harvard Kennedy School Misinformation Review , January 2022. doi: 10.37016/mr-2020-91 . URL https://misinforeview.hks.harvard.edu/article/research-note-tiplines-to-uncover-misinformation-on-encrypted-platforms-a-case-study-of-the-2019-indian-general-election-on-whatsapp/ .

Hale [2022] Scott A. Hale. meedan/temporal_clustering, March 2022. URL https://github.com/meedan/temporal_clustering/tree/main .

Emerson et al. [2017] Scott D. Emerson, Martin Guhn, and Anne M. Gadermann. Measurement invariance of the Satisfaction with Life Scale: reviewing three decades of research. Quality of Life Research , 26(9):2251–2264, September 2017. ISSN 1573-2649. doi: 10.1007/s11136-017-1552-2 . URL https://doi.org/10.1007/s11136-017-1552-2 .

Roemer [1998] John E. Roemer. Theories of distributive justice . Harvard Univ. Press, Cambridge, Mass., 1. harvard univ. press paperback ed edition, 1998. ISBN 978-0-674-87920-1 978-0-674-87919-5.

Bentham [1789] Jeremy Bentham. An Introduction to the Principles of Morals and Legislation. In J. H. Burns and H. L. A. Hart, editors, The Collected Works of Jeremy Bentham: An Introduction to the Principles of Morals and Legislation . Oxford University Press, January 1789. ISBN 978-0-19-820516-6. doi: 10.1093/oseo/instance.00077240 . URL http://www.oxfordscholarlyeditions.com/view/10.1093/actrade/9780198205166.book.1/actrade-9780198205166-work-1 .

Lanctot et al. [2023] Marc Lanctot, Kate Larson, Yoram Bachrach, Luke Marris, Zun Li, Avishkar Bhoopchand, Thomas Anthony, Brian Tanner, and Anna Koop. Evaluating Agents using Social Choice Theory, December 2023. URL http://arxiv.org/abs/2312.03121 . arXiv:2312.03121 [cs] version: 2.

Boubdir et al. [2023] Meriem Boubdir, Edward Kim, Beyza Ermis, Sara Hooker, and Marzieh Fadaee. Elo Uncovered: Robustness and Best Practices in Language Model Evaluation, November 2023. URL http://arxiv.org/abs/2311.17295 . arXiv:2311.17295 [cs].

Negahban et al. [2017] Sahand Negahban, Sewoong Oh, and Devavrat Shah. Rank Centrality: Ranking from Pairwise Comparisons. Operations Research , 65(1):266–287, 2017. ISSN 0030-364X. URL https://www.jstor.org/stable/26153541 . Publisher: INFORMS.

Supplementary Material

补充材料

Appendix A Prism Data Access and Format

附录 A Prism 数据访问与格式

The data can be accessed on Github at https://github.com/HannahKirk/prism-alignment, and also on HuggingFace at https://huggingface.co/datasets/HannahRoseKirk/prism-alignment. The dataset has a permanent DOI: 10.57967/hf/2113.

数据可在 Github 访问：https://github.com/HannahKirk/prism-alignment，也可在 HuggingFace 访问：https://huggingface.co/datasets/HannahRoseKirk/prism-alignment。该数据集有永久 DOI：10.57967/hf/2113。

There dataset is organised in two primary JSON lines files:

该数据集组织为两个主要的 JSON lines 文件：

• • The Survey (survey.jsonl): The survey where participants answer questions such as their stated preferences for LLM behaviours, their familarity with LLMs, a self-description and some basic demographics. Each row is a single participant in our dataset, identified by a user_id.

• • 问卷（survey.jsonl）：参与者回答诸如对 LLM 行为的陈述偏好、对 LLM 的熟悉程度、一段自我描述以及一些基本人口统计等问题的问卷。每一行是数据集中的一名参与者，由 user_id 标识。

• • The Conversations (conversations.jsonl): Each participants’ multiple conversation trees with LLMs and associated feedback. Each row is a single conversation, identified by a conversation_id, that can be matched back to a participant’s survey profile via the user_id. The conversation itself is stored as a list of dictionaries representing human and model turns in the conversation_history column, which broadly follows the format of widely used Chat APIs (see single entry schema on the next page).

• • 对话（conversations.jsonl）：每位参与者与 LLM 的多棵对话树及相关反馈。每一行是一场对话，由 conversation_id 标识，并可通过 user_id 匹配回该参与者的问卷档案。对话本身作为表示人类与模型轮次的字典列表，存放在 conversation_history 列中，大体遵循广泛使用的 Chat API 格式（见下一页的单条记录模式）。

Additionally, for ease of secondary analysis we provide a more granular and flattened format of the conversations data:

此外，为便于二次分析，我们提供对话数据一种更细粒、更扁平的格式：

• • The Utterances (utterances.jsonl): Each row is a single scored utterance (human input - model response - score). Each row has an utterance_id that can be mapped back to the conversation data using conversation_id or the survey using user_id. The model responses and scores per each user input are in long format. Because of this format, the user inputs will be repeated for the set of model responses in a single interaction turn.

• • 语句（utterances.jsonl）：每一行是一条已评分的语句（人类输入 - 模型回复 - 分数）。每一行有一个 utterance_id，可通过 conversation_id 映射回对话数据，或通过 user_id 映射回问卷。每个用户输入对应的模型回复和分数采用长表格式。由于这一格式，同一交互轮次中一组模型回复会重复出现该用户输入。

We also provide code for transforming the conversations to a wide format. That is, each row is now a single turn within a conversation. For the first interaction where up to four models respond, we have model_{a/b/c/d} as four distinct columns and score_{a/b/c/d} as another four columns. Note that for subsequent turns, the same model responds and there are only two responses so model/score_{c/d} will always be missing.

我们也提供把对话转换成宽表格式的代码。也就是说，每一行现在是一场对话中的单轮。对最多有四个模型回应的第一次交互，我们把 model_{a/b/c/d} 作为四列，把 score_{a/b/c/d} 作为另外四列。注意在随后各轮中，是同一个模型在回复，并且只有两条回复，因此 model/score_{c/d} 将始终缺失。

Finally, for every text instance in Prism, we provide metadata on the language detection, personal or private information (PII) detection and moderation flags. The Metadata is provided seperately to the main data files (metadata.jsonl).

最后，对 Prism 中的每一段文本实例，我们提供语言检测、个人或私人信息（PII）检测以及审核标记的元数据。元数据与主数据文件分开提供（metadata.jsonl）。

We provide codebooks for The Survey (Section V.1), The Conversations (Section V.2), The Utterances (Section V.3) and The Metadata (Section V.4).

我们为问卷（第 V.1 节）、对话（第 V.2 节）、语句（第 V.3 节）和元数据（第 V.4 节）提供编码手册。

Format of Entries in Conversations Data

{
"conversation_id": "c1",
"user_id": "user123",
"conversation_type": ["unguided", "values guided", "controversy guided"],
"opening_prompt": "[USER PROMPT]",
"conversation_turns": [2-22],
"conversation_history": [
{
"turn": 0,
"role": "user",
"content": "[USER PROMPT]"
},
{
"turn": 0,
"role": "model",
"content": "[MODEL RESPONSE]",
"model_name": "M1",
"model_provider": "P1",
"score": [1-100],
"if_chosen": false,
"within_turn_id":0
},
{
"turn": 0,
"role": "model",
"content": "[MODEL RESPONSE]",
"model_name": "M2",
"model_provider": "P2",
"score": [1-100],
"if_chosen": true,
"within_turn_id":1

},
//... Additional list items for remaining model responses (up to 4 in total)
{
"turn": 1,
"role": "user",
"content": "[USER PROMPT]"
},
{
"turn": 1,
"role": "model",
"content": "[MODEL RESPONSE]",
"model_name": "M2",
"model_provider": "P2",
"score": [1-100],
"if_chosen": true,
"within_turn_id":0
},
{
"turn": 1,
"role": "model",
"content": "[MODEL RESPONSE]",
"model_name": "M2",
"model_provider": "P2",
"score": [1-100],
"if_chosen": false,
"within_turn_id":1
}
//... Additional turns follow the same pattern as turn 1
],
"performance_attributes": {
"fluency": [1-100],
"factuality": [1-100],
"helpfulness": [1-100],
//....Additional attribute ratings
},
"open_feedback": "[FREE-TEXT]"
}

对话数据中条目的格式

{
"conversation_id": "c1",
"user_id": "user123",
"conversation_type": ["unguided", "values guided", "controversy guided"],
"opening_prompt": "[USER PROMPT]",
"conversation_turns": [2-22],
"conversation_history": [
{
"turn": 0,
"role": "user",
"content": "[USER PROMPT]"
},
{
"turn": 0,
"role": "model",
"content": "[MODEL RESPONSE]",
"model_name": "M1",
"model_provider": "P1",
"score": [1-100],
"if_chosen": false,
"within_turn_id":0
},
{
"turn": 0,
"role": "model",
"content": "[MODEL RESPONSE]",
"model_name": "M2",
"model_provider": "P2",
"score": [1-100],
"if_chosen": true,
"within_turn_id":1

},
//... 其余模型回复的列表项（总共最多 4 条）
{
"turn": 1,
"role": "user",
"content": "[USER PROMPT]"
},
{
"turn": 1,
"role": "model",
"content": "[MODEL RESPONSE]",
"model_name": "M2",
"model_provider": "P2",
"score": [1-100],
"if_chosen": true,
"within_turn_id":0
},
{
"turn": 1,
"role": "model",
"content": "[MODEL RESPONSE]",
"model_name": "M2",
"model_provider": "P2",
"score": [1-100],
"if_chosen": false,
"within_turn_id":1
}
//... 随后各轮遵循与第 1 轮相同的模式
],
"performance_attributes": {
"fluency": [1-100],
"factuality": [1-100],
"helpfulness": [1-100],
//....更多属性评分
},
"open_feedback": "[FREE-TEXT]"
}

Appendix B Prism Data Statement

附录 B Prism 数据声明

We provide a data statement [69] to document the generation and provenance of Prism.

我们提供一份数据声明 [69]，用以记录 Prism 的生成过程与出处。

B.1 Curation Rationale

The Prism Alignment Project, funded by a variety of academic and industry sources (see Disclosure of Funding), aims to diversify human feedback datasets. All participants are recruited via the Prolific platform. The sample is described in Section 2.3, with additional details in Appendix J. The primary purpose of the dataset is for academic research into how different people interact with LLMs and perceive their outputs. However, we do not prohibit the use of the dataset to develop, test and/or evaluate AI systems so long as usage complies with the dataset license (Section C.2).

B.1 策展理据

Prism Alignment Project 由多种学术和产业来源资助（见资助披露），旨在使人类反馈数据集更加多样。所有参与者均通过 Prolific 平台招募。样本在第 2.3 节描述，更多细节见附录 J。该数据集的主要用途是学术研究：考察不同的人如何与 LLM 交互、又如何看待其输出。不过，只要使用遵守数据集许可（第 C.2 节），我们并不禁止将该数据集用于开发、测试和/或评估 AI 系统。

B.2 Language Variety

The language of human- or model-written text was not explicitly restricted to English. However, the task instructions were written English, and fluency in English was included as a screening filter. As a result of these factors, 99% of text instances are in English (see Appendix E for breakdowns per type of text instance and by other language). There is scope for wide social and regional variation even within a language. Given we have speakers residing in 38 countries (born in 75 countries), we likely have various forms of English, especially by level of fluency (see Table 5). Information about which varieties of English are represented is not available.

B.2 语言变体

人类或模型所写文本并未被明确限制为英语。不过任务说明是用英语写的，英语流利也被用作筛选条件。由于这些因素，99% 的文本实例是英语（按文本实例类型以及其他语言的分解见附录 E）。即便在同一种语言内部，也存在广泛的社会和地区变异。鉴于我们的说话者居住在 38 个国家（出生于 75 个国家），我们很可能包含多种英语形式，尤其是流利程度上的差异（见 Table 5）。关于哪些英语变体被代表，我们没有可用信息。

B.3 Speaker Demographics

There are two sets of “speaker” roles in Prism: human participants and large language models (LLMs). Both roles contribute to the characteristics of the text utterances in the dataset.

B.3 说话者人口统计

Prism 中有两组“说话者”角色：人类参与者和大语言模型（LLM）。两种角色都对数据集中文本语句的特征有贡献。

Participant Characteristics

We provide full demographic breakdowns of participant characteristics in Table 5. We provide full geographic breakdowns in Table 8. Despite substantial improvements on sample diversity compared to early widely-used human feedback datasets (see Table 6, Table 7), Prism still skews White, Educated, and Western. This is partly driven by census-representative samples from the US and UK, which can be removed or downsampled for future research. Prism only contains participants sourced from one crowdworking platform (Prolific), so inherits sample biases from this narrow pool—for example, participants are active internet users, incentivised by hourly payment on a specific task that they self-select into.

参与者特征

我们在 Table 5 中提供参与者特征的完整人口统计分解，在 Table 8 中提供完整地理分解。尽管相对早期广泛使用的人类反馈数据集，样本多样性已有明显改善（见 Table 6、Table 7），Prism 仍偏向白人、受过教育和西方。这在一定程度上是由美国和英国的普查代表性样本驱动的，未来研究可以将其移除或降采样。Prism 的参与者只来自一个众包平台（Prolific），因此继承了这一狭窄来源池的样本偏差——例如，参与者是活跃的互联网用户，受到按时计酬的激励，并自行选择进入一项特定任务。

Model Characteristics

Given fast-paced changes to the LLM landscape, Prism is designed to be as model-agnostic as possible. We include 21 models from various different families, capabilities and sizes (for a summary see ). 12/21 models are accessed via commerical APIs, and 9/21 are open-access via HuggingFace. Model-specific characteristics will affect the text characteristics, especially if they have already been alignment-tuned.

模型特征

鉴于 LLM 格局变化很快，Prism 被设计得尽可能与具体模型无关。我们纳入来自不同家族、能力和规模的 21 个模型（摘要见 ）。其中 12/21 个模型通过商业 API 访问，9/21 个通过 HuggingFace 开放获取。模型特有的特征会影响文本特征，尤其是当它们已经做过对齐微调时。

Models as Participants

Throughout the study we strongly requested that participants did not use LLMs to write their “human” responses, playing both to their integrity (please don’t do it), their role in the research (we really need you to not do it), and their incentives (you won’t be paid if you do it). We did not directly test nor implement tools to technologically prevent participants from using LLMs on their behalf. We randomly sample 25 instances from human-written texts: system strings and self-descriptions from the Survey; opening prompts and open feedback from the Conversations ($n=100$). An annotator (paper author) manually inspected these and labelled none as model-written text. For instances of sufficient length (46/100, >50 words), we recorded the predicted probability of AI-generated text from an LLM-text detector, where 76% had $\leq 1\%$ score.[^9] For the remainder ($n=11$), a second annotator (paper author) gave a tie-break, labelling none as model-generated.

把模型当作参与者

在整个研究中，我们强烈要求参与者不要用 LLM 来写他们的“人类”回复，同时诉诸他们的诚信（请不要这样做）、他们在研究中的角色（我们确实需要你们不要这样做）以及他们的激励（如果这样做就不会得到报酬）。我们没有直接测试，也没有部署技术工具来阻止参与者让 LLM 代劳。我们从人类撰写的文本中随机抽取 25 条实例：问卷中的系统字符串和自我描述；对话中的开场提示和开放反馈（$n=100$）。一名标注者（论文作者）人工检查了这些文本，没有一条被标为模型撰写。对足够长的实例（46/100，超过 50 词），我们记录了一个 LLM 文本检测器给出的 AI 生成文本预测概率，其中 76% 的分数 $\leq 1\%$。[^9] 对其余实例（$n=11$），第二名标注者（论文作者）做了决胜判定，没有一条被标为模型生成。

B.4 Annotator Demographics

The “annotators” are “speakers”—the same human participants who answer the survey, interact with the LLMs, and provide structured and unstructured feedback. See Section B.3.

B.4 标注者人口统计

这里的“标注者”就是“说话者”——同一批回答问卷、与 LLM 交互、并提供结构化和非结构化反馈的人类参与者。见第 B.3 节。

B.5 Speech Situation

All participants were recruited via Prolific. They were paid £9/hour. The survey was hosted on Qualtrics (www.qualtrics.com), and the conversations on Dynabench (www.dynabench.org).

B.5 言语情境

所有参与者均通过 Prolific 招募。他们的时薪为 9 英镑。问卷托管在 Qualtrics（www.qualtrics.com），对话托管在 Dynabench（www.dynabench.org）。

All data was collected between 22nd November 2023 and 22nd December 2023. The time of the data collection period did affect the topics of discussion: for example, one topic concerns Christmas holiday celebrations while another discusses the Israel–Palestine Conflict.

所有数据收集于 2023 年 11 月 22 日至 2023 年 12 月 22 日之间。数据收集时段确实影响了讨论话题：例如有一个话题涉及圣诞节假期庆祝，另一个则讨论以色列–巴勒斯坦冲突。

The primary modality of Prism is written language, combined with structured ratings or structured survey data. The conversations between participants and LLMs happened synchronously via live API connections with models in the backend of our interface. We have not edited or moderated any survey responses, participant prompts or model responses. All conversations happened as part of this research project, so the primary ‘intended audience’ was the researchers, though participants were informed of additional plans to distribute and release the data in the consent form (see Appendix D).

Prism 的主要模态是书面语言，并结合结构化评分或结构化问卷数据。参与者与 LLM 的对话通过我们界面后端与模型的实时 API 连接同步进行。我们没有编辑或审核任何问卷回答、参与者提示或模型回复。所有对话都作为本科研项目的一部分发生，因此主要的“预期受众”是研究者，不过参与者在知情同意书中已被告知还有分发和发布数据的额外计划（见附录 D）。

B.6 Text Characteristics

We summarise text characteristics in Appendix M. For the survey responses, the text provides details on the participant and their views about LLMs via short-form free-text responses (we requested 2-5 sentences in their own words). For the conversations, there are three different types: unguided, values guided and controversy guided, as described in the main paper (Section 2.2). Each conversation type contains a different distribution of topics. Overall, Prism is skewed towards subjective, values-driven and controversial dialogue. The human-written texts within a conversation typically consist of single sentence prompts, on average 13 words long. Prompts receive up to four model responses generated by a variety of LLMs. We instruct the LLMs to limit their response to 50 words or less. Most unsuccessfully abide by this instruction: the average response length is 89 words. We release metadata (see Appendix E) with each text instance including information on detected language, automated and manual PII checks and moderation flags (e.g. if it contains sexual, hateful or violent content).

B.6 文本特征

我们在附录 M 中汇总文本特征。对问卷回答而言，文本通过简短的自由文本回复（我们请他们用自己的话写 2–5 句）提供关于参与者及其对 LLM 看法的细节。对对话而言，有三种不同类型：无引导、价值引导和争议引导，如正文所述（第 2.2 节）。每种对话类型包含不同的话题分布。总体来看，Prism 偏向主观、由价值驱动以及具有争议性的对话。一场对话中的人类文本通常是单句提示，平均 13 个词。提示会收到由多种 LLM 生成的最多四条模型回复。我们指示 LLM 把回复限制在 50 词或更短。大多数模型未能遵守这一指示：平均回复长度为 89 词。我们为每个文本实例发布元数据（见附录 E），包括检测到的语言、自动与人工 PII 检查以及审核标记（例如是否包含色情、仇恨或暴力内容）。

B.7 Recording Quality

During data collection, our interface experienced two distributed denial of service (DDoS) attacks: one on 28th November 2023 and another on 1st December 2023. The primary way that these attacks may have affected recording quality was via interrupting participants’ conversation sessions (most then later returned to the interface to complete their conversations a couple hours or days later). These participants’ data points may differ to those who had a smoother continuous experience in the task.

B.7 记录质量

在数据收集期间，我们的界面经历了两次分布式拒绝服务（DDoS）攻击：一次在 2023 年 11 月 28 日，另一次在 2023 年 12 月 1 日。这些攻击可能影响记录质量的主要方式，是打断参与者的对话会话（其中大多数人后来会在几小时或几天后回到界面完成对话）。这些参与者的数据点，可能不同于那些任务体验更连续、更顺畅的人。

B.8 Author Characteristics and Positionality Statement

We aimed to operate in the subjective paradigm [15, 16] and have as little influence as possible on how participants interacted with models (e.g. no annotation guidelines for how to rate responses). As a team of researchers, we come from a variety of backgrounds (genders, ethnicities, countries of birth, native languages) and are involved with AI research, either in an academia (6/12) or industry (6/12).

B.8 作者特征与立场声明

我们希望在主观范式下运作 [15, 16]，并尽可能少地影响参与者如何与模型交互（例如，不为如何给回复打分提供标注指南）。作为一个研究团队，我们来自多样的背景（性别、族裔、出生国、母语），并从事 AI 研究，要么在学术界（6/12），要么在产业界（6/12）。

Privacy and deanonymisation

The conversations in Prism are highly personal, for example detailing views towards abortion, religion, immigration, workplace disputes or intimate relationships. We have pseudo-anonymised the data, checked for PII (Appendix E), sought informed consent from every participant (Appendix D), provided options for participants to withdraw their data, and clearly stipulated that attempts of deanonymisation violate our dataset’s terms and conditions (Appendix C). However, despite following these best practices, the risk for deanonymisation remains. We include a reporting mechanism on our website and GitHub for any participants and researchers to report issues.

隐私与去匿名化

Prism 中的对话相当个人化，例如详述对堕胎、宗教、移民、职场纠纷或亲密关系的看法。我们已对数据做了假名化处理，检查了 PII（附录 E），向每位参与者征求知情同意（附录 D），为参与者提供撤回数据的选项，并明确约定试图去匿名化违反我们数据集的条款（附录 C）。不过，尽管遵循了这些最佳实践，去匿名化的风险仍然存在。我们在网站和 GitHub 上提供了报告机制，供参与者和研究者报告问题。

Harmful and unsafe content

We asked participants to engage the LLMs in controversial conversations. This comes with the benefit of expanding human preference data to discursive areas with the greatest expected degree of interpersonal disagreement, but at the risk of encouraging hateful, bigoted, biased or otherwise harmful content. Harmful content is an issue in other human feedback datasets, where some opt to moderate conversations prior to public release [113] and others retain toxic content for the purpose of future research into conversational AI safety [112, 111]. Compared to these previous datasets, Prism has an exceptionally low level of flagged content as measured via the OpenAI moderation API (0.06% overall, and $<0.003\%$ for subcategories of sexually-explicit, violent, hateful, self-harm and harassment). However, the recall of this API may be low [111]; so, this could be an underestimate. From examining prompts closest to topic centroids (Section R.2), it is clear there are some prompts with potential for harm. We provide metadata for every text instance in Prism, and opt to not filter any conversations. We believe it is a critical area of research to understand how state-of-the-art models respond when they are prompted to engage in such conversations, and how different people with diverse lived experiences react to safety interventions.

有害与不安全内容

我们请参与者与 LLM 进行有争议的对话。这样做的好处是把人类偏好数据扩展到预期人际分歧最大的话语领域，但风险是鼓励仇恨、偏见、偏颇或其他有害内容。有害内容在其他人类反馈数据集中也是问题：有些选择在公开发布前审核对话 [113]，另一些则为了未来的对话 AI 安全研究而保留有毒内容 [112, 111]。与这些先前数据集相比，按 OpenAI moderation API 衡量，Prism 被标记的内容水平异常低（总体 0.06%，对色情露骨、暴力、仇恨、自伤和骚扰等子类则 $<0.003\%$）。不过该 API 的召回率可能偏低 [111]；因此这可能是低估。从最接近话题质心的提示来看（第 R.2 节），显然有些提示具有造成伤害的可能。我们为 Prism 中的每个文本实例提供元数据，并选择不过滤任何对话。我们认为，理解最先进的模型在被提示进行此类对话时如何回应，以及有着多样生活经历的不同的人如何对安全干预作出反应，是一个关键的研究领域。

Participation-washing and intended societal impact

In our setting, we claim what Sloane et al. 2022 calls participation as work, that is offering fair remuneration and attribution of the consensual labour of workers contributing to our project. Notably, many participants (those familiar and unfamiliar with AI) contacted the researchers and reported enjoying or learning from the task, suggesting there was an “education quotient” or role of participation as experience [53]. Compared to “passive” participation in annotation tasks or pre-training datasets [33], our process is more active for participants because it foregrounds the opportunity to provide their feedback, opinions and preferences, not just labels. “Participatory” also signals our goal to have communities more involved in alignment fine-tuning of models and see Prism as a first step demonstrating this need. These aims evoke notions of participation as justice—including more people at the table of LLM design and development but we note that participation is in reality thin, because while we seek their view, we cannot grant participants the power to change behaviours of deployed LLMs [131]. Even the etymological roots of participation centre on the notion of “sharing” [53] but there is no guarantee that the human workers upon whom the success of RLHF relies on, partake in any share of the profits from more usable or preferred LLM technologies. We release Prism in the hope it moves the needle towards more inclusive and diverse research on human-AI interactions, emphasising the central role of those who contribute their time and voice to generating human feedback data. Ultimately, how these contributions have impact depends on those in power (industry labs, academics, policymakers), because “the experience of participation must include the sense not only of having spoken, but of having been heard” [53, p.18,].

参与漂洗与预期社会影响

在我们的设定中，我们主张的是 Sloane 等人 2022 所称的作为工作的参与，即为向项目贡献劳动、且出于自愿的工作者提供公平报酬和署名。值得注意的是，许多参与者（无论是否熟悉 AI）联系了研究者，并表示享受这项任务或从中有所学习，表明其中存在一种“教育分量”，或作为体验的参与的角色 [53]。相比标注任务或预训练数据集中的“被动”参与 [33]，我们的过程对参与者更为主动，因为它把提供反馈、意见和偏好——而不只是标签——的机会放在前台。“参与式”也标示我们的目标：让社区更多地参与模型的对齐微调，并把 Prism 视为展示这一需求的第一步。这些目标唤起了作为正义的参与的理念——让更多人坐到 LLM 设计与开发的桌前——但我们注意到，参与在现实中仍然很薄，因为虽然我们征求他们的看法，却无法赋予参与者改变已部署 LLM 行为的权力 [131]。即便 participation 的词源也围绕“分享”这一概念 [53]，却无法保证 RLHF 成功所依赖的那些人类工作者，能够分享来自更可用或更受偏好的 LLM 技术的任何利润。我们发布 Prism，希望它能推动关于人机交互的研究走向更加包容和多样，并强调那些贡献时间和声音来生成人类反馈数据的人的中心角色。归根结底，这些贡献如何产生影响取决于掌权者（产业实验室、学者、政策制定者），因为“参与的体验必须不仅包括已经发过言，也包括已经被听见” [53, p.18]。

The curse of dimensionality (or intersectionality)

Our findings suggest dialogue and model choice are driven somewhat by group affiliation and somewhat by idiosyncratic variance. However, Prism contains a rich array of information on each participant with both structured and unstructured components. There are endless ways we could have divided the data or understood participant identity, and despite our best efforts to assess sensitivity to design choices, each alternative may have resulted in very different outcomes [120], and we are under-powered to test so many sparse combinations. Using less sparse groupings introduces biases—for example, focusing on region risks lumping together participants from particular geographies as “cultures” [82]. While we split out the UK and US to avoid these countries dominating their respective regions, there remain varying degrees of country-wise entropy in other regions—the Middle East has 94% individuals from Israel, and 100% of Non-US Northern Americans are Canadian (see Appendix H). Similarly, we use more aggregated ethnicity and religion groupings for statistical power, but amorphous and heterogeneous categories like “Other” have limited or flawed real-world meaning as “Other” contains, for example, both those who identify as Indigenous or First Peoples and as Middle Eastern or Arab. It is an exciting direction for future work to explore free-form characterisations of identity (e.g. the free-text profile or system string) or ex-post groupings of people’s preferences [9], and examine how findings change when we break away from neatly-observed but essentialising demographic traits [133].

维度诅咒（或交叉性）

我们的发现表明，对话和模型选择在一定程度上由群体归属驱动，在一定程度上由特异差异驱动。不过，Prism 对每位参与者都包含丰富的信息，既有结构化成分也有非结构化成分。划分数据或理解参与者身份的方式无穷无尽；尽管我们尽力评估对设计选择的敏感性，每一种替代做法都可能带来非常不同的结果 [120]，而我们没有足够的效力去检验如此多稀疏组合。使用不那么稀疏的分组又会引入偏差——例如，聚焦地区有可能把特定地理来源的参与者笼统当作“文化” [82]。虽然我们把英国和美国单独列出，以免这两个国家主导各自所在地区，其他地区仍然存在不同程度的国别熵——中东有 94% 的个体来自以色列，而非美国的北美洲则 100% 是加拿大人（见附录 H）。类似地，为了统计效力，我们使用更汇总的族裔和宗教分组，但像“其他”这样模糊而异质的类别作为现实类别的意义有限或有缺陷：例如“其他”既包含认同为原住民或第一民族的人，也包含认同为中东人或阿拉伯人的人。探索身份的自由形式刻画（例如自由文本档案或系统字符串），或对人们偏好做事后分组 [9]，并考察当我们摆脱整齐可观察却本质化的人口统计特征时发现会如何变化 [133]，是未来工作中令人兴奋的方向。

The confounding effect of many moving cogs in a conversation

Beyond the complexities of intersectional identity and idiosyncratic variance of individuals within identity groups, other sources of variance in Prism present a challenge for controlled experiments; particularly, the high-dimensionality of what exact topics each participant chooses to talk about, which models randomly get selected in-the-loop, and the stochasticity in their responses from a non-deterministic temperature. It is hard to pin down robust mechanisms of preference differences amongst individuals with so many sources of variation. We opted for choice of input prompt and conversation to be a free parameter in Prism as a more naturalistic setting of LLM use and because we wanted to understand dialogue diversity among participants. We do empirically find some regions of fixed prompt-response pairs from individuals who self-select into asking the same prompts as other participants (see Section S.4).

一场对话中许多活动部件的混杂效应

除了交叉身份的复杂性以及身份组别内部个体的特异差异之外，Prism 中的其他差异来源也给受控实验带来挑战；尤其是每位参与者究竟选择谈论哪些话题的高维性、循环中随机选中哪些模型，以及非确定性温度带来的回复随机性。差异来源如此之多，很难确定个体间偏好差异的稳健机制。我们选择把输入提示和对话当作 Prism 中的自由参数，既是为了更接近 LLM 使用的自然情境，也是因为我们想理解参与者之间的对话多样性。我们确实在经验上找到一些固定的提示–回复对区域，来自那些自行选择去问与其他参与者相同提示的个体（见第 S.4 节）。

Noisy signals and misaligned incentives

Relatedly, our conclusions may be confounded by measurement invariance given our explicit focus on subjective, fluid and cardinal devices. This echos the economist’s view, that it is foolish to rely too heavily on cardinal ratings over ordinal rankings to make interpersonal comparisons, or enforce preference construction, where intrinsic feelings are noisily-quantified on numeric scales. There are also issues of preference falsification: while participants are financially incentivised to participate, they may not honestly report their preferences over models. We cannot rule out the possibility that participants select a ‘bad’ model to lock in for the subsequent turns of conversation if it is more interesting (thus preferable in our narrow task confines) to talk to a more offensive or controversial model, or to try to ‘jailbreak it’ [112]. In hindsight, it may have been a smarter design choice to force participants to rank model responses, or to collect both ratings and rankings (notwithstanding decision fatigue), or make attempts to elicit more interpersonally comparable data via a willingness-to-pay monetary unit. Previous work also raises concerns over relying on human feedback as ‘gold standard’, for example whether participants can accurately rate factuality of an output, or are anchored on formatting and ‘first impressions’ (as we and Hosking et al. 2024 both find). Preferences, especially at a fine-grained level like in Prism, have high context-dependency [135], so we caution against taking the ratings as revealing some objective truth, instead staying firmly rooted in the subjective paradigm [15, 2].

嘈杂信号与错位的激励

与此相关，鉴于我们明确聚焦主观、流动和基数工具，我们的结论可能受到测量不变性的混淆。这呼应了经济学家的看法：过度依赖基数量表而非序数排序来做人际比较，或去强制进行偏好构造——也就是把内在感受嘈杂地量化到数字量表上——是不明智的。此外还有偏好伪装的问题：虽然参与者有金钱激励去参与，他们未必会诚实地报告对模型的偏好。我们无法排除这种可能：参与者选出一个“差”模型锁定到随后各轮，如果在我们狭窄的任务范围内，与一个更具冒犯性或争议性的模型交谈、或试图“越狱”它 [112]，会更有趣（因而更可取）。事后看来，强制参与者给模型回复排序，或同时收集评分和排序（尽管会有决策疲劳），或尝试通过愿意支付的货币单位来诱发更可人际比较的数据，也许会是更明智的设计选择。先前工作也提出把人类反馈当作“金标准”的担忧，例如参与者能否准确评价输出的事实性，或是否被格式和“第一印象”所锚定（我们与 Hosking 等人 2024 都发现了这一点）。偏好，尤其是 Prism 中这种细粒度的偏好，具有高度的语境依赖性 [135]，因此我们提醒不要把这些评分当作揭示某种客观真理，而应牢牢停留在主观范式中 [15, 2]。

Still the “tyranny of the (English-speaking) crowdworker”

Much of AI, NLP and now RLHF is underpinned by crowdworker labour [136]. Despite our aims to include more diverse voices in LLM development processes, we avoid overstating claims on diversity. Prism still only contains crowdworkers, who have significant sample biases [137]; can only be so “representative” given the relatively small sample sizes; must be digital natives given the platformed nature of the work; and possess different incentives for engagement [138]. Furthermore, while Prism gains some dialectical diversity from different geographies of English, from varying speaker fluency, and from some contributions in other languages (1%, mainly Spanish), it is almost exclusively in English. Cultural diversity can only be measured so far without also accounting for linguistic diversity [61]. Furthermore, while we try to sample from many regions, our sample is still dominated by White Western participants, especially when considering cultural phylogeny [82], i.e., the non-independence of populations with shared history or migrations of peoples (for example, Australia vs UK vs Canada). We encourage future work prioritising human feedback collection in other languages to understand how models handle sociocultural and linguistic interactions [115].

仍然是“（英语）众包工作者的暴政”

许多 AI、自然语言处理以及如今的 RLHF，都以众包劳动为底盘 [136]。尽管我们的目标是在 LLM 开发过程中纳入更多样的声音，我们避免在多样性上做出过度的主张。Prism 仍然只包含众包工作者，他们有显著的样本偏差 [137]；在相对较小的样本量下也只能做到有限的“代表性”；鉴于工作的平台化性质，他们必须是数字原住民；并且拥有不同的参与激励 [138]。此外，虽然 Prism 从不同地理的英语、从说话者流利程度的差异、以及从少量其他语言的贡献（1%，主要是西班牙语）中获得了一些方言多样性，它几乎完全是英语。若不考虑语言多样性，文化多样性只能被度量到有限的程度 [61]。另外，尽管我们试图从许多地区抽样，样本仍由白人西方参与者主导，尤其是在考虑文化系统发育 [82] 时，也就是有着共享历史或人群迁徙的人群并非相互独立（例如澳大利亚 vs 英国 vs 加拿大）。我们鼓励未来工作优先以其他语言收集人类反馈，以理解模型如何处理社会文化与语言的交互 [115]。

The ever-changing stream of pre-aligned models

When data collection began in mid-November, Prism contained the top ranking models on publicly available leaderboards but new models have since emerged, including Gemini [139], Mixtral [140], Claude-3 [141], Command-R [142] and Llama-3 [143]. There is an incompatibility between the current pace of model releases and doing human participant research that requires lengthy processes of ethics approval, interface design, data processing and manual annotation. The expense and inconvenience of doing human research increases the attractiveness of simulating responses, usually with GPT-4 [108]. So, while Prism does miss out on the newest players to enter the battle arena, we do provide carefully-sourced human data (including a survey which stands independently from the LLM conversations) combined with a wide distribution of model texts; so we hope the utility of the data persists in the coming years even as models change. We are still potentially limited when comparing open and closed-access models: while the former allows full transparency over system prompts, closed-access models can obscure additional instructions as hidden context. Including models from the same family allows comparisons by version or size, but introducing clones (models producing very similar outputs) can distort preference rankings [9]. Prism is also limited by value-lock in [108]—the models are already tuned to cultural perspectives or alignment norms [34, 35], which precludes observing certain group preferences towards a wider set of behaviours [37, 144], and renders participants “thin” because they are “limited to existing designs with pre-existing purposes.” [25, p.3,].

不断变化的预对齐模型之流

当数据收集在 11 月中旬开始时，Prism 包含了公开排行榜上名列前茅的模型，但此后又出现了新模型，包括 Gemini [139]、Mixtral [140]、Claude-3 [141]、Command-R [142] 和 Llama-3 [143]。当前的模型发布速度，与需要漫长伦理审批、界面设计、数据处理和人工标注的人类参与者研究之间并不匹配。做人类研究的费用与不便，增加了用模拟回复（通常用 GPT-4 [108]）的吸引力。因此，虽然 Prism 错过了进入对战场的最新选手，我们确实提供了仔细采集的人类数据（包括一份独立于 LLM 对话的问卷），并结合了分布广泛的模型文本；因此我们希望即便模型发生变化，数据在未来几年仍保持用处。在比较开放获取与封闭获取模型时我们仍可能受限：前者允许对系统提示完全透明，封闭获取模型则可能把额外指令作为隐藏语境掩盖起来。纳入同一家族的模型允许按版本或规模比较，但引入克隆（产生非常相似输出的模型）可能扭曲偏好排名 [9]。Prism 也受价值锁定 [108] 的限制——这些模型已经被调节到特定的文化视角或对齐规范 [34, 35]，这使我们无法观察某些群体对更广行为集合的偏好 [37, 144]，也使参与者变得“薄”，因为他们“被限制在已有设计及其既定目的之中。”[25, p.3]。

Appendix C Prism Data Clause

附录 C Prism 数据条款

Purpose

The Dataset is provided for the purpose of research and educational use in the field of natural language processing, conversational agents, social science and related areas; and can be used to develop or evaluate artificial intelligence, including Large Language Models (LLMs).

目的

本数据集提供用于自然语言处理、对话智能体、社会科学及相关领域的研究和教育用途；也可用于开发或评估人工智能，包括大语言模型（LLM）。

Usage Restrictions

Users of the Dataset should adhere to the terms of use for a specific model when using its generated responses. This includes respecting any limitations or use case prohibitions set forth by the original model’s creators or licensors.

使用限制

使用数据集中某模型生成的回复时，用户应遵守该特定模型的使用条款。这包括尊重原模型创建者或许可方设定的任何限制或用例禁令。

Content Warning

The Dataset contains raw conversations that may include content considered unsafe or offensive. Users must apply appropriate filtering and moderation measures when using this Dataset for training purposes to ensure the generated outputs align with ethical and safety standards.

内容警告

本数据集包含原始对话，其中可能包括被视为不安全或冒犯性的内容。用户在将本数据集用于训练时必须采取适当的过滤和审核措施，以确保生成的输出符合伦理与安全标准。

No Endorsement of Content

The conversations and data within this Dataset do not reflect the views or opinions of the Dataset creators, funders or any affiliated institutions. The dataset is provided as a neutral resource for research and should not be construed as endorsing any specific viewpoints.

不背书内容

本数据集中的对话和数据并不反映数据集创建者、资助方或任何相关机构的观点或意见。该数据集作为中立的研究资源提供，不应被理解为对任何特定观点的背书。

No Deanonymisation

The User agrees not to attempt to re-identify or de-anonymise any individuals or entities represented in the Dataset. This includes, but is not limited to, using any information within the Dataset or triangulating other data sources to infer personal identities or sensitive information.

禁止去匿名化

用户同意不试图重新识别或去匿名化数据集中所代表的任何个人或实体。这包括但不限于：使用数据集内的任何信息，或与其他数据源交叉比对，以推断个人身份或敏感信息。

Limitation of Liability

The authors and funders of this Dataset will not be liable for any claims, damages, or other liabilities arising from the use of the dataset, including but not limited to the misuse, interpretation, or reliance on any data contained within.

责任限制

本数据集的作者和资助方不对因使用该数据集而产生的任何索赔、损害或其他责任负责，包括但不限于对其中任何数据的误用、解释或依赖。

C.2 Licence and Attribution

Human-written texts (including prompts) within the dataset are licensed under the Creative Commons Attribution 4.0 International License (CC-BY-4.0). Model responses are licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC-BY-NC-4.0). Use of model responses must abide by the original model provider licenses.

C.2 许可与署名

数据集中的人类撰写文本（包括提示）采用知识共享署名 4.0 国际许可（CC-BY-4.0）。模型回复采用知识共享署名-非商业性使用 4.0 国际许可（CC-BY-NC-4.0）。使用模型回复必须遵守原模型提供方的许可。

For proper attribution when using this dataset in any publications or research outputs, please cite with the DOI. Suggested Citation: Kirk, H. R., Whitefield, A., Röttger, P., Bean, A., Margatina, K., Ciro, J., Mosquera, R., Bartolo, M., Williams, A., He, H., Vidgen, B., & Hale, S. A. (2024). The PRISM Alignment Dataset. https://doi.org/10.57967/hf/2113

在任何出版物或研究成果中使用本数据集时，请用 DOI 正确署名。建议引用：Kirk, H. R., Whitefield, A., Röttger, P., Bean, A., Margatina, K., Ciro, J., Mosquera, R., Bartolo, M., Williams, A., He, H., Vidgen, B., & Hale, S. A. (2024). The PRISM Alignment Dataset. https://doi.org/10.57967/hf/2113

C.3 Dataset Maintenance

As the authors and maintainers of this dataset, we commit to no further updates to the dataset following its initial release. The dataset is self-contained and does not rely on external links or content, ensuring its stability and usability over time without the need for ongoing maintenance.

C.3 数据集维护

作为本数据集的作者和维护者，我们承诺在首次发布后不再对数据集做进一步更新。该数据集是自包含的，不依赖外部链接或内容，从而在无需持续维护的情况下保证其长期稳定和可用性。

C.4 Data Rights Compliance and Issue Reporting

We are committed to complying with data protection rights, including but not limited to regulations such as the General Data Protection Regulation (GDPR). If any individual included in the dataset wishes to have their data removed, we provide a straightforward process for issue reporting and resolution on our Github. Concerned parties are encouraged to contact the authors directly via the provided contact form link on the Github. Upon receiving a request, we will engage with the individual to verify their identity and proceed to remove the relevant entries from the dataset. We commit to addressing and resolving such requests within 30 days of verification.

C.4 数据权利合规与问题报告

我们致力于遵守数据保护权利，包括但不限于通用数据保护条例（GDPR）等法规。如果数据集中包含的任何个人希望移除其数据，我们在 Github 上提供了直接的问题报告与处理流程。相关方可通过 Github 上提供的联系表单链接直接联系作者。收到请求后，我们将与该个人接洽以核实身份，并着手从数据集中移除相关条目。我们承诺在核实后 30 天内处理和解决此类请求。

Appendix D Informed Consent

附录 D 知情同意

This research was reviewed by, and received ethics clearance through, a subcommittee of the University of Oxford Central University Research Ethics Committee [OII_C1A_23_088]. The following text was displayed to all participants to collect informed consent.

本研究已由牛津大学中央大学研究伦理委员会的一个分委会审查并获得伦理许可 [OII_C1A_23_088]。以下文本向所有参与者展示，用以收集知情同意。

Your Feedback on AI Language Models

你对 AI 语言模型的反馈

We appreciate your interest in participating in this study. The aim of this research is to understand people’s preferences and perceptions regarding AI Language Model behaviours, also referred to as Large Language Models (LLMs), Generative AI Language Models, AI ChatBots or Virtual Assistants. AI language models are computer programs designed to generate text. They can respond to questions or prompts by producing written responses. We want to learn more about how people like you use and perceive these AI language models.

感谢你有兴趣参与本研究。本研究的目的是理解人们对 AI 语言模型行为的偏好与看法，这类模型也被称为大语言模型（LLM）、生成式 AI 语言模型、AI 聊天机器人或虚拟助手。AI 语言模型是设计用来生成文本的计算机程序。它们可以通过产生书面回复来回应问题或提示。我们希望更多地了解像你这样的人如何使用并看待这些 AI 语言模型。

Please first make sure you are using a laptop or desktop computer, and you are not using a mobile device. Our task is NOT compatible with mobile devices.

请先确保你使用的是笔记本电脑或台式电脑，而不是移动设备。我们的任务与移动设备不兼容。

Please then read through this information before agreeing to participate (if you wish to).

然后请在同意参与（如果你愿意的话）之前阅读这些信息。

You may ask any questions before deciding to take part by contacting the research team. The Principal Researcher is Hannah Rose Kirk, and the Principle Investigator is Dr Scott. A. Hale, who are both affiliated with the Oxford Internet Institute at the University of Oxford.

在决定是否参与之前，你可以通过联系研究团队提出任何问题。首席研究员是 Hannah Rose Kirk，项目负责人是 Scott. A. Hale 博士，两人均就职于牛津大学牛津互联网研究所。

What does the task involve?

If you decide to participate, there are two stages.

任务包含什么？

如果你决定参与，共有两个阶段。

In this stage, you will be asked to fill in a short survey about yourself and your thoughts on AI language models.

在这一阶段，你将被要求填写一份关于你自己以及你对 AI 语言模型看法的简短问卷。

In the next stage, you will have conversations with AI language models by providing prompts and rating their responses using a user-friendly interface. The prompts can be on various topics, and you don’t need any specific knowledge to participate. Your input will help us understand your preferences and opinions about how these AI language models work.

在下一阶段，你将通过一个便于使用的界面，向 AI 语言模型提供提示并给它们的回复打分，从而与它们对话。提示可以涉及各种话题，你不需要任何特定知识即可参与。你的输入将帮助我们理解你对这些 AI 语言模型如何工作的偏好和看法。

Both stages should take between 55-65 minutes. No background knowledge is required.

两个阶段合计应在 55–65 分钟之间。 不要求任何背景知识。

Please note that you will be interacting with an AI language model. The research team cannot directly control and are not responsible for the text generated by these models. There is a possibility that the models produce biased, inaccurate or harmful language. The risks to you as an individual are equivalent to those you would be exposed to if you use AI language models via interfaces like ChatGPT.

请注意，你将与一个 AI 语言模型交互。研究团队无法直接控制这些模型生成的文本，也不对其负责。模型有可能产生有偏见、不准确或有害的语言。对你作为个体的风险，相当于你通过 ChatGPT 这类界面使用 AI 语言模型时会面临的风险。

Do I have to take part?

No, participation is voluntary. If you do decide to take part, you may stop at any point for any reason before submitting your answers by closing the browser. However, we are only able to pay participants who complete the task. For demographic information, we have included a ‘Prefer not to say’ option for each set of questions should you prefer not to answer a particular question.

我必须参与吗？

不必，参与是自愿的。如果你决定参与，可以在提交回答之前的任何时候、出于任何原因，通过关闭浏览器停止。不过我们只能向完成任务的参与者支付报酬。对于人口统计信息，每组问题都包含“不愿透露”选项，如果你不想回答某一问题可以使用。

Can I withdraw my participation and data?

Yes, you may stop the study at any time. Please note that if you withdraw within a stage of the study you will not be paid for that stage or any subsequent incomplete stages, but you will be paid for any stages that you have already completed. You can withdraw your data from the study. The cut-off date for withdrawing your data is 14 days after you submitted the data. Please email members of the research team (see contact details below) within this 14-day window to withdraw your data from the study.

我可以撤回参与和数据吗？

可以，你可以随时停止研究。请注意，如果你在某一阶段中途退出，将不会获得该阶段及随后未完成阶段的报酬，但已经完成的阶段会获得报酬。你可以从研究中撤回自己的数据。撤回数据的截止日期是你提交数据后 14 天。请在这 14 天窗口内给研究团队成员发邮件（联系方式见下方），以从研究中撤回你的数据。

How will my data be used?

The data collected from your participation will be pseudo-anonymized (stored with a unique numeric ID) and stored securely. It will be used for research purposes, and your personal information will remain confidential. The data will be analysed to gain insights into diverse preferences and perceptions regarding AI language model behaviours. At the end of the study, the pseudo-anonymised data collected will be released publicly for future research. The findings of this study may be published in academic journals or presented at conferences, and the results will be written up for a DPhil degree. Your individual identity will not be disclosed at any point in data release or publication. We do not collect any personal, private identifying information, IP addresses or contact details. The data we will collect that could identify you will be some demographic information (gender, age, nationality, religion, etc.), and short self-written survey answers.

我的数据将如何被使用？

从你的参与中收集的数据将被假名化（用唯一的数字 ID 存储）并安全保存。它将用于研究目的，你的个人信息将保持保密。数据将被分析，以洞察人们对 AI 语言模型行为的多样偏好和看法。研究结束时，经过假名化的数据将被公开发布，供未来研究使用。本研究的发现可能发表在学术期刊或在会议上报告，结果也将写入一篇 DPhil 学位论文。在数据发布或发表的任何环节都不会披露你的个人身份。我们不收集任何个人的、可识别身份的私人信息、IP 地址或联系方式。我们收集的、有可能识别你的数据，将是一些人口统计信息（性别、年龄、国籍、宗教等），以及简短的自写问卷回答。

The responses you provide will be stored in a password-protected electronic file on University of Oxford secure servers and may be used in academic publications, conference presentations or reports for external organisations. We will release a clean, PII-checked and pseudo-anonymised form of the data on an open-access, public data repository. Raw research data will be stored for 3 years after publication or public release of the research. We would like to use the data in future studies, and to share data with other researchers (e.g. in online databases). Data will have identifying information removed before it is shared with other researchers or results are made public. The data that we collect from you may be transferred to, stored and/ or processed at a destination outside the UK and the European Economic Area. By submitting your personal data, you agree to this transfer, storing or processing.

你提供的回答将以密码保护的电子文件形式存储在牛津大学的安全服务器上，并可能用于学术出版物、会议报告或给外部组织的报告。我们将在一个开放获取的公共数据仓库上发布经过清理、PII 检查和假名化处理的数据。原始研究数据将在研究成果发表或公开发布后保存 3 年。我们希望在未来研究中使用这些数据，并与其他研究者分享数据（例如在在线数据库中）。在与其他研究者分享或公开结果之前，数据中的识别信息将被移除。我们从你那里收集的数据可能被传输到、存储于和/或处理于英国和欧洲经济区以外的目的地。提交你的个人数据，即表示你同意此种传输、存储或处理。

Who has reviewed this research?

This research has been reviewed by, and received ethics clearance through, a subcommittee of the University of Oxford Central University Research Ethics Committee [OII_C1A_23_088].

谁审查了这项研究？

本研究已由牛津大学中央大学研究伦理委员会的一个分委会审查并获得伦理许可 [OII_C1A_23_088]。

Who do I contact if I have a concern or I wish to complain?

If you have a concern about any aspect of this research, please speak to Hannah Rose Kirk (hannah.kirk@oii.ox.ac.uk) or their supervisor Dr. Scott A. Hale (scott.hale@oii.ox.ac.uk), and we will do our best to answer your query. We will acknowledge your concern within 10 working days and give you an indication of how it will be dealt with. If you remain unhappy or wish to make a formal complaint, please contact the Chair of the Research Ethics Committee at the University of Oxford who will seek to resolve the matter as soon as possible: Social Sciences & Humanities Interdivisional Research Ethics Committee; Email: ethics@socsci.ox.ac.uk; Address: Research Services, University of Oxford, Boundary Brook House, Churchill Drive, Headington, Oxford OX3 7GB.

Please note that you may only participate in this survey if you are 18 years of age or over.

请注意，只有年满 18 岁或以上，你才可以参与本问卷。

$\bigcirc$ I certify that I am 18 years of age or over

$\bigcirc$ 我确认自己年满 18 岁或以上

If you have read the information above and agree to participate with the understanding that the data (including any personal data) you submit will be processed accordingly, please tick the box below to start.

如果你已阅读以上信息，并同意在理解所提交的数据（包括任何个人数据）将据此处理的前提下参与，请勾选下方方框以开始。

$\bigcirc$ Yes, I agree to take part

$\bigcirc$ 是的，我同意参与

Appendix E Metadata Processing

附录 E 元数据处理

For each text instance in Prism, we attach three pieces of metadata: detected language flags, detected private or personally identifiable information (PII) flags, and detected moderation flags.

对 Prism 中的每个文本实例，我们附加三类元数据：检测到的语言标记、检测到的私人或个人可识别信息（PII）标记，以及检测到的审核标记。

E.1 Structuring the Metadata

There are five types of text instances. Two appear in the survey (self_description, system_string) and have a 1:1 matching with each user (user_id). One appears at the conversation level (open_feedback) and has a 1:1 matching with each convo_id and a many:1 matching with each user_id because each participant has multiple conversations. Finally, the last two occur within each turn of a conversation, where for each single user_prompt there are multiple model responses (model_response). We structure the metadata so it can be merged uniquely, without duplication. We release one file, where each text instance is tied to its metadata via the identifying information shown in Table 1, and a column_id for matching whether the text is [system_string, self_description, user_prompt, model_response, open_feedback].

E.1 组织元数据

文本实例有五种类型。两种出现在问卷中（self_description、system_string），并与每个用户（user_id）呈 1:1 对应。一种出现在对话层面（open_feedback），与每个 convo_id 呈 1:1 对应，并与每个 user_id 呈多对一对应，因为每位参与者有多场对话。最后两种出现在对话的每一轮中，每条 user_prompt 对应多条模型回复（model_response）。我们组织元数据，使其可以唯一合并、不产生重复。我们发布一个文件，其中每个文本实例通过 Table 1 所示的识别信息与其元数据绑定，并用 column_id 来匹配该文本属于 [system_string, self_description, user_prompt, model_response, open_feedback] 中的哪一种。

		user	$\overbrace{\makebox[20.00003pt]{}}^{\text{\rotatebox{90.0}{{interaction}}}}$	within turn
Text Instance	Study Stage	convo	turn
self_description	Survey	✓
system_string	Survey	✓
user_prompt	Conversations		✓	✓
model_response	Conversations		✓	✓	✓
open_feedback	Conversations		✓

Table 1: Identifiers of text instance types in Prism.

		用户	$\overbrace{\makebox[20.00003pt]{}}^{\text{\rotatebox{90.0}{{交互}}}}$	轮内
文本实例	研究阶段	convo	turn
self_description	问卷	✓
system_string	问卷	✓
user_prompt	对话		✓	✓
model_response	对话		✓	✓	✓
open_feedback	对话		✓

表 1：Prism 中各类文本实例的标识符。

PII

To identify whether a textual instance in our dataset contains personal and identifiable information (PII) we used the package scrubadub.[^10] Specifically we used the function scrubadub.clean(text) which replaces the phone numbers and email addresses with anonymous IDs, if they are found in the input. We flag with $1$ instances that are altered (i.e., PII was identified) and $0$ those that remained unchanged.

PII

为识别数据集中某个文本实例是否包含个人可识别信息（PII），我们使用了 scrubadub 软件包。[^10] 具体来说，我们使用函数 scrubadub.clean(text)，它会在输入中发现电话号码和电子邮件地址时将其替换为匿名 ID。我们对被改动的实例（即识别出了 PII）标记为 $1$，对保持不变的实例标记为 $0$。

Moderation

To measure content moderation we use the OpenAI Moderation endpoint.[^11] The API takes as an input a textual instance and outputs a json file with an overall boolean flag (flagged) whether there input potentially harmful (True), otherwise False. The API also returns a flag for a list of specific moderation categories that can be used to further filter and inspect the data. The categories are sexual, hate, harassment, self-harm, sexual/minors, hate/threatening, violence/graphic, self-harm/intent, self-harm/instructions, harassment/threatening and violence. Similar to the overall flag, for each category, the value is True if the model flags the corresponding category as violated, False otherwise. Finally, the API returns a dictionary of per-category scores that denote the model’s confidence that the input violates the OpenAI’s policy for the category. The value is between $0$ and $1$, where higher values denote higher confidence.

审核

为衡量内容审核，我们使用 OpenAI Moderation 接口。[^11] 该 API 以一个文本实例为输入，输出一个 json 文件，其中有一个总体布尔标记（flagged），表示输入是否可能有害（True），否则为 False。API 还会为一组具体审核类别返回标记，可用于进一步过滤和检查数据。这些类别是 sexual、hate、harassment、self-harm、sexual/minors、hate/threatening、violence/graphic、self-harm/intent、self-harm/instructions、harassment/threatening 和 violence。与总体标记类似，对每个类别，若模型将该类别标为违规则为 True，否则为 False。最后，API 返回一个按类别的分数字典，表示模型对该输入违反 OpenAI 该类别政策的置信度。取值在 $0$ 到 $1$ 之间，越高表示置信度越高。

Language Detection

To detect the language of each text instance in our dataset we used the LangID codebase.[^12] LangID is a popular python package that efficiently detects the language of an input and currently supports $97$ languages. Specifically, we use the langid.classify(text) function and store a string for the detected language.

语言检测

为检测数据集中每个文本实例的语言，我们使用了 LangID 代码库。[^12] LangID 是一个流行的 python 软件包，能高效检测输入语言，目前支持 $97$ 种语言。具体来说，我们使用 langid.classify(text) 函数，并存储检测到的语言字符串。

E.3 Manual Review

The overall proportions of texts flagged for PII, Non-English or Moderation is low (see Table 2). However, when inspecting the few positive flags, many were false positives, especially on lang-detect and PII. While a false positive on language may be relatively inconsequential, any automated flags of PII are concerning. Accordingly, we manually annotate any instances where pii_flag==True ($n=167$) for participant-written text. We find that none of them actually contain PII.[^13]

E.3 人工复核

被标记为含 PII、非英语或需审核的文本总体比例很低（见 Table 2）。不过，在检查少数阳性标记时，许多是假阳性，尤其是在语言检测和 PII 上。语言上的假阳性相对来说可能无关紧要，但任何自动标出的 PII 都令人担忧。因此，我们对 pii_flag==True 的参与者撰写文本实例做了人工标注（$n=167$）。我们发现其中没有一条真正包含 PII。[^13]

Category	Is English	Contains PII	Manually-Checked PII	Is Moderation Flagged	Total Instances
Overall	105,229 (98.8%)	1,111 (1.0%)	NA	634 (0.6%)	106,554 (100.0%)
user_prompt	26,545 (97.7%)	66 (0.2%)	0 (0.0%)	454 (1.7%)	27,172 (100.0%)
model_response	67,715 (99.0%)	944 (1.4%)	NA	162 (0.2%)	68,371 (100.0%)
self_description	1,496 (99.7%)	10 (0.7%)	0 (0.0%)	7 (0.5%)	1,500 (100.0%)
system_string	1,493 (99.5%)	16 (1.1%)	0 (0.0%)	0 (0.0%)	1,500 (100.0%)
open_feedback	7,980 (99.6%)	75 (0.9%)	0 (0.0%)	11 (0.1%)	8,011 (100.0%)

Table 2: Meta-Data Summary. For each metadata category (language, PII and moderation), we show $N(\%)$ for the dataset as a whole (Overall) and broken down by each type of text instance in Prism.

类别	是否英语	含 PII	经人工核对的 PII	是否被审核标记	实例总数
总体	105,229 (98.8%)	1,111 (1.0%)	NA	634 (0.6%)	106,554 (100.0%)
user_prompt	26,545 (97.7%)	66 (0.2%)	0 (0.0%)	454 (1.7%)	27,172 (100.0%)
model_response	67,715 (99.0%)	944 (1.4%)	NA	162 (0.2%)	68,371 (100.0%)
self_description	1,496 (99.7%)	10 (0.7%)	0 (0.0%)	7 (0.5%)	1,500 (100.0%)
system_string	1,493 (99.5%)	16 (1.1%)	0 (0.0%)	0 (0.0%)	1,500 (100.0%)
open_feedback	7,980 (99.6%)	75 (0.9%)	0 (0.0%)	11 (0.1%)	8,011 (100.0%)

表 2：元数据汇总。 对每个元数据类别（语言、PII 和审核），我们给出整个数据集（总体）以及按 Prism 中每种文本实例类型分解的 $N(\%)$。

	Human-written	Model-written
	N	(%)	N	(%)
sexual	21	0.05%	11	0.02%
hate	154	0.40%	36	0.05%
harassment	387	1.01%	127	0.19%
self-harm	24	0.06%	8	0.01%
sexual/minors	4	0.01%	4	0.01%
hate/threatening	17	0.04%	2	0.00%
self-harm/intent	26	0.07%	5	0.01%
self-harm/instructions	13	0.03%	8	0.01%
harassment/threatening	33	0.09%	7	0.01%
violence	52	0.14%	13	0.02%
Total	38,183	100.00%	68,371	100.00%

Table 3: Breakdown of flags from the OpenAI Moderation API. We show counts and percentages where the text was flagged (==True), as well as total counts. Human-written text includes user_prompt, self_description, system_string, open_feedback; Model-written text is only model_response.

	人类撰写	模型撰写
	N	(%)	N	(%)
sexual	21	0.05%	11	0.02%
hate	154	0.40%	36	0.05%
harassment	387	1.01%	127	0.19%
self-harm	24	0.06%	8	0.01%
sexual/minors	4	0.01%	4	0.01%
hate/threatening	17	0.04%	2	0.00%
self-harm/intent	26	0.07%	5	0.01%
self-harm/instructions	13	0.03%	8	0.01%
harassment/threatening	33	0.09%	7	0.01%
violence	52	0.14%	13	0.02%
总计	38,183	100.00%	68,371	100.00%

表 3：OpenAI Moderation API 标记的分解。 我们展示文本被标记（==True）的计数和百分比，以及总计数。人类撰写文本包括 user_prompt、self_description、system_string、open_feedback；模型撰写文本仅包括 model_response。

	Human-written	Model-written
	Language	N	(%)	Language	N	(%)
1	en	37,514	98.25%	en	67,715	99.04%
2	es	175	0.46%	es	236	0.35%
3	fr	71	0.19%	de	67	0.10%
4	it	70	0.18%	fr	63	0.09%
5	de	60	0.16%	la	43	0.06%
6	nl	42	0.11%	nl	37	0.05%
7	pt	41	0.11%	it	29	0.04%
8	pl	34	0.09%	sl	19	0.03%
9	da	24	0.06%	hu	15	0.02%
10	ro	13	0.03%	sv	14	0.02%
	Other	139	0.36%	Other	133	0.19%
	Total	38,183	100.00%	Total	68,371	100.00%

Table 4: Breakdown of languages detected by LangID. We show the top-10 detected languages, then other and total counts. Human-written text includes user_prompt, self_description, system_string, open_feedback; Model-written text is only model_response.

	人类撰写	模型撰写
	语言	N	(%)	语言	N	(%)
1	en	37,514	98.25%	en	67,715	99.04%
2	es	175	0.46%	es	236	0.35%
3	fr	71	0.19%	de	67	0.10%
4	it	70	0.18%	fr	63	0.09%
5	de	60	0.16%	la	43	0.06%
6	nl	42	0.11%	nl	37	0.05%
7	pt	41	0.11%	it	29	0.04%
8	pl	34	0.09%	sl	19	0.03%
9	da	24	0.06%	hu	15	0.02%
10	ro	13	0.03%	sv	14	0.02%
	其他	139	0.36%	其他	133	0.19%
	总计	38,183	100.00%	总计	68,371	100.00%

表 4：LangID 检测到的语言分解。 我们展示检测到的前 10 种语言，以及其他与总计。人类撰写文本包括 user_prompt、self_description、system_string、open_feedback；模型撰写文本仅包括 model_response。

Appendix F Annotating Ethnicity, Religion and Gender

附录 F 标注族裔、宗教与性别

We ask people to describe their ethnic and religious affiliations in their own words because for a global survey, there are no immediately obvious preset categories. In the survey data, we release this original self-description ({ethnicity, religion}_self_described). However, there are 264 unique strings for ethnicity, and 137 unique strings for religion. For some analysis, it is valuable to have aggregate groupings. To attain this grouping, we first used gpt-4-turbo to categorise the strings, but found some errors and essentialising generalisations, for example, if someone answered with a nationality not an ethnic group like american, gpt would return white.[^14]

我们请人们用自己的话描述其族裔和宗教归属，因为对一项全球问卷而言，并不存在一眼就合适的预设类别。在问卷数据中，我们发布这一原始自我描述（{ethnicity, religion}_self_described）。不过族裔有 264 个独特字符串，宗教有 137 个独特字符串。对某些分析来说，有汇总分组是有价值的。为得到这一分组，我们先用 gpt-4-turbo 给这些字符串分类，但发现一些错误和本质化的概括，例如如果有人回答的是国籍而不是族群（如 american），gpt 会返回 white。[^14]

Accordingly, we used a second round of manual human annotation to verify these automated labels. Two annotators (authors of the paper) first made independent judgements then discussed any disagreements. For ethnicity, some participants also had answered a Prolific screening question on their simplified ethnicity, though we did not have this information for all participants as it was not mandatory. We thus annotate all unique combinations of the self-described string, and the Prolific ethnicity information ($n=343$). In ambiguous cases (e.g. the aforementioned american response), we relied on this additional ethnicity information, and in its absence, defaulted to a Prefer not to say response. For religion, we do not have any additional information provided by the Prolific pre-screening questionnaire, so verification decisions were made on the basis of the self-describe string alone. The annotators agreed on 94% of ethnicity cases (discussing and resolving the remaining 20); and 96% of religion cases (discussing and resolving the remaining 5).

因此，我们用第二轮人工标注来核对这些自动标签。两名标注者（论文作者）先各自独立判断，再讨论任何分歧。对族裔，一些参与者还回答过 Prolific 关于简化族裔的筛选问题，不过并非所有参与者都有这一信息，因为它不是强制的。于是我们对自我描述字符串与 Prolific 族裔信息的所有独特组合做了标注（$n=343$）。在含糊情形下（例如上述 american 回答），我们依赖这一额外族裔信息；若缺失，则默认归为 不愿透露。对宗教，我们没有 Prolific 预筛选问卷提供的额外信息，因此核实决定只基于自我描述字符串。标注者在 94% 的族裔情形上一致（讨论并解决了其余 20 例）；在 96% 的宗教情形上一致（讨论并解决了其余 5 例）。

We highlight two general findings from our disagreements which may be of interest to people analysing or categorising our data in the future. Firstly, ethnicity and nationality are complex. Take for example the UK census, where Chinese, Banglaeshi, Indian and Pakistani are all listed as sub-categories of the Asian ethnic group.[^15] Ethnicity is a multi-faceted term which can include nationality, language group, skin colour, religion, among other characteristics [145]. Studies have shown that survey participants can interpret the term ethnic group through a variety of subjective lens [146, 147]. During annotation, we tried to gather information on whether group terms commonly refer to an ethnic group, but some subjectivity and naivety are inevitable; so, we encourage future researchers to carefully consider their own categorisations depending on the question at hand. Secondly, the belonging and believing aspects of religion intersect [148, 149], and it is not immediately clear how to categorise an individual that culturally affiliates with religion but simultaneously identities as an atheist or non-believer. Studies have revealed that the belonging and believing axis of religion are important for conditioning behaviours such as trust, pro-socality and altruism [150, 151, 152]. In general, we annotated a mention of a religion as assigned to that religion (not distinguishing between the belonging and believing channels) but it remains to be seen whether one axis is more salient for values and opinions towards AI systems.

我们强调从分歧中得到的两点一般发现，它们可能对未来分析或归类我们数据的人有用。第一，族裔与国籍是复杂的。以英国普查为例，华裔、孟加拉裔、印度裔和巴基斯坦裔都被列为亚洲族群的子类别。[^15] 族裔是一个多面向的术语，可以包括国籍、语言群体、肤色、宗教等特征 [145]。研究表明，问卷参与者可以通过多种主观透镜来理解“族群”这一术语 [146, 147]。在标注过程中，我们试图收集关于群体术语是否通常指一个族群的信息，但某些主观性和幼稚之处难以避免；因此我们鼓励未来的研究者根据手头的问题仔细考虑自己的归类。第二，宗教的归属与信仰面向相互交织 [148, 149]，如何归类一个在文化上认同宗教、同时自视为无神论者或不信者的个体，并非一目了然。研究已经表明，宗教的归属与信仰轴对于调节信任、亲社会性和利他等行为很重要 [150, 151, 152]。总体而言，我们把提及某一宗教的情形归到该宗教（不区分归属与信仰这两条通道），但哪一条轴对人们关于 AI 系统的价值和意见更为突出，仍有待观察。

Note for gender, we provided a standard multiple choice question with options: Female, Male, Non-binary / third gender, Prefer not to say and Prefer to self-describe. Only 3 individuals opted to self-describe, which we then annotated and only assimilated in very clear cut cases,[^16] else we grouped it as Prefer not to say to avoid over-riding a participant’s self-identification.

关于性别需要说明：我们提供了一个标准的多项选择题，选项为：女性、男性、非二元 / 第三性别、不愿透露和希望自行描述。只有 3 人选择自行描述，我们随后做了标注，并且只在非常明确的情形下予以归并，[^16] 否则归为 不愿透露，以免覆盖参与者的自我认同。

Appendix G Participant Demographics

附录 G 参与者人口统计

We present full demographic breakdowns in Table 5. We also compare the breakdowns in Prism to some early human feedback datasets which provide demographic information (Table 6).

我们在 Table 5 中给出完整的人口统计分解，并在 Table 6 中把 Prism 的分解与一些提供人口统计信息的早期人类反馈数据集相比较。

Total Participants	1,500	100
With conversations	1,396	93.1 %
Just survey	104	6.9 %
Age
25-34 years old	454	30.3 %
18-24 years old	297	19.8 %
35-44 years old	237	15.8 %
45-54 years old	208	13.9 %
55-64 years old	197	13.1 %
65+ years old	106	7.1 %
Prefer not to say	1	0.1 %
Gender
Male	757	50.5 %
Female	718	47.9 %
Non-binary / third gender	21	1.4 %
Prefer not to say	4	0.3 %
Self-Reported Ethnicity*
White	969	64.6 %
Black / African	122	8.1 %
Hispanic / Latino	121	8.1 %
Asian	95	6.3 %
Mixed	68	4.5 %
Middle Eastern / Arab	14	0.9 %
Indigenous / First Peoples	8	0.5 %
Other	17	1.1 %
Prefer not to say	86	5.7 %
Self-Reported Religion*
Non-religious	762	50.8 %
Christian	487	32.5 %
Agnostic	71	4.7 %
Jewish	42	2.8 %
Muslim	31	2.1 %
Spiritual	18	1.2 %
Buddhist	12	0.8 %
Folk religion	6	0.4 %
Hindu	5	0.3 %
Sikh	3	0.2 %
Other	4	0.3 %
Prefer not to say	59	3.9 %
Employment Status
Working full-time	712	47.5 %
Working part-time	265	17.7 %
Student	191	12.7 %
Unemployed, seeking work	113	7.5 %
Retired	104	6.9 %
Homemaker / Stay-at-home parent	46	3.1 %
Unemployed, not seeking work	46	3.1 %
Prefer not to say	23	1.5 %
Education
University Bachelors Degree	637	42.5 %
Graduate / Professional degree	241	16.1 %
Some University but no degree	236	15.7 %
Completed Secondary School	209	13.9 %
Vocational	125	8.3 %
Some Secondary	24	1.6 %
Completed Primary School	16	1.1 %
Some Primary	3	0.2 %
Prefer not to say	9	0.6 %
Martial Status
Never been married	870	58.0 %
Married	463	30.9 %
Divorced / Separated	123	8.2 %
Widowed	21	1.4 %
Prefer not to say	23	1.5 %
English Proficiency
Native speaker	886	59.1 %
Fluent	405	27.0 %
Advanced	160	10.7 %
Intermediate	42	2.8 %
Basic	7	0.5 %
Regions
US	338	22.5 %
Europe	313	20.9 %
UK	292	19.5 %
Latin America and the Caribbean	146	9.7 %
Australia and New Zealand	129	8.6 %
Africa	118	7.9 %
Asia	60	4.0 %
Northern America	50	3.3 %
Middle East	50	3.3 %
Oceania	1	0.1 %
Prefer not to say	3	0.2 %

Table 5: Full Demographics Breakdowns. Counts and percentages of participants by standard demographic variables. Overall, Prism utilises a large and demographically-diverse sample, especially compared to some previous human feedback datasets (see Table 6); but it still generally skews towards young, white and educated populations. *For ethnicity and religion, see details in Appendix F.

参与者总数	1,500	100
参与了对话	1,396	93.1 %
仅问卷	104	6.9 %
年龄
25-34 岁	454	30.3 %
18-24 岁	297	19.8 %
35-44 岁	237	15.8 %
45-54 岁	208	13.9 %
55-64 岁	197	13.1 %
65 岁及以上	106	7.1 %
不愿透露	1	0.1 %
性别
男性	757	50.5 %
女性	718	47.9 %
非二元 / 第三性别	21	1.4 %
不愿透露	4	0.3 %
自我报告的族裔*
白人	969	64.6 %
黑人 / 非洲裔	122	8.1 %
西班牙裔 / 拉丁裔	121	8.1 %
亚裔	95	6.3 %
混血	68	4.5 %
中东 / 阿拉伯	14	0.9 %
原住民 / 第一民族	8	0.5 %
其他	17	1.1 %
不愿透露	86	5.7 %
自我报告的宗教*
无宗教	762	50.8 %
基督教	487	32.5 %
不可知论	71	4.7 %
犹太教	42	2.8 %
伊斯兰教	31	2.1 %
灵性信仰	18	1.2 %
佛教	12	0.8 %
民间宗教	6	0.4 %
印度教	5	0.3 %
锡克教	3	0.2 %
其他	4	0.3 %
不愿透露	59	3.9 %
就业状况
全职工作	712	47.5 %
兼职工作	265	17.7 %
学生	191	12.7 %
失业，正在求职	113	7.5 %
退休	104	6.9 %
家庭主妇/夫 / 全职父母	46	3.1 %
失业，不在求职	46	3.1 %
不愿透露	23	1.5 %
教育
大学本科学位	637	42.5 %
研究生 / 专业学位	241	16.1 %
上过大学但无学位	236	15.7 %
完成中学	209	13.9 %
职业教育	125	8.3 %
部分中学	24	1.6 %
完成小学	16	1.1 %
部分小学	3	0.2 %
不愿透露	9	0.6 %
婚姻状况
从未结婚	870	58.0 %
已婚	463	30.9 %
离婚 / 分居	123	8.2 %
丧偶	21	1.4 %
不愿透露	23	1.5 %
英语水平
母语者	886	59.1 %
流利	405	27.0 %
高级	160	10.7 %
中级	42	2.8 %
初级	7	0.5 %
地区
美国	338	22.5 %
欧洲	313	20.9 %
英国	292	19.5 %
拉丁美洲和加勒比	146	9.7 %
澳大利亚和新西兰	129	8.6 %
非洲	118	7.9 %
亚洲	60	4.0 %
北美洲	50	3.3 %
中东	50	3.3 %
大洋洲	1	0.1 %
不愿透露	3	0.2 %

表 5：完整人口统计分解。 按标准人口统计变量列出的参与者计数和百分比。总体来看，Prism 使用了一个规模较大、人口统计多样的样本，尤其是与一些先前的人类反馈数据集相比（见 Table 6）；但它总体上仍偏向年轻、白人和受过教育的人群。*关于族裔和宗教，详见附录 F。

Category	Bai et al. 2022a	Ouyang et al. 2022	Glaese et al. 2022	Ganguli et al. 2022	Stiennon et al. 2020	Ours
Total Participants	28$\ddagger$	40		324		1,500
Demographic Respondents	28	19	533	115	21	1,500
Gender
Male	53.6 %	47.4 %	45.0 %	47.0 %	38.1 %	50.5 %
Female	46.4 %	42.1 %	54.0 %	52.2 %	61.9 %	47.9 %
Non-binary	0.0 %	5.3 %	1.0 %	0.9 %	0.0 %	1.4 %
Prefer not to say/Other	0.0 %	5.3 %	0.0 %	0.0 %	0.0 %	0.3 %
Sexual Orientation
Heterosexual	89.3 %	-	84.0 %	81.7 %	-	-
Lesbian or Gay	7.1 %	-	5.0 %	4.3 %	-	-
Bisexual	0.0 %	-	9.0 %	12.2 %	-	-
Uncertain	3.6 %	-	-	0.9 %	-	-
Prefer not to say/Other	0.0 %	-	2.0 %	0.9 %	-	-
Age					†	-
18-24	7.1 %	26.3 %	11.0 %	0.0 %	-	19.8 %
25-34	39.3 %	47.4 %	37.0 %	25.2 %	42.9 %	30.3 %
35-44	42.9 %	10.5 %	24.0 %	33.9 %	23.8 %	15.8 %
45-54	10.7 %	10.5 %	16.0 %	23.5 %	23.8 %	13.9 %
55-64	0.0 %	5.3 %	9.0 %	13.9 %	9.5 %	13.1 %
65+	0.0 %	0.0 %	3.0 %	1.7 %	0.0 %	7.1 %
Prefer not to say	0.0 %	-	-	1.7 %	-	0.1 %
Ethnicity
White/Caucasian	67.9 %	31.6 %	81.0 %	81.7 %	42.9 %	64.6 %
Asian	10.7 %	57.9 %	8.0 %	2.6 %	28.6 %	6.3 %
Black/African descent	3.6 %	10.5 %	4.0 %	8.7 %	-	8.1 %
Hispanic/Latino	3.6 %	15.8 %	1.0 %	0.9 %	4.8 %	8.1 %
Native American	0.0 %	0.0 %	0.0 %	2.6 %	9.6 %	0.5 %
Middle Eastern	0.0 %	0.0 %	1.0 %	0.9 %	4.8 %	0.9 %
Prefer not to say/Other	14.3 %	-	5.0 %	2.6 %	9.6 %	11.5 %
Education
No University Degree	17.9 %	10.5 %	0.0 %	34.8 %	14.3 %	40.8 %
Undergraduate Degree	57.1 %	52.6 %	66.0 %	53.9 %	57.1 %	42.5 %
Graduate Degree	14.3 %	36.8 %	34.0 %	10.4 %	28.1 %	16.1 %
Prefer not to say/Other	10.7 %	-	-	0.9 %	-	0.6 %

†Age group values for Stiennon et al. 2020 are reported for ten-year age groups starting from 20-29. We have placed the values in the row where the top end of these groups would appear to align with groups reported by the majority of studies.
$\ddagger$ Bai et al. 2022a provide two reports of demographic data. We use the one corresponding to the participants who contributed more than 80% of the total feedback.

Table 6: Demographic data compared to prior work. Comparisons of Prism to early and widely-known RLHF studies using human feedback for language models. See Section 4 for more current datasets.

类别	Bai et al. 2022a	Ouyang et al. 2022	Glaese et al. 2022	Ganguli et al. 2022	Stiennon et al. 2020	本文
参与者总数	28$\ddagger$	40		324		1,500
提供人口统计的应答者	28	19	533	115	21	1,500
性别
男性	53.6 %	47.4 %	45.0 %	47.0 %	38.1 %	50.5 %
女性	46.4 %	42.1 %	54.0 %	52.2 %	61.9 %	47.9 %
非二元	0.0 %	5.3 %	1.0 %	0.9 %	0.0 %	1.4 %
不愿透露/其他	0.0 %	5.3 %	0.0 %	0.0 %	0.0 %	0.3 %
性取向
异性恋	89.3 %	-	84.0 %	81.7 %	-	-
女同性恋或男同性恋	7.1 %	-	5.0 %	4.3 %	-	-
双性恋	0.0 %	-	9.0 %	12.2 %	-	-
不确定	3.6 %	-	-	0.9 %	-	-
不愿透露/其他	0.0 %	-	2.0 %	0.9 %	-	-
年龄					†	-
18-24	7.1 %	26.3 %	11.0 %	0.0 %	-	19.8 %
25-34	39.3 %	47.4 %	37.0 %	25.2 %	42.9 %	30.3 %
35-44	42.9 %	10.5 %	24.0 %	33.9 %	23.8 %	15.8 %
45-54	10.7 %	10.5 %	16.0 %	23.5 %	23.8 %	13.9 %
55-64	0.0 %	5.3 %	9.0 %	13.9 %	9.5 %	13.1 %
65+	0.0 %	0.0 %	3.0 %	1.7 %	0.0 %	7.1 %
不愿透露	0.0 %	-	-	1.7 %	-	0.1 %
族裔
白人/高加索裔	67.9 %	31.6 %	81.0 %	81.7 %	42.9 %	64.6 %
亚裔	10.7 %	57.9 %	8.0 %	2.6 %	28.6 %	6.3 %
黑人/非洲裔	3.6 %	10.5 %	4.0 %	8.7 %	-	8.1 %
西班牙裔/拉丁裔	3.6 %	15.8 %	1.0 %	0.9 %	4.8 %	8.1 %
美洲原住民	0.0 %	0.0 %	0.0 %	2.6 %	9.6 %	0.5 %
中东	0.0 %	0.0 %	1.0 %	0.9 %	4.8 %	0.9 %
不愿透露/其他	14.3 %	-	5.0 %	2.6 %	9.6 %	11.5 %
教育
无大学学位	17.9 %	10.5 %	0.0 %	34.8 %	14.3 %	40.8 %
本科学位	57.1 %	52.6 %	66.0 %	53.9 %	57.1 %	42.5 %
研究生学位	14.3 %	36.8 %	34.0 %	10.4 %	28.1 %	16.1 %
不愿透露/其他	10.7 %	-	-	0.9 %	-	0.6 %

†Stiennon et al. 2020 的年龄组数值按从 20-29 开始的十年年龄组报告。我们把这些数值放在各组上端看起来与多数研究报告的分组相对齐的那一行。
$\ddagger$ Bai et al. 2022a 提供了两份人口统计数据报告。我们使用的是对应贡献了全部反馈 80% 以上的那批参与者的那一份。

表 6：与先前工作比较的人口统计数据。 将 Prism 与早期且广为人知、使用人类反馈训练语言模型的 RLHF 研究相比较。更新近的数据集见第 4 节。

Category	Bai et al. 2022a	Ouyang et al. 2022	Glaese et al. 2022	Ganguli et al. 2022	Stiennon et al. 2020	Ours
United States	100 %	17 %	0 %	100 %	60 %	26 %
United Kingdom	0 %	0 %	100 %	0 %	7 %	23 %
Philippines	0 %	22 %	0 %	0 %	7 %	0 %
Bangladesh	0 %	22 %	0 %	0 %	0 %	0 %
All Others	0 %	39 %†	0 %	0 %	27 %$\ddagger$	51 %*

†One resident each from Albania, Brazil, Canada, Columbia, India, Uruguay, and Zimbabwe
$\ddagger$ One resident each from South Africa, Serbia, Turkey, India
See Table 8 for our breakdowns.

Table 7: Geographic data compared to prior work. Participant countries of residence in Prism compared to early and widely-known RLHF studies using human feedback for language models.

类别	Bai et al. 2022a	Ouyang et al. 2022	Glaese et al. 2022	Ganguli et al. 2022	Stiennon et al. 2020	本文
美国	100 %	17 %	0 %	100 %	60 %	26 %
英国	0 %	0 %	100 %	0 %	7 %	23 %
菲律宾	0 %	22 %	0 %	0 %	7 %	0 %
孟加拉国	0 %	22 %	0 %	0 %	0 %	0 %
所有其他	0 %	39 %†	0 %	0 %	27 %$\ddagger$	51 %*

†分别来自阿尔巴尼亚、巴西、加拿大、哥伦比亚、印度、乌拉圭和津巴布韦的各一名居民
$\ddagger$ 分别来自南非、塞尔维亚、土耳其、印度的各一名居民
我们的分解见 Table 8。

表 7：与先前工作比较的地理数据。 Prism 中参与者的居住国，与早期且广为人知、使用人类反馈训练语言模型的 RLHF 研究相比较。

Appendix H Participant Geographies

附录 H 参与者地理分布

We present full geographic breakdowns in Table 8. Fig. 6 is an enlarged version from Fig. 1. We compare geographic data to prior work in Table 7. For regional classifications, we use the UN definitions.[^17] We also throughout the main paper use location_special_region, which splits out the UK and the US. Regional breakdowns by birth country are shown in Fig. 7.

我们在 Table 8 中给出完整的地理分解。图 6 是图 1 的放大版本。我们在 Table 7 中把地理数据与先前工作相比较。地区分类采用联合国定义。[^17] 我们在正文中还全程使用 location_special_region，把英国和美国单独列出。按出生国划分的地区分解见图 7。

Figure 6: Geographic distribution of Prism participants by birth country. Our sampling aims were for breath (coverage across UN global regions) and depth (representative demographic coverage within UK and US samples).

图 6：按出生国看 Prism 参与者的地理分布。我们的抽样目标是广度（覆盖联合国全球地区）和深度（在英国和美国样本内获得有代表性的人口统计覆盖）。

	Country of Birth	Country of Residence
United States	338	22.5 %	386	25.7 %
United Kingdom	292	19.5 %	340	22.7 %
South Africa	91	6.1 %	86	5.7 %
Mexico	69	4.6 %	67	4.5 %
Australia	65	4.3 %	72	4.8 %
New Zealand	64	4.3 %	77	5.1 %
Chile	63	4.2 %	65	4.3 %
Canada	50	3.3 %	54	3.6 %
Israel	47	3.1 %	61	4.1 %
Nigeria	19	1.3 %	0	0.0 %
Spain	19	1.3 %	18	1.2 %
Germany	17	1.1 %	13	0.9 %
Belgium	17	1.1 %	17	1.1 %
Hungary	17	1.1 %	16	1.1 %
Poland	17	1.1 %	14	0.9 %
Ireland	17	1.1 %	15	1.0 %
Latvia	16	1.1 %	14	0.9 %
Denmark	15	1.0 %	15	1.0 %
Czechia	15	1.0 %	14	0.9 %
Norway	15	1.0 %	15	1.0 %
France	14	0.9 %	12	0.8 %
Italy	14	0.9 %	13	0.9 %
Greece	14	0.9 %	13	0.9 %
Switzerland	14	0.9 %	14	0.9 %
Finland	12	0.8 %	13	0.9 %
Estonia	11	0.7 %	10	0.7 %
Austria	11	0.7 %	10	0.7 %
Slovenia	10	0.7 %	10	0.7 %
Netherlands	9	0.6 %	8	0.5 %
India	9	0.6 %	0	0.0 %
Japan	9	0.6 %	11	0.7 %
Korea, Republic of	9	0.6 %	7	0.5 %
Portugal	8	0.5 %	7	0.5 %
Romania	7	0.5 %	0	0.0 %
Philippines	7	0.5 %	0	0.0 %
Sweden	7	0.5 %	6	0.4 %
Russian Federation	6	0.4 %	0	0.0 %
Ukraine	4	0.3 %	0	0.0 %
Bangladesh	4	0.3 %	0	0.0 %
China	4	0.3 %	0	0.0 %
Hong Kong	3	0.2 %	0	0.0 %
Pakistan	3	0.2 %	0	0.0 %
Ghana	3	0.2 %	0	0.0 %
Dominican Republic	3	0.2 %	0	0.0 %
Venezuela, Bolivarian Republic of	3	0.2 %	0	0.0 %
Indonesia	3	0.2 %	0	0.0 %
Viet Nam	2	0.1 %	0	0.0 %
Sri Lanka	2	0.1 %	0	0.0 %
Turkey	2	0.1 %	0	0.0 %
Argentina	2	0.1 %	0	0.0 %
Kazakhstan	2	0.1 %	0	0.0 %
Slovakia	1	0.1 %	0	0.0 %
Sudan	1	0.1 %	0	0.0 %
Tonga	1	0.1 %	0	0.0 %
Afghanistan	1	0.1 %	0	0.0 %
Nepal	1	0.1 %	0	0.0 %
Honduras	1	0.1 %	0	0.0 %
Belarus	1	0.1 %	0	0.0 %
Bosnia and Herzegovina	1	0.1 %	0	0.0 %
Brazil	1	0.1 %	0	0.0 %
Bulgaria	1	0.1 %	0	0.0 %
Colombia	1	0.1 %	0	0.0 %
Cuba	1	0.1 %	0	0.0 %
Côte d’Ivoire	1	0.1 %	0	0.0 %
Malaysia	1	0.1 %	0	0.0 %
Guyana	1	0.1 %	0	0.0 %
Iceland	1	0.1 %	1	0.1 %
Jamaica	1	0.1 %	0	0.0 %
Kenya	1	0.1 %	0	0.0 %
Kuwait	1	0.1 %	0	0.0 %
Lithuania	1	0.1 %	0	0.0 %
Luxembourg	1	0.1 %	2	0.1 %
Malawi	1	0.1 %	0	0.0 %
Zambia	1	0.1 %	0	0.0 %
Tanzania, United Republic of	0	0.0 %	1	0.1 %
Lesotho	0	0.0 %	1	0.1 %
Uruguay	0	0.0 %	1	0.1 %
Prefer not to say	3	0.2 %	1	0.1 %

Table 8: Full Geographic Breakdowns. We collect country of birth and current country of residence. Prism contains participants born in 75 different countries, residing in 38 different countries.

	出生国	居住国
美国	338	22.5 %	386	25.7 %
英国	292	19.5 %	340	22.7 %
南非	91	6.1 %	86	5.7 %
墨西哥	69	4.6 %	67	4.5 %
澳大利亚	65	4.3 %	72	4.8 %
新西兰	64	4.3 %	77	5.1 %
智利	63	4.2 %	65	4.3 %
加拿大	50	3.3 %	54	3.6 %
以色列	47	3.1 %	61	4.1 %
尼日利亚	19	1.3 %	0	0.0 %
西班牙	19	1.3 %	18	1.2 %
德国	17	1.1 %	13	0.9 %
比利时	17	1.1 %	17	1.1 %
匈牙利	17	1.1 %	16	1.1 %
波兰	17	1.1 %	14	0.9 %
爱尔兰	17	1.1 %	15	1.0 %
拉脱维亚	16	1.1 %	14	0.9 %
丹麦	15	1.0 %	15	1.0 %
捷克	15	1.0 %	14	0.9 %
挪威	15	1.0 %	15	1.0 %
法国	14	0.9 %	12	0.8 %
意大利	14	0.9 %	13	0.9 %
希腊	14	0.9 %	13	0.9 %
瑞士	14	0.9 %	14	0.9 %
芬兰	12	0.8 %	13	0.9 %
爱沙尼亚	11	0.7 %	10	0.7 %
奥地利	11	0.7 %	10	0.7 %
斯洛文尼亚	10	0.7 %	10	0.7 %
荷兰	9	0.6 %	8	0.5 %
印度	9	0.6 %	0	0.0 %
日本	9	0.6 %	11	0.7 %
韩国	9	0.6 %	7	0.5 %
葡萄牙	8	0.5 %	7	0.5 %
罗马尼亚	7	0.5 %	0	0.0 %
菲律宾	7	0.5 %	0	0.0 %
瑞典	7	0.5 %	6	0.4 %
俄罗斯联邦	6	0.4 %	0	0.0 %
乌克兰	4	0.3 %	0	0.0 %
孟加拉国	4	0.3 %	0	0.0 %
中国	4	0.3 %	0	0.0 %
香港	3	0.2 %	0	0.0 %
巴基斯坦	3	0.2 %	0	0.0 %
加纳	3	0.2 %	0	0.0 %
多米尼加共和国	3	0.2 %	0	0.0 %
委内瑞拉玻利瓦尔共和国	3	0.2 %	0	0.0 %
印度尼西亚	3	0.2 %	0	0.0 %
越南	2	0.1 %	0	0.0 %
斯里兰卡	2	0.1 %	0	0.0 %
土耳其	2	0.1 %	0	0.0 %
阿根廷	2	0.1 %	0	0.0 %
哈萨克斯坦	2	0.1 %	0	0.0 %
斯洛伐克	1	0.1 %	0	0.0 %
苏丹	1	0.1 %	0	0.0 %
汤加	1	0.1 %	0	0.0 %
阿富汗	1	0.1 %	0	0.0 %
尼泊尔	1	0.1 %	0	0.0 %
洪都拉斯	1	0.1 %	0	0.0 %
白俄罗斯	1	0.1 %	0	0.0 %
波斯尼亚和黑塞哥维那	1	0.1 %	0	0.0 %
巴西	1	0.1 %	0	0.0 %
保加利亚	1	0.1 %	0	0.0 %
哥伦比亚	1	0.1 %	0	0.0 %
古巴	1	0.1 %	0	0.0 %
科特迪瓦	1	0.1 %	0	0.0 %
马来西亚	1	0.1 %	0	0.0 %
圭亚那	1	0.1 %	0	0.0 %
冰岛	1	0.1 %	1	0.1 %
牙买加	1	0.1 %	0	0.0 %
肯尼亚	1	0.1 %	0	0.0 %
科威特	1	0.1 %	0	0.0 %
立陶宛	1	0.1 %	0	0.0 %
卢森堡	1	0.1 %	2	0.1 %
马拉维	1	0.1 %	0	0.0 %
赞比亚	1	0.1 %	0	0.0 %
坦桑尼亚联合共和国	0	0.0 %	1	0.1 %
莱索托	0	0.0 %	1	0.1 %
乌拉圭	0	0.0 %	1	0.1 %
不愿透露	3	0.2 %	1	0.1 %

表 8：完整地理分解。 我们收集出生国和当前居住国。Prism 包含出生于 75 个不同国家、居住于 38 个不同国家的参与者。

Figure 7: Skewed regional entropy in Prism. The hierarchical tree diagram uses participant birth location, mapping (i) special location (splitting out the US and UK), which is used in the main paper, (ii) UN-defined subregions, and (iii) ISO country codes. There is an over-representation of UK and US participants due to the census samples. In most regions besides Europe, participation is dominated by one or two birth countries. The two small vertical boxes are Prefer not to say (in red), and Oceania (in navy). Note: 88% of Prism participants are born and currently reside in the same country.

图 7：Prism 中偏斜的地区熵。 层次树图使用参与者出生地，映射 (i) 特殊地点（把美国和英国单独列出，正文中使用），(ii) 联合国定义的次区域，以及 (iii) ISO 国家代码。由于普查样本，英国和美国参与者过度代表。在欧洲以外的大多数地区，参与由一两个出生国主导。两个细小的竖直框是不愿透露（红色）和大洋洲（藏青色）。注： Prism 中 88% 的参与者出生国与当前居住国相同。

Appendix I Participant LLM Usage and Familiarity

附录 I 参与者的 LLM 使用与熟悉程度

We present breakdowns on experience with LLMs in Table 9. We did not require participants to be familiar with LLMs so provide the following primer:

我们在 Table 9 中给出关于 LLM 经验的分解。我们并不要求参与者熟悉 LLM，因此提供了如下导引：

This research is about Artificial Intelligence (AI) Language Models.

本研究关注人工智能（AI）语言模型。

These models are also sometimes referred to as Generative AI, Large Language Models (LLMs), Conversational Agents, AI Chat Bots or Virtual Assistants.

这些模型有时也被称为生成式 AI、大语言模型（LLM）、对话智能体、AI 聊天机器人或虚拟助手。

They are advanced computer programs that can understand and generate human-like text. These models learn from large amounts of text data on the internet to generate their responses.

它们是能够理解并生成类人文本的先进计算机程序。这些模型从互联网上的大量文本数据中学习，以生成回复。

One example you might have heard is ChatGPT, where people can have a conversation with an AI language model via an internet website.

你可能听说过的一个例子是 ChatGPT，人们可以通过一个互联网网站与 AI 语言模型对话。

LLM Direct Use
Yes	1,162	77.5 %
No	259	17.3 %
Unsure	79	5.3 %
LLM Indirect Use
Yes	1,104	73.6 %
No	215	14.3 %
Unsure	181	12.1 %
LLM Familiarity
Somewhat familiar	920	61.3 %
Very familiar	424	28.3 %
Not familiar at all	156	10.4 %
LLM Frequency of Use
Once per month	374	24.9 %
Every week	316	21.1 %
More than once a month	291	19.4 %
Less than one a year	162	10.8 %
Every day	110	7.3 %
Not shown question	247	16.5 %
LLM Use Cases
Research: Fact-checking or gaining overviews on specific topics.	617	49.2 %
Professional Work: Assisting in drafting, editing, or brainstorming content for work.	469	37.4 %
Creative Writing: Generating story ideas, dialogues, poems or other writing prompts.	392	31.3 %
Technical or Programming Help: Seeking programming guidance, code generation, software recommendations, or debugging assistance.	337	26.9 %
Lifestyle and Hobbies: Looking for recipes, craft ideas, home decoration tips, or hobby-related information.	310	24.7 %
Homework Assistance: Getting help with school or university assignments.	286	22.8 %
Personal Recommendations: Seeking book, music or movie recommendations.	266	21.2 %
Casual Conversation: Engaging in small talk, casual chats, or joke generation.	262	20.9 %
Language Learning: Using it as a tool for language practice or translation.	229	18.3 %
Source Suggestions: Creating or finding bibliographies, information sources or reading lists.	217	17.3 %
Daily Productivity: Setting reminders, making to-do lists, or productivity tips.	216	17.2 %
Historical or News Insight: Getting summaries or background on historical events or news and current affairs.	183	14.6 %
Well-being Guidance: Seeking general exercise routines, wellness or meditation tips.	159	12.7 %
Games: Playing text-based games, generating riddles or puzzles.	143	11.4 %
Travel Guidance: Getting destination recommendations, planning holidays, or cultural etiquette tips.	133	10.6 %
Medical Guidance: Seeking health-related advice or medical guidance.	130	10.4 %
Financial Guidance: Asking about financial concepts or general investing ideas.	107	8.5 %
Relationship Advice: Seeking general self-help or relationship advice for family, friends or partners.	98	7.8 %
Other	124	9.9 %

Table 9: Survey of Participants’ LLM Usage: The majority of participants have used LLMs directly (via a dedicated chat interface) and indirectly (embedded in products or services). Note only participants who answered Yes to LLM Direct Use or LLM Indirect Use ($n=1253$, 84%) are shown LLM Freq of Use and LLM Use Cases. For Use Cases, we show the % of these participants who selected each use case (can be multiple so $\sum\neq 1$). Exact question phrasing is reported in the survey codebook (Section V.1).

直接使用 LLM
是	1,162	77.5 %
否	259	17.3 %
不确定	79	5.3 %
间接使用 LLM
是	1,104	73.6 %
否	215	14.3 %
不确定	181	12.1 %
对 LLM 的熟悉程度
有些熟悉	920	61.3 %
非常熟悉	424	28.3 %
完全不熟悉	156	10.4 %
LLM 使用频率
每月一次	374	24.9 %
每周	316	21.1 %
每月不止一次	291	19.4 %
一年不到一次	162	10.8 %
每天	110	7.3 %
未展示该问题	247	16.5 %
LLM 使用场景
研究：核查事实或获得特定话题的概述。	617	49.2 %
专业工作：协助起草、编辑或为工作头脑风暴内容。	469	37.4 %
创意写作：生成故事想法、对白、诗歌或其他写作提示。	392	31.3 %
技术或编程帮助：寻求编程指导、代码生成、软件推荐或调试协助。	337	26.9 %
生活方式与爱好：寻找食谱、手工想法、家居装饰建议或爱好相关信息。	310	24.7 %
作业协助：为学校或大学作业寻求帮助。	286	22.8 %
个人推荐：寻求书籍、音乐或电影推荐。	266	21.2 %
闲聊：进行小对话、随意聊天或生成笑话。	262	20.9 %
语言学习：把它当作语言练习或翻译工具。	229	18.3 %
来源建议：创建或查找参考文献、信息来源或阅读清单。	217	17.3 %
日常效率：设置提醒、列待办清单或效率建议。	216	17.2 %
历史或新闻洞察：获得历史事件或新闻时事的摘要或背景。	183	14.6 %
福祉指导：寻求一般锻炼计划、健康或冥想建议。	159	12.7 %
游戏：玩文本游戏、生成谜语或益智题。	143	11.4 %
旅行指导：获得目的地推荐、规划假期或文化礼仪建议。	133	10.6 %
医疗指导：寻求健康相关建议或医学指导。	130	10.4 %
财务指导：询问财务概念或一般投资想法。	107	8.5 %
关系建议：为家人、朋友或伴侣寻求一般自助或关系建议。	98	7.8 %
其他	124	9.9 %

表 9：参与者 LLM 使用情况问卷：大多数参与者直接使用过 LLM（通过专门的聊天界面）也间接使用过（嵌入在产品或服务中）。注意只有对直接使用 LLM或间接使用 LLM回答是的参与者（$n=1253$，84%）才会被展示LLM 使用频率和LLM 使用场景。对使用场景，我们展示这些参与者中选择每一项的百分比（可多选，因此 $\sum\neq 1$）。确切的问题表述见问卷编码手册（第 V.1 节）。

I.1 Other Identified Usecases

In addition to the usecases in Table 9, 122 participants used the “Other” option to add a usecase in their own words. Many of these just add more specific details to the pre-provided categories. In addition, there were a few interesting themes:

I.1 其他被识别的使用场景

除了 Table 9 中的使用场景外，122 名参与者用“其他”选项用自己的话补充了一种使用场景。其中许多只是给预先提供的类别加上更具体的细节。此外，还有几个有意思的主题：

• • Customer Service: Many of the participants noted having interacted with LLMs in customer support chats, often with negative sentiment (“Usually forced to interact with chatbots to get something done”, “Customer service bots I cannot avoid”, “Insurance companies direct you to chatbots, usually useless”).

• • 客户服务： 许多参与者提到曾在客户支持聊天中与 LLM 交互，往往带有负面情绪（“Usually forced to interact with chatbots to get something done”，“Customer service bots I cannot avoid”，“Insurance companies direct you to chatbots, usually useless”）。

• • Prolific and Other Online Surveys: One of the more common (and potentially concerning) answers mentioned research participation e.g. “Studies like this one”, “Doing Prolific tests”, but it may be that they mean AI is the subject of the study: “AI research subject on research platforms Prolific, others.” or “It’s sometimes required as part of a survey on Prolific.” We encourage future work on whether there is noticeable difference in these participants’ answers elsewhere in our task.

• • Prolific 和其他在线问卷： 较常见（也潜在令人担忧）的一种回答提到研究参与，例如“Studies like this one”，“Doing Prolific tests”，不过他们的意思也可能是 AI 是研究对象：“AI research subject on research platforms Prolific, others.” 或 “It’s sometimes required as part of a survey on Prolific.” 我们鼓励未来工作考察这些参与者在本任务其他部分的回答是否有明显不同。

• • AI Understanding or Testing: A few participants mentioned “Trying to gain an understanding into AI and its capabilities” or “Gauging progress/viability of AI models”. Many others indicated curiosity or exploratory use e.g. “Just to test it out and see what it’s all about” or “Casual interest in the new technology”.

• • 理解或测试 AI： 一些参与者提到“Trying to gain an understanding into AI and its capabilities”或“Gauging progress/viability of AI models”。还有许多人表示出于好奇或探索性使用，例如“Just to test it out and see what it’s all about”或“Casual interest in the new technology”。

• • Professional or Job Tasks: Participants added details on professional usecases like resume help, interview prep, CV writing, HR-tasks, Excel help, or emails.

• • 专业或工作任务： 参与者补充了专业使用场景的细节，例如简历帮助、面试准备、CV 撰写、人力资源任务、Excel 帮助或邮件。

• • Creative (Multimodal) Use-cases: Participants gave additional detail like writing YouTube scripts, generating gift card text or designing characters for games as well as multimodal creative outputs like generating drawings or images.

• • 创意（多模态）使用场景： 参与者给出了额外细节，例如撰写 YouTube 脚本、生成礼品卡文案或为游戏设计角色，以及生成绘画或图像等多模态创意输出。

• • Domain-Specific Usecases: Medical, Financial and Educational usecases are all mentioned.

• • 领域特定的使用场景： 医疗、财务和教育类使用场景都被提到。

Appendix J Screening and Recruitment Process

附录 J 筛选与招募流程

We recruit workers via Prolific (https://www.prolific.com/). We apply two initial screening criteria: (i) participants must be fluent in English because Prism targets monolingual models and language data, and (ii) participants must have been born and reside in the same country to avoid biasing our sample towards expats living abroad. There is a skewed country-wise distribution of active workers who meet this criteria (see Table 10). For example, of the 21,084 workers in Europe (passing screening), 17% are Portuguese, 15% German and 14% Polish; and all 6,584 workers in Africa are located in South Africa. To account for this, we set up country-specific studies in each country with at least one eligible worker, balance study spots across regions, and ensure no single country has more than 100 open spots (apart from the Rep Samples in the UK and US). We collected information on country of birth and country of current residence during our survey (separate to workers’ stored Prolific details), and find that 179 participant (12%) have different birth and reside countries. We do not exclude these individuals from our sample.

我们通过 Prolific（https://www.prolific.com/）招募工作者。我们施加两项初始筛选标准：(i) 参与者必须英语流利，因为 Prism 针对单语模型和语言数据；(ii) 参与者必须出生并居住在同一国家，以免样本偏向旅居国外的侨民。满足这一标准的活跃工作者在国家层面的分布是偏斜的（见 Table 10）。例如，在欧洲的 21,084 名通过筛选的工作者中，17% 是葡萄牙人，15% 是德国人，14% 是波兰人；而非洲的全部 6,584 名工作者都位于南非。为应对这一点，我们在每个至少有一名合格工作者的国家设立国别研究，在地区之间平衡研究名额，并确保没有任何一个国家有超过 100 个开放名额（英国和美国的代表性样本除外）。我们在问卷中收集了出生国和当前居住国信息（独立于工作者存储在 Prolific 上的资料），并发现 179 名参与者（12%）的出生国与居住国不同。我们并不把这些个体从样本中排除。

	Rep	Gender	Launched	Approved	Prolific Fluent English Speakers	Cost
Study	Sample	Bal	(2023)	Submissions	(All)	(Born=Reside)	(£)
Total	2	25	-	1,500		111,572		100,585		14,850.00
US	✓	✗	27-11	386	25.7 %	38,114	34.2 %	36,205	36.0 %	3,821.40
UK	✓	✗	27-11	341	22.7 %	37,408	33.5 %	33,678	33.5 %	3,375.90
South Africa	✗	✓	22-11	88	5.9 %	7,061	6.3 %	6,584	6.5 %	871.20
New Zealand	✗	✓	24-11	77	5.1 %	511	0.5 %	389	0.4 %	762.30
Australia	✗	✓	24-11	71	4.7 %	1,968	1.8 %	1,550	1.5 %	702.90
Mexico	✗	✓	24-11	69	4.6 %	2,021	1.8 %	1,943	1.9 %	683.10
Chile	✗	✓	23-11	65	4.3 %	455	0.4 %	416	0.4 %	643.50
Israel	✗	✗	25-11	61	4.1 %	310	0.3 %	272	0.3 %	603.90
Canada	✗	✓	22-11	54	3.6 %	3,687	3.3 %	3,031	3.0 %	534.60
Asia	✗	✗	24-11	18	1.2 %	196	0.2 %	32	0.0 %	178.20
Spain	✗	✓	23-11	18	1.2 %	1,252	1.1 %	942	0.9 %	178.20
Belgium	✗	✓	23-11	17	1.1 %	376	0.3 %	281	0.3 %	168.30
Hungary	✗	✓	24-11	16	1.1 %	537	0.5 %	456	0.5 %	158.40
Ireland	✗	✓	23-11	15	1.0 %	640	0.6 %	502	0.5 %	148.50
Denmark	✗	✗	23-11	15	1.0 %	119	0.1 %	65	0.1 %	148.50
Norway	✗	✗	23-11	15	1.0 %	91	0.1 %	59	0.1 %	148.50
Switzerland	✗	✓	23-11	14	0.9 %	205	0.2 %	104	0.1 %	138.60
Poland	✗	✓	23-11	14	0.9 %	2,975	2.7 %	2,850	2.8 %	138.60
Czech Republic	✗	✓	24-11	14	0.9 %	238	0.2 %	229	0.2 %	138.60
Latvia	✗	✓	23-11	14	0.9 %	173	0.2 %	162	0.2 %	138.60
Greece	✗	✓	24-11	14	0.9 %	809	0.7 %	747	0.7 %	138.60
Finland	✗	✓	23-11	13	0.9 %	152	0.1 %	117	0.1 %	128.70
Germany	✗	✓	24-11	13	0.9 %	3,152	2.8 %	2,295	2.3 %	128.70
Italy	✗	✓	24-11	12	0.8 %	2,037	1.8 %	1,857	1.8 %	118.80
France	✗	✓	24-11	12	0.8 %	957	0.9 %	681	0.7 %	118.80
Slovenia	✗	✓	24-11	10	0.7 %	232	0.2 %	220	0.2 %	99.00
Austria	✗	✓	24-11	10	0.7 %	231	0.2 %	156	0.2 %	99.00
Estonia	✗	✓	24-11	10	0.7 %	251	0.2 %	237	0.2 %	99.00
Netherlands	✗	✓	24-11	8	0.5 %	1,460	1.3 %	1,028	1.0 %	79.20
Portugal	✗	✓	24-11	7	0.5 %	3,649	3.3 %	3,284	3.3 %	69.30
Sweden	✗	✓	24-11	6	0.4 %	274	0.2 %	196	0.2 %	59.40
Luxembourg	✗	✗	23-11	2	0.1 %	15	0.0 %	6	0.0 %	19.80
Iceland	✗	✗	23-11	1	0.1 %	16	0.0 %	11	0.0 %	9.90

Table 10: Summary of Recruitment Studies We present study-wise breakdowns ($n=33$). Each study was created based on the constraints of Prolific’s pool of workers. We show here all the countries with at least 1 fluent English speaker, and the counts for fluent English Speakers who were born and currently reside in that same country. We show the whether each study was screened for a special representative sample (Rep Sample) or if it was balanced on participant gender (Gender Bal). In some cases, there were too few active participants per country to balance by gender without comprising participant privacy. We also show when the first batch was launched (all dates are in 2023) and approximate cost (at £9 per hour per participant).

	代表性	性别	启动	已通过	Prolific 英语流利使用者	费用
研究	样本	平衡	(2023)	提交数	(全部)	(出生=居住)	(£)
合计	2	25	-	1,500		111,572		100,585		14,850.00
US	✓	✗	27-11	386	25.7 %	38,114	34.2 %	36,205	36.0 %	3,821.40
UK	✓	✗	27-11	341	22.7 %	37,408	33.5 %	33,678	33.5 %	3,375.90
南非	✗	✓	22-11	88	5.9 %	7,061	6.3 %	6,584	6.5 %	871.20
新西兰	✗	✓	24-11	77	5.1 %	511	0.5 %	389	0.4 %	762.30
澳大利亚	✗	✓	24-11	71	4.7 %	1,968	1.8 %	1,550	1.5 %	702.90
墨西哥	✗	✓	24-11	69	4.6 %	2,021	1.8 %	1,943	1.9 %	683.10
智利	✗	✓	23-11	65	4.3 %	455	0.4 %	416	0.4 %	643.50
以色列	✗	✗	25-11	61	4.1 %	310	0.3 %	272	0.3 %	603.90
加拿大	✗	✓	22-11	54	3.6 %	3,687	3.3 %	3,031	3.0 %	534.60
亚洲	✗	✗	24-11	18	1.2 %	196	0.2 %	32	0.0 %	178.20
西班牙	✗	✓	23-11	18	1.2 %	1,252	1.1 %	942	0.9 %	178.20
比利时	✗	✓	23-11	17	1.1 %	376	0.3 %	281	0.3 %	168.30
匈牙利	✗	✓	24-11	16	1.1 %	537	0.5 %	456	0.5 %	158.40
爱尔兰	✗	✓	23-11	15	1.0 %	640	0.6 %	502	0.5 %	148.50
丹麦	✗	✗	23-11	15	1.0 %	119	0.1 %	65	0.1 %	148.50
挪威	✗	✗	23-11	15	1.0 %	91	0.1 %	59	0.1 %	148.50
瑞士	✗	✓	23-11	14	0.9 %	205	0.2 %	104	0.1 %	138.60
波兰	✗	✓	23-11	14	0.9 %	2,975	2.7 %	2,850	2.8 %	138.60
捷克共和国	✗	✓	24-11	14	0.9 %	238	0.2 %	229	0.2 %	138.60
拉脱维亚	✗	✓	23-11	14	0.9 %	173	0.2 %	162	0.2 %	138.60
希腊	✗	✓	24-11	14	0.9 %	809	0.7 %	747	0.7 %	138.60
芬兰	✗	✓	23-11	13	0.9 %	152	0.1 %	117	0.1 %	128.70
德国	✗	✓	24-11	13	0.9 %	3,152	2.8 %	2,295	2.3 %	128.70
意大利	✗	✓	24-11	12	0.8 %	2,037	1.8 %	1,857	1.8 %	118.80
法国	✗	✓	24-11	12	0.8 %	957	0.9 %	681	0.7 %	118.80
斯洛文尼亚	✗	✓	24-11	10	0.7 %	232	0.2 %	220	0.2 %	99.00
奥地利	✗	✓	24-11	10	0.7 %	231	0.2 %	156	0.2 %	99.00
爱沙尼亚	✗	✓	24-11	10	0.7 %	251	0.2 %	237	0.2 %	99.00
荷兰	✗	✓	24-11	8	0.5 %	1,460	1.3 %	1,028	1.0 %	79.20
葡萄牙	✗	✓	24-11	7	0.5 %	3,649	3.3 %	3,284	3.3 %	69.30
瑞典	✗	✓	24-11	6	0.4 %	274	0.2 %	196	0.2 %	59.40
卢森堡	✗	✗	23-11	2	0.1 %	15	0.0 %	6	0.0 %	19.80
冰岛	✗	✗	23-11	1	0.1 %	16	0.0 %	11	0.0 %	9.90

表 10：招募研究汇总 我们给出按研究分解的结果（$n=33$）。每项研究都是基于 Prolific 工作者池的约束创建的。这里展示所有至少有 1 名英语流利使用者的国家，以及出生并当前居住在同一国家的英语流利使用者计数。我们标明每项研究是否按特殊代表性样本筛选（代表性样本），或是否按参与者性别平衡（性别平衡）。在有些情形下，每个国家的活跃参与者太少，无法在不损害参与者隐私的前提下按性别平衡。我们也给出第一批启动的时间（所有日期都在 2023 年）以及大致费用（按每位参与者每小时 9 英镑）。