Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)

人工蜂群心智：语言模型（及之外）的开放式同质化（Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)）

Liwei Jiang    Yuanjun Chai    Margaret Li    Mickel Liu    Raymond Fok    Nouha Dziri    Yulia Tsvetkov    Maarten Sap Affiliation:  University of Washington   Carnegie Mellon University    Alon Albalak    Yejin Choi Affiliation:  Allen Institute for Artificial Intelligence  Lila Sciences  Stanford University lwjiang@cs.washington.edu Code : https://github.com/liweijiang/artificial-hivemind Infinity-Chat Collection :  liweijiang/artificial-hivemind

Liwei Jiang、Yuanjun Chai、Margaret Li、Mickel Liu、Raymond Fok、Nouha Dziri、Yulia Tsvetkov、Maarten Sap（华盛顿大学 / 卡内基梅隆大学）；Alon Albalak、Yejin Choi（Allen Institute for AI / Lila Sciences / 斯坦福大学）
联系：lwjiang@cs.washington.edu
代码：https://github.com/liweijiang/artificial-hivemind
Infinity-Chat 合集：liweijiang/artificial-hivemind

Abstract

摘要

Large language models (LMs) often struggle to generate diverse, human-like creative content, raising concerns about the long-term homogenization of human thought through repeated exposure to similar outputs. Yet scalable methods for evaluating LM output diversity remain limited, especially beyond narrow tasks such as random number or name generation, or beyond repeated sampling from a single model. To address this gap, we introduce Infinity-Chat, a large-scale dataset of 26K diverse, real-world, open-ended user queries that admit a wide range of plausible answers with no single ground truth. We introduce the first comprehensive taxonomy for characterizing the full spectrum of open-ended prompts posed to LMs, comprising 6 top-level categories (e.g., creative content generation, brainstorm & ideation) that further breaks down to 17 subcategories. Using Infinity-Chat, we present a large-scale study of mode collapse in LMs, revealing a pronounced Artificial Hivemind effect in open-ended generation of LMs, characterized by (1) intra-model repetition, where a single model consistently generates similar responses, and more so (2) inter-model homogeneity, where different models produce strikingly similar outputs. Infinity-Chat also includes 31,250 human annotations, across absolute ratings and pairwise preferences, with 25 independent human annotations per example. This enables studying collective and individual-specific human preferences in response to open-ended queries. Our findings show that state-of-the-art LMs, reward models, and LM judges are less well calibrated to human ratings on model generations that elicit differing idiosyncratic annotator preferences, despite maintaining comparable overall quality. Overall, Infinity-Chat presents the first large-scale resource for systematically studying real-world open-ended queries to LMs, revealing critical insights to guide future research for mitigating long-term AI safety risks posed by the Artificial Hivemind.

大语言模型（LM）常难以生成多样、类人的创造性内容，引发人们担心：反复暴露于相似输出会导致人类思维长期同质化。然而，可扩展地评测 LM 输出多样性的方法仍然有限，尤其是在随机数/姓名生成等狭窄任务之外，或超出对单一模型反复采样的设定。为填补缺口，我们提出 Infinity-Chat：含 26K 条多样、真实世界、允许宽范围合理解且无单一标准答案的开放式用户查询的大规模数据集。我们给出首个刻画面向 LM 的开放式提示全谱系的综合分类体系，含 6 个顶层类别（如创意内容生成、头脑风暴与构思）并细分为 17 个子类。基于 Infinity-Chat，我们对 LM 的模式崩塌做大规模研究，揭示开放式生成中显著的*人工蜂群心智（Artificial Hivemind）*效应，特征为：(1) 模型内重复——单个模型持续生成相似回答；更甚者 (2) 模型间同质化——不同模型产出惊人相似的输出。Infinity-Chat 还含 31,250 条人类标注（绝对评分与成对偏好），每例 25 名独立标注者，从而可研究开放式查询下的集体与个体特异人类偏好。我们发现，当前最强 LM、奖励模型与 LM 裁判 在会引发标注者特异偏好分歧的模型生成上，与人类评分校准更差，尽管总体质量相当。总体而言，Infinity-Chat 是系统研究面向 LM 的真实世界开放式查询*的首个大规模资源，揭示缓解人工蜂群心智*带来的长期 AI 安全风险的关键洞见。

1 Introduction

1 引言

Figure 1: Responses to the query “Write a metaphor about time” clustered by applying PCA to reduce sentence embeddings to two dimensions. Each of the 25 models generates 50 responses using top-p sampling ($p=0.9$) and temperature $=1.0$. Despite the diversity of model families and sizes, the responses form just two visually prominent regions in the 2D projection: a dominant cluster on the left centered on the metaphor “time is a river,” and a smaller cluster on the right revolving around variations of “time is a weaver.”

图 1：对查询 “写一个关于时间的隐喻” 的回答，经 PCA 将句嵌入降到二维后聚类。25 个模型各用 top-p 采样（$p=0.9$）与温度 $=1.0$ 生成 50 条回答。尽管模型族与规模多样，二维投影上回答只形成 两个视觉上突出的区域：左侧主导簇以隐喻“时间是河流”为中心，右侧较小簇围绕“时间是织工”的变体。

Large language models (LMs) are the core backbone of modern AI systems but often fail to produce the diverse, human-like creativity expected in open-ended tasks that do not have a ground truth answer West and Potts (2025); Zhang et al. (2024); Zhang et al. (2025b); Li et al. (2024b). This shortfall has sparked growing concerns about the long-term homogenization of human thought, as users are repeatedly exposed to similar outputs Sourati et al. (2025); Bommasani et al. (2022). While ensemble methods or model “swarms” have been proposed to enhance diversity Feng et al. (2025b); Feng et al. (2025a), their scalable evaluations of diversity in real-world settings are still lacking Gerstgrasser et al. (2024). Existing benchmarks often target stylized tasks such as persona generation (Ge et al., 2024), keyword-driven storytelling (Chakrabarty et al., 2024), or random number generation (Zhang et al., 2024; West and Potts, 2025), and often rely on narrowly defined tests centered on poetry or figurative language (Padmakumar et al., 2025; Zhang et al., 2025b). Yet, these settings fail to capture the open-endedness and pluralism of real-world user interactions.

大语言模型是现代 AI 系统的核心骨干，但在没有标准答案的开放式任务中，常无法产出预期的多样、类人创造力（West and Potts, 2025；Zhang et al., 2024；Zhang et al., 2025b；Li et al., 2024b）。这一短板引发对人类思维长期同质化的担忧——用户反复暴露于相似输出（Sourati et al., 2025；Bommasani et al., 2022）。虽有集成或模型“蜂群”等方法被提出以增强多样性（Feng et al., 2025b；Feng et al., 2025a），真实场景下可扩展的多样性评测仍不足（Gerstgrasser et al., 2024）。既有基准常瞄准人格生成、关键词驱动故事或随机数生成等风格化任务，或集中于诗歌与比喻语言的狭义测试，难以刻画真实用户交互的开放度与多元性。

We introduce Infinity-Chat, a large-scale dataset of 26K real-world open-ended queries spanning diverse, naturally occurring prompts mined from WildChat Zhao et al. (2024). These queries admit a wide range of plausible answers with no single correct response. We further develop the first comprehensive taxonomy of open-ended LM queries, encompassing 6 top-level categories (e.g., Brainstorm & Ideation, and less explored types such as Speculative & Hypothetical Scenarios, and Skill Development) and 17 subcategories grounded in natural chatbot-user interactions.

我们提出 Infinity-Chat：从 WildChat（Zhao et al., 2024）挖掘的 26K 条真实世界开放式查询，覆盖多样、自然出现的提示，允许宽范围合理解且无单一正确答案。我们进一步给出首个开放式 LM 查询综合分类体系，含 6 个顶层类别（如头脑风暴与构思，以及较少被探索的思辨/假想场景、技能发展等）与 17 个扎根于自然聊天交互的子类。

Using Infinity-Chat, we systematically study intra- and inter-model mode collapse across 70+ open and closed source LMs (25 detailed in the main paper). We uncover a pronounced Artificial Hivemind effect: (1) intra-model repetition, where a single model repeatedly generates similar outputs, and, more critically, (2) inter-model homogeneity, where different models independently converge on similar ideas with minor variations in phrasing. The latter warns that model ensembles may not yield true diversity when their constituents share overlapping alignment and training priors.

基于 Infinity-Chat，我们系统研究跨 70+ 开源与闭源 LM（正文详述 25 个）的模型内与模型间模式崩塌。我们揭示显著的人工蜂群心智效应：(1) 模型内重复——单个模型反复生成相似输出；更关键的是 (2) 模型间同质化——不同模型独立收敛到高度相似的回答。

Beyond generative behaviors, we also examine whether LMs are calibrated to assess alternative responses of comparable quality to open-ended queries. To enable this study, we collect 31,250 human annotations on distinct model responses in Infinity-Chat, encompassing both absolute quality ratings and pairwise preferences, with dense annotations from 25 independent annotators per query–response pair. Our results show that LMs, reward models, and LM-based judges are often miscalibrated with respect to human ratings on responses that elicit divergent, idiosyncratic preferences among annotators despite comparable overall quality. This exposes key limitations in current modeling pipelines, which tend to assume a single, consensus notion of quality and thus overlook or fail to reward the diverse, pluralistic preferences that arise in open-ended responses.

除了基因行为之外,我们还审查 * 是否对LMs进行校准,以评估对开放式询问的替代性质量答复*。 我可不想再有其他的答案。 为了能够进行这项研究,我们收集了31,250个人类对Infinity-Chat中不同模型反应的说明,包括绝对质量评分和双向偏好,每个查询25个独立注释者* 答复对等* 。 我们的结果显示,LM、奖励模型和基于LM的评委往往在“引起注释者之间不同和特殊偏好的答复”上对人类的评分有误,尽管总体质量相当。 这暴露了目前建模管道的主要局限性,它倾向于采用单一的、协商一致的质量概念,从而忽视或未能奖励在无限制反应中产生的多样化、多元的偏好。

Altogether, our work introduces a comprehensive framework for evaluating realistic open-endedness, diversity, and pluralistic alignment in LMs, both within and across LMs. By integrating real-world queries, a taxonomy of query types, and dense human annotations, Infinity-Chat provides a useful resource for diagnosing the Artificial Hivemind effect and for guiding the development of safer, more expressive, and more resourceful LMs that better empower human creativity.

总体而言,我们的工作引入了一个全面的框架,用以评估在多边安排内部和相互之间在多边安排中现实的开放性、多样性和多元一致性。 通过整合真实世界的查询,查询类型的分类,以及密集的人类注释,Infinity-Chat为诊断人工蜂窝效应提供了有用的资源,并指导了更安全,更表现力更强,更有才智的LMs的发展,更好地增强人类的创造力.

2 Infinity-Chat: Real-World Open-Ended Queries with Diverse Responses

2 Infinity-Chat：带多样回答的真实世界开放式查询

Figure 2: A taxonomy of real-world open-ended queries that invite diverse model responses that are mined from in-the-wild user-chatbot interactions, categorized into 6 top-level and 17 fine-grained subcategories, along with their occurrence percentages.

图二:关于真实世界开放式查询的分类,这些查询邀请了各种模型的答复,这些答复来自用户与机器人的互动,分为6个顶级和17个细纹亚类,及其发生百分比。

Most existing LM alignment datasets prioritize response correctness over diversity, and rarely include multiple distinctive responses to the same prompt. This overlooks the inherent variability of open-ended queries, which often admit several equally valid answers. This gap motivates our first central research question: What types of open-ended queries do users actually pose to language models?

大多数现有的LM校正数据集将响应正确性置于多样性之上,很少包含对同一即时的多个不同响应. 这忽略了不限名额的查询的内在差异,这些查询往往接受若干同样有效的答复。 这一差距激发了我们的第一个中心研究问题:* 用户对语言模型的实际贡献是何种类型的开放式查询?*

Mining in-the-wild open-ended user queries. We construct Infinity-Chat, a dataset of real-world open-ended queries to language models, by filtering and refining user inputs from WildChat (Zhao et al., 2024). From 37,426 high-quality, single-turn GPT-4 queries (English, non-toxic, 15–200 characters), GPT-4o classifies each by whether it seeks meaningful information, is a greeting or model inquiry, and allows single or multiple valid responses. Ambiguous queries are revised for clarity. The result is an extensive collection of 26,070 open-ended and 8,817 closed-ended queries, which elicit diverse, high-quality LM responses. Full details of the query mining process are provided in §Appendix B.1.

** 处理无限制用户的无限制查询。 ** 我们通过过滤和精炼来自WildChat(Zhao等人,2024年)的用户输入,构建了Infinity-Chat(Infinity-Chat),这是现实世界对语言模型的开放式查询数据集. 从37,426个高质量,单转的GPT-4查询(英语,无毒,15–200个字符)中,GPT-4o按其是否寻求有意义的信息对每个查询进行分类,是一种问候或模式查询,允许单次或多次有效回复. 为澄清起见,修改了含糊的询问。 其结果是广泛收集了26,070个不限名额的询问和8,817个不限名额的询问,从而获得了多种高质量的LM回复. 查询挖掘过程的全部细节见§Appendix B.1.

Figure 3: A word cloud visualizing new open-ended categories mined from in-the-wild queries.

图3:云可视化** 新开放类别**

Categorizing the diverse landscape of open-ended queries. To understand the types of open-ended queries users pose to LMs, we develop a taxonomy of fine-grained categories. We adopt a semi-automatic process to construct the taxonomy. Starting with $\sim$100 mined queries, we manually assign tentative labels, then iteratively refine and group them into a hierarchical structure. This results in 6 high-level categories and 17 fine-grained sub-categories, as shown in Figure 2. Next, we scale the annotation process to the full set of open-ended queries using GPT-4o. We instruct GPT-4o to label each user query with one or more of the existing open-ended categories, and to detect novel types beyond the seed categories. Full details of the taxonomy construction process are provided in §Appendix B.2.

** 分类各种不限名额的查询。 ** 为了了解用户对LMS的开放式查询类型,我们开发了精细分类法. 我们采用半自动过程来构建分类学. 从$\sim$ 100 地雷查询开始,我们手动指定了暂定标签,然后反复细化,将它们分成一个等级结构。 如图2所示,这导致6个高级类别和17个精细分类。 接下来,我们用GPT-4o将注释过程放大为完整的一组开放式查询. 我们指示GPT-4o将每个用户查询贴上一个或多个现有开放类别标签,并检测种子类别以外的新类型。 分类构建过程的全部细节见§Appendix B.2.

As shown in Figure 2, while Creative Content Generation dominates (58.0%), we identify several underexplored yet popular types, such as Alternative Writing Genres (38.5%), Concept Explanation (23.6%), Skill Development (23.5%), Analytical & Interpretive Questions (22.6%), and Hypothetical Scenarios (22.2%). Notably, 15.2% of queries involve Brainstorming & Ideation, underscoring users’ reliance on LMs for direct ideas and inspirations, and raising concerns about the long-term risk of homogenized thinking driven by overly uniform AI outputs.

如图2所示,虽然创意内容生成占多数(58.0%),但我们发现了几种未得到充分探讨和流行的类型,如替代写作流派(38.5%)、概念解释(23.6%)、技能发展(23.5%)、分析与解释问题(22.6%)和假想(22.2%)。 值得注意的是,15.2%的查询涉及集思广益和思想,强调用户依赖低度元数据来直接提出想法和启发,并引起人们对AI产出过于一致驱动的同质化思维的长期风险的关切。

In addition to our pre-defined categories, we identify 314 novel ones. Figure 3 visualizes a word cloud of the most prominent keywords, such as “Cultural,” “Analysis,” “Ethical,” “Historical,” “Media,” and “Humor,” highlighting previously underexplored dimensions of open-ended query categories.

除了我们预先确定的类别外,我们还确定了314个新的类别。 图3将“文化”、“分析”、“道德”、“历史”、“媒体”和“Humor”等最突出关键词的词云直观化,突出了以前未充分探讨的开放式查询类别的维度。

With Infinity-Chat, we introduce the first comprehensive taxonomy of real-world open-ended queries that invite diverse responses. This dataset serves as a rich resource for studying LMs’ capacity to generate varied appropriate outputs, and for advancing pluralistic alignment of LMs.

我们用Infinity-Chat,介绍第一种关于现实世界开放性问询的全面分类,邀请了不同的回答。 该数据集是研究LMs产生各种适当产出的能力和推动LMs多元化调整的丰富资源。

Figure 4: The heatmap shows degree of repetition in responses to open-ended queries generated by the same LMs. For each model, we generate 50 responses per query across 100 open-ended queries from Infinity-Chat100. We then compute the average pairwise sentence embedding similarities for each query’s response pool and measure the percentage of queries falling into the similarity ranges indicated on the y-axis. Under the sampling parameters (top-p $=0.9$, temperature $=1.0$), the average pairwise similarity among responses to the same prompt typically exceeds 0.8. As a reference, randomly paired responses from the global pool 100% fall within the 0.1–0.2 range. This is only a reference for the similarity scale, not a matched baseline for same-prompt convergence.

图4:热图显示对同一LMs产生的无限制询问的答复重复度。 对于每个模式,我们从Infinity-Chat100的100个不限名额的查询中,每个查询得到50个答复。 然后,我们计算每个查询响应集合的平均对等句子嵌入相似性,并测量属于Y轴上所示相似范围的查询的百分比。 在采样参数(top-*p * $=0.9$,温度 $=1.0$)下,对同一时速的反应之间平均对等性通常超过0.8. 作为参考,来自全球集合的随机配对反应100%在0.1-0.2范围内。 这仅仅是相似度表的参考,而不是同一瞬间趋同的匹配基准.

3 Artificial Hivemind: Intra- and Inter-Model Homogeneity in LMs

3 人工蜂群心智：语言模型的模型内与模型间同质化

Using a subset of 100 representative open-ended queries from Infinity-Chat (denoted Infinity-Chat100, human verified to be open-ended as detailed in §Appendix B.3), we systematically examine the “Artificial Hivemind” of LMs. We focus on two aspects: (1) intra-model repetition, where the same LM fails to generate diverse outputs, and (2) inter-model homogeneity, where different models produce similar outputs. Prior studies have explored intra-model repetition at small scales or with synthetic tasks (e.g., random number/name generation) West and Potts (2025); Zhang et al. (2024). In contrast, we conduct a large-scale study on real-world open-ended questions, spanning 70+ LMs (25 detailed in the main paper, representing the strongest or largest models from major model families), providing the first systematic analysis of cross-model output convergence. Full experimental setup, complete model results, and examples are provided in §Appendix C.

使用 Infinity-Chat 中 100 条有代表性的开放式查询子集（记为 Infinity-Chat100，经人类验证确为开放式，见附录 B.3），我们系统考察 LM 的“人工蜂群心智”。聚焦两方面：(1) 模型内重复——同一 LM 无法生成多样输出；(2) 模型间同质化——不同模型产出相似输出。先前工作在小规模或合成任务（如随机数/姓名生成）上探索过模型内重复（West and Potts, 2025；Zhang et al., 2024）。相比之下，我们在真实世界开放式问题上做大规模研究，覆盖 70+ 个 LM（正文详述 25 个，代表各主要模型族最强或最大模型），给出跨模型输出收敛的首次系统分析。完整实验设置、全模型结果与样例见附录 C。

Figure 5: The heatmap shows degree of repetition in responses to open-ended queries generated by the same LMs. Using min-p sampling with parameters (top-p $=1.0$, min-p $=0.1$, temperature $=2.0$), the average pairwise similarity among responses to the same prompt typically exceeds 0.8.

图 5：热图显示 同一 LM 对开放式查询生成回答的重复程度。使用 min-p 采样（top-p $=1.0$，min-p $=0.1$，温度 $=2.0$）时，对同一提示的回答平均成对相似度通常超过 0.8。

Intra-model repetition. For each model, we sample 50 responses per query from Infinity-Chat100, compute the average pairwise embeddings similarity within each response pool[^1], and report the percentage of queries falling into different similarity ranges. Despite using high-stochasticity decoding parameters (top-p $=0.9$, t $=1.0$), responses from the same model remain highly repetitive, as shown in Figure 4: in 79% of cases, the average similarity exceeds 0.8. Since higher temperatures tend to produce incoherent text, these results show that even under maximally aggressive sampling, LMs still fail to generate diverse responses to open-ended queries.

模型内重复。 对每个模型，我们从 Infinity-Chat100 每查询采样 50 条回答，计算回答池内平均成对嵌入相似度[^1]，并报告落入不同相似度区间的查询占比。尽管使用高随机性解码参数（top-p $=0.9$，t $=1.0$），同一模型的回答仍高度重复，如图 4：79% 的情形平均相似度超过 0.8。更高温度往往产生不连贯文本，这些结果表明即便在尽可能激进的采样下，LM 仍无法对开放式查询生成多样回答。

Recent work introduces min-p decoding Nguyen et al. (2025), a dynamic strategy for enhancing generation diversity that adjusts the sampling threshold based on model confidence. We evaluate min-p decoding with the same setup and compute pairwise sentence embedding similarities. As shown in Figure 5, while min-p reduces extreme repetition (fewer pairs above 0.9), $81\%$ of response pairs still exceed 0.7 similarity and $61.2\%$ exceed 0.8, revealing mode collapse even under diversity-oriented decoding.

min-p 解码。 近期工作提出 min-p 解码（Nguyen et al., 2025），一种基于模型置信度动态调整采样阈值、以增强生成多样性的策略。我们以相同设置评估 min-p，并计算成对句嵌入相似度。如图 5，虽 min-p 减少极端重复（超过 0.9 的对更少），仍有 81% 的回答对相似度超过 0.7、61.2% 超过 0.8，说明面向多样性的解码下仍存在模式崩塌。

Despite its promise, min-p is not widely adopted, as it is better suited for creative tasks and less effective for close-ended ones Nguyen et al. (2025). More broadly, inference-time interventions such as decoding strategies are an important part of the solution, but relying on them alone places much of the burden on users to select and configure appropriate methods. We therefore view model training as another crucial component of a broader, multifaceted approach: training-time solutions can help preserve output diversity more robustly and by default, without requiring user intervention at inference time. For the complete breakdown of results across all models, see §Appendix C.2.

尽管有前景，min-p 尚未被广泛采用，因其更适创意任务、对封闭式任务效果较差（Nguyen et al., 2025）。更广地看，解码策略等推理时干预是解决方案的重要部分，但单靠它们会把选择与配置合适方法的负担大半放在用户身上。因此我们把模型训练视为更广多管齐下路径的另一关键组件：训练时方案可默认、更稳健地保留输出多样性，而无须用户在推理时干预。全模型结果分解见附录 C.2。

Figure 6: Average pairwise sentence embedding similarities between responses from different models reveal substantial semantic overlap across model outputs. Qualitative examples further illustrate that different models often produce strikingly similar responses to fully open-ended queries, including extended verbatim spans, underscoring the extent of repetition across models in open-ended generation tasks. All responses are generated using top-p $=0.9$ and temperature $=1.0$.

图6:将不同模型的答复之间的相似性嵌入到平均对等句中,显示出模型产出之间的语义上的巨大重叠。 定性实例进一步说明,不同模式往往对完全开放的询问作出惊人的类似答复,包括延长逐字记录,强调在不限成员名额的一代任务中不同模式之间重复的程度。 所有答复均使用顶p $=0.9$和温度$=1.0$生成。

Inter-model homogeneity. Not only do individual models repeatedly generate similar content, but different model sizes and families also produce highly repetitive outputs, sometimes sharing substantial phrase overlaps. As shown in Figure 6, the average pairwise similarity between responses from different models ranges from $71\%$ to $82\%$, with some pairs notably higher. For example, DeepSeek-V3 and qwen-max-2025-01-25 share a similarity of $0.82$, while DeepSeek-V3 and gpt-4o-2024-11-20 reach $0.81$. Interestingly, OpenAI’s GPT models and Qwen’s API models tend to have higher similarities even with models outside their own families. Although the exact causes remain unclear due to proprietary training details, possible explanations include shared data pipelines across regions or contamination from synthetic data. We highlight the need for future work to rigorously investigate the sources of such cross-model repetition.

模型间同质化。 不仅单个模型反复生成相似内容，不同规模与模型族也产出高度重复的输出，有时共享大量短语重叠。如图 6，不同模型回答之间的平均成对相似度介于 71% 到 82%，部分模型对更高。

Beyond general trends, we further analyze how repetition emerges at the instance level. As in prior work Li et al. (2024b); Namuduri et al. (2025), we observe verbatim phrase overlaps within responses from the same model. Surprisingly, such overlaps are also prevalent across different models, even for fully open-ended queries with large output spaces. For example, Figure 6 shows that DeepSeek-V3 and gpt-4o-2024-11-20 generate overlapping phrases like “Elevate your iPhone with our,” “sleek, without compromising,” and “with bold, eye-catching” in answer to the query “Create a description with 2-3 sentences for an iPhone case collection that is a slim-fitted case with bold designs.” In some cases, models output identical responses: for “Generate a motto for a social media page focused on successes, wealth, and self-help,” both qwen-max-2025-01-25 and qwen-plus-2025-01-25 generate “Empower Your Journey: Unlock Success, Build Wealth, Transform Yourself.” These instance-level verbatim overlaps illustrate the severity of the “Artificial Hivemind” effect across models. Paraphrases of the same open-ended queries also lead to verbatim overlaps, as illustrated in Tables 17–18 in § Appendix C.4.

除了一般趋势外,我们进一步分析在实例一级如何出现重复。 同先前的工作Li等人(2024年b);Namuduri等人(2025年)一样,我们观察到同一种模型在答复中出现重叠。 令人惊讶的是,这些重叠也普遍存在于不同的模型中,甚至对于输出空间大的完全开放的查询也是如此. 例如,图6显示,DeepSeek-V3和gpt-4o-2024-11-20产生重叠短语,如“用我们的iPhone“sleek,不妥协”和“用大胆的、捕眼”来回答“为iPhone案例集创造2-3句的描述,该案例集是一个带有大胆设计的微小尺寸案例”。 有些情况下,模型会输出相同的答复 : 对于“为社交媒体网页提供一条以成功、财富和自助为重点的座右铭 ” , Quen-max-2025-01-25 和 qwen-plus-2025-01-25 都会产生“Empower Your Journey: Unlock Eurney, Building Wealth, Transform yourself ” 。 这些实例层面的逐字记录重叠说明了“人工蜂窝”效应在各种模型中的严重性。 如附录C.4表17至表18所示,对相同的无限制查询的表述也导致逐字记录重叠。

Beyond surface-level overlap, repetition also manifests semantically: models converge on core ideas using different phrasing. As shown in Figure 1 (more in Figure 15-18), for the query “Write a metaphor about time,” 50 responses from each of 25 models form just two primary visual clusters: a dominant one centered on “time is a river” and a secondary one on “time is a weaver.” This convergence of abstract concepts reveals the “Artificial Hivemind” exposed in more subtle forms.

除了表层重叠外,重复还表现成群:模型使用不同的词句在核心思想上汇合. 如图1(见图15-18)所示,对于“写一个比喻时间”的问题,25个模型中的每个模型的50个答复仅组成两个主要视觉集群:一个以“时间为河流”为中心,另一个以“时间为织物”为中心。 抽象概念的这种趋同揭示了以更微妙的形式暴露的“人工蜂窝”。

Figure 7: The left histogram shows the distribution of Shannon entropy across the 25 human annotations for each (Query, Response 1, Response 2) triplet, where annotators judge which response is better. Given open-ended queries, multiple high-quality responses are possible, often leading to disagreement among annotators and, on average, high entropy. With 25 annotations, label distributions can vary widely across triplets. The middle panel presents example triplets from different entropy regions, and the right bar plots show their corresponding label distributions.

图7:左直方图显示Shannon Entropy在25个人类注解中的分布情况,每张说明**(查询,反应1,反应2)三重体,其中注释者判断哪个反应更好。 考虑到开放式的询问,多种高质量的答复是可能的,往往导致注释者之间的分歧,而且平均来说会导致高 en. 有了25个注解,标签的分布可以在三胞胎之间有很大的差异. ** 中间 面板展示了来自不同 en区的例子三重体,** 右** 条图显示了它们相应的标签分布.

Figure 8: The avg.# of unique source models among the top-$N$ most similar responses to each open-ended query across 25 LMs.

图8:顶端-$N$中对25个LMs的每个无限制查询最相似的答复中唯一来源模型的avg.#。

To quantify response uniformity across models, we examine the extent to which outputs from different models become indistinguishable from one another. Given 25 unique models, each generating 50 outputs to queries from Infinity-Chat, we ideally expect greater diversity across different models than from within a single model. To measure this, we identify the top $N$ most similar outputs for each query and count the unique models contributing to that set. A higher count suggests stronger cross-model similarity. As shown in Figure 8, the most similar responses often originate from multiple models. For instance, with $N=50$, perfectly disjoint responses would yield all 50 from a single model. Yet, we find an average of $\sim$8 unique models per top-50 cluster, with some queries exceeding 10, indicating distinct models frequently generate highly similar content, sometimes resulting in higher inter- than intra-model similarity. See full inter-model homogeneity results in §Appendix C.3.

为量化跨模型回答均一性，我们考察不同模型输出在多大程度上彼此不可区分。 给定 25 个不同模型，各对 Infinity-Chat 查询生成 50 条输出，理想上应期望跨模型多样性高于单模型内部。为此，我们对每查询识别 top $N$ 最相似输出。

To summarize, our work provides further evidence of high syntactic repetition across different models. While a full causal analysis is beyond the scope of this study, our findings motivate future research to investigate whether such repetition arises from pretraining data, alignment processes, memorization, contamination, or generalization.

总结而言，我们的工作进一步证明不同模型之间存在高句法重复。完整因果分析超出本文范围，但发现促使未来研究去探究这种重复是否源于预训练数据、对齐过程、记忆、污染或泛化。

Figure 9: The left histogram shows the distribution of Shannon Entropy computed over the 25 human ratings for each (Query, Response) pair. Given the open-ended nature of the queries, multiple responses can be valid, leading to diverse preferences across annotators. As a result, the annotation label distributions vary significantly across examples. The middle panel presents representative (Query, Response) pairs from different entropy regions, and the right bar plots display their corresponding label distributions.

图9:左直方图显示Shannon Entropy的分布情况,每张**(查询,答复)**对25人收视率计算。 鉴于询问的开放性,多种答复可以有效,从而导致各注释者有不同的偏好。 因此,注释标签的分布因实例而大不相同. 中面板呈现来自不同 en区的代表性(查询,回应)配对,右条图显示其相应的标签分布.

4 How Do LMs, Reward Models, and LM Judges Handle Alternative Responses to Open-Ended Queries?

4 语言模型、奖励模型与 LM 裁判如何处理开放式查询的替代回答？

Having established the generative homogeneity of LMs, in this section, we examine whether the ratings of LMs, reward models, and LM judges are calibrated to match human scores given different responses to open-ended queries from Infinity-Chat.

在确立 LM 的生成同质化之后，本节考察 LM、奖励模型与 LM 裁判 的评分，在面对 Infinity-Chat 开放式查询的不同回答时，是否与人类分数校准一致。

4.1 Gathering Distributional Annotations Across Many Humans

4.1 跨大量人类收集分布性标注

Humans may have divergent preferences over similar-quality alternative responses to open-ended queries. To study how models handle such diversity, we need densely annotated data that captures distributional human preferences. Existing alignment datasets, like HelpSteer3 Wang et al. (2025c), typically contain only sparse labels (e.g., 3 annotators per item). To address this, we collect both absolute ratings (1–5 scale for response quality) and pairwise preference ratings (strong/weak preference between two responses to the same query), each with extensive annotations. For absolute ratings, we sample 15 responses for each of 50 prompts from Infinity-Chat100 and collect 25 ratings per (Query, Response), yielding $25\times 15\times 50=18{,}750$ labels. For pairwise preference rating, we sample 10 response pairs per prompt and gather 25 annotations per (Query, Response 1, Response 2), totaling $25\times 10\times 50=12{,}500$ labels. This is the first large-scale human-annotated dataset with dense human ratings on alternative responses to the same open-ended queries, providing both absolute and pairwise preference labels for fine-grained analysis of human idiosyncratic and collective preference distributions. Full details of the human annotation process are provided in §Appendix D.1.

人类对于类似质量的对无限制询问的替代性答复,可能有不同的偏好。 为了研究模型如何处理这种多样性,我们需要大量附加说明的数据,以记录人类的分布偏好。 现有的对齐数据集,如HelpSteer3 Wang et al. (2025c),通常只包含少量的标签(如每件3个注释器). 为了解决这个问题,我们收集了“绝对的评分* (对应质量的1至5分)”和“不公平的偏好评分*(对同一查询的两个答复之间强烈/弱的偏好)”,每个都附有广泛的说明。 对于绝对的收视率,我们从Infinity-Chat100的50个提示中分别抽取15个响应,并收集25个收视率(Query, Response),产生$25\times 15\times 50=18{,}750$标签. 对于双向偏好评分,我们每秒抽取10对响应对,并收集每对25个注释(查询,反应1,反应2),总计$25\times 10\times 50=12{,}500$标签. 这是首个人类附加说明的大规模数据集,对相同开放性查询的替代响应给出了密集的人类评分,为人类特异性和集体偏好分布的精细分析提供了绝对和对偶偏好标签. 人类注释过程的全部细节见§Appendix D.1.

Figure 7 shows the distribution of Shannon entropy over human preference annotations for (Query, Response 1, Response 2) triplets. Annotators often disagree on which response is better, resulting in entropy skewed toward the higher end. As shown by the bar charts on the right, label distributions vary widely across examples: some response pairs show near-uniform support across all options, indicating substantial annotator disagreement for alternative responses of open-ended queries. Figure 9 shows a similar trend in the entropy of human annotations for absolute ratings of (Query, Response) pairs.

图7显示Shannon Entropy相对于人类偏好说明(Query, Response 1, Response 2)三胞胎的分布. 注释者往往对哪个反应更好持不同意见,结果造成 en向更高端倾斜。 如右侧的条形图所示,标签的分布因实例而大不相同:一些响应对子显示在所有选项中都有近乎统一的支持,表明对开放式询问的替代答复存在实质性的注释者分歧. 图9显示人类注解对对对(查询,回应)的绝对评分有相似的趋势.

4.2 Gathering LMs, Reward Models, and LM Judges Ratings

4.2 收集语言模型、奖励模型与 LM 裁判的评分

We aim to assess how LMs, reward models, and LM judges align with human ratings when evaluating alternative responses to open-ended queries. Specifically, we compare 3 types of model-generated ratings against human annotations. LM scores are derived from response perplexity given the query. Reward model scores are based on standardized scalar reward outputs. LM judge ratings follow standard prompting protocols using two rubrics: an overall quality score and the HHH rubric (Helpfulness, Harmlessness, Honesty) Bai et al. (2022a). See §Appendix D.2 for the full list of 56 state-of-the-art LMs, 6 top-ranked reward models (per RewardBench Lambert et al. (2024)), 4 LM judges (including GPT-4o and Prometheus Kim et al. (2024) variants), and details on the rating procedures.

我们的目标是在评价对不限名额的询问的替代答复时,评估LMs、奖励模型和LM法官如何与人的评级保持一致。 具体来说,我们比较了三种模式生成的收视率和人类说明. LM的分数来源于由于询问而导致的响应偏差. ** 奖励模式** 分数以标准分级奖励产出为依据。 ** LM judge** 评分遵循标准提示协议,使用两个分数:整体质量分数和HHH分数(帮助,无助,诚实)Bai等人(2022a). 见§Appendix D.2 完整的56个最先进的LM,6个排名最高的奖励模式(per Reward Bench Lambert等人(2024年)),4个LM评委(包括GPT-4o和Prometheus Kim等人(2024年)的变体),以及评分程序的细节.

Figure 10: We compute Spearman’s correlation coefficients between human-annotated and model-generated absolute rating scores, including LM perplexity scores, reward model scalar outputs, and LM judge scalar ratings. Correlations are calculated for the full set, as well as two groups of subsets: (a) responses with similar human-rated quality, and (b) responses with high human disagreement. The results show that correlations are notably lower in these two subsets, indicating weaker alignment between model scores and human judgments in cases of subtle or contested quality differences.

图10:我们计算斯皮尔曼在人类附加说明和模型生成的绝对评分分之间的关联系数,包括LM复杂评分,奖励模型标分输出,以及LM评分标分评分. 对全集以及两组子集的关联性进行了计算:(a) 具有相似的人类评级质量 的响应;(b) 具有人类高度分歧 的响应。 结果显示,这两个子集的相关性明显较低,表明在质量差异微妙或有争议的情况下,模型分数与人类判断之间的比对较弱。

4.3 Comparing Model Ratings to Human Scores for Responses to Open-Ended Queries

4.3 将模型评分与人类对开放式查询回答的打分比较

We examine how model ratings align with human judgments on (1) similar-quality alternative responses to the same open-ended queries and (2) responses with high annotator disagreement.

我们研究了模型评级如何与人类对(1) * 相同质量的替代答复* 对相同开放式询问的判断一致,(2) 与* 高级注释者意见不一致* 的答复。

Motivation for comparing model scores to average human ratings. Our motivation stems from how reward models (or LM judges) are used in training to evaluate responses to open-ended queries without a single ground truth. Different annotators may prefer different answers, yet their average ratings are often similar, implying multiple responses can be equally high-quality. Current reward models, however, fail to capture this equivalence, assigning diverging scores and causing downstream models to overvalue one response despite comparable human approval. To address this, we collect 25 human ratings per example to capture diverse preferences, using the average score to reflect shared human judgment. We then test whether LMs, reward models, and LM judges correlate less reliably with responses that humans broadly consider comparably good, hence our choice to compute human correlation using average human ratings.

** 将模型分数与人类平均分数相比较的动机。 ** 我们的动机来自在培训中如何利用奖励模式(或LM法官)来评价对没有单一依据的无限制询问的答复。 不同的注解者可能更喜欢不同的答案,然而他们的平均评分往往相似,这意味着多个响应同样可以达到高品质. 然而,目前的奖励模式未能抓住这种等同性,分配了不同的分数,导致下游模式不顾类似的人类认可而高估一个响应。 为了解决这个问题,我们收集了每个例子的25个人类评级,以捕捉不同的偏好,利用平均得分来反映人类的共同判断. 然后,我们检验LM、奖励模型和LM评断是否与人类广泛认为相当好的反应有不太可靠的关联,因此我们选择使用人类的平均评分计算人类的相关性。

Models show weaker alignment with human ratings for alternative responses of similar quality. We hypothesize that models are less aligned with human judgments on similar-quality examples, as models are typically trained with more clearly differentiated responses.

** 模型显示,在类似质量的替代反应方面,与人类评级的一致程度较低。 ** 我们假设模型与人类对类似质量实例的判断不太一致,因为模型通常通过更明确区分的对策加以培训。

For the absolute rating setup, we identify similar-quality (Query, Response) pairs by filtering out outliers using Tukey’s fences Data et al. (2016). This method defines outliers as points beyond $Q_{1}-k\cdot\mathrm{IQR}$ or $Q_{3}+k\cdot\mathrm{IQR}$, where $Q_{1}$ and $Q_{3}$ are the 25th and 75th percentiles and $\mathrm{IQR}=Q_{3}-Q_{1}$. We vary the constant $k$ from 0.5 (aggressive filtering) to 3.0 (conservative filtering) in increments of 0.5 to generate subsets of increasing similarity. We then compute Pearson correlations between model and human absolute ratings on the full set and these filtered subsets, as shown in Figure 10 (a). For the pairwise preference rating setup, we identify similar-quality (Query, Response 1, Response 2) triplets by ranking examples by how many annotators rate the two responses as similar in quality. We select the top 60%–95% most similar examples to form subsets, with the 60% subset containing examples of higher similarity. We then compute Pearson correlations between the score differences of model ratings and those of human ratings across the full set and each subset, as shown in Figure 11 (a).

对于绝对评级的设置,我们通过使用Tukey的栅栏过滤出输出点来识别出类似质量的对(查询,反应). Data等人(2016年). 这种方法将输出点定义为$Q_{1}-k\cdot\mathrm{IQR}$或$Q_{3}+k\cdot\mathrm{IQR}$以外的点,其中$Q_{1}$和$Q_{3}$是第25和第75百分位数和$\mathrm{IQR}=Q_{3}-Q_{1}$. 我们把恒定的$k$从0.5(主动过滤)到3.0(保守过滤)的递增0.5不等,以产生越来越相似的子集. 然后,我们计算整个集的模型和人类绝对收视率与这些被过滤的子集之间的皮尔逊关系,如图10(a)所示. 对于*pairwise professional service * 设置,我们通过根据有多少个注释者将两个答复评为在质量上相似,来排名实例,来识别相似的质量(Query, Response 1, Response 2)三重. 我们选择了前60\95%最相似的例子来组成子集,其中60%的子集包含更高相似性的例子. 然后我们计算整个集和每个子集的模型评级与人类评级的分数差异之间的皮尔逊关联,如图11(a)所示.

Figure 11: We compute Spearman’s correlation coefficients between human-annotated and model-generated pairwise preference rating scores, comparing the full set to two groups of subsets: (a) responses with similar human-rated quality, and (b) responses with high human disagreement.

图11:我们计算了Spearman在人为附加说明和模型生成的** Pairwise偏好评分分数之间的关联系数,将完整套分** 与两组子集进行对比:(a) 与相似的人类评级质量 和(b) 与** 人类高度分歧** 的对应。

Our results show that correlations between human ratings and those of LMs, reward models, and LM judges drop significantly on similar-quality subsets, for both absolute and pairwise preference rating setups. Since there is no single gold-standard approach for selecting subsets of responses with similar quality given our data structure, we additionally report results using alternative subset selection methods in Table 19 (§ Appendix D.3). Our findings remain consistent across methods, highlighting the need for better modeling of fine-grained distinctions among equally high-quality responses to open-ended queries. For full results, including alternative grouping methods and model-level breakdowns, see § Appendix D.3.

• 我们的结果表明,在绝对和对等优先评级设置方面,人类评级与LMs、奖励模型和LM评委之间的相关性显著下降。 * 由于鉴于我们的数据结构,没有单一的黄金标准办法来选择质量类似的答复子集,因此我们还使用表19中的备选子集选择方法(§附录D.3)报告结果。 我们的调查结果在各种方法上仍然一致,强调必须更好地在对不限名额的询问作出同样高质量的答复之间作出细微的区分。 全部结果,包括备选分组方法和模型一级细分,见附录D.3。

Model judgments are less aligned where annotators disagree. We hypothesize that model ratings are less calibrated to human judgments on examples with high annotator disagreement, as models are primarily trained on examples with higher human agreement.

** 在注释者不同意的情况下,示范判决不太一致。 ** 我们假设,模型的评级不太符合人类对高度注释者意见不一的例子的判断,因为模型主要是在人类认同程度较高的事例上接受训练。

For the absolute rating setup, we identify disagreement by ranking (Query, Response) pairs by Shannon entropy across 25 human labels. We then select the top ${2,4,6,8,10,12,14,16}$% highest-entropy examples as disagreed subsets. Figure 10 (b) shows the Pearson correlations between model and human ratings across the full set and these subsets. For the pairwise preference rating setup, we quantify disagreement for each (Query, Response 1, Response 2) triplet using percentage disagreement: $P_{\text{disagree}}=1-\frac{\max(C_{\text{prefer 1}},C_{\text{prefer 2}})+0.5\cdot C_{\text{tie}}}{C_{\text{total}}}$, where $C$ denotes the count of annotations per preference type. We retain the top 60% to 95% most disagreed examples, with the 60% subset representing stronger disagreement. Pearson correlations between model and human score differences across the full set and each subset are shown in Figure 11 (b).

对于绝对的评级设置,我们通过Shannon Entropy在25个人类标签上的排名(查询,反应)来识别分歧. 然后,我们选择最高的 ${2,4,6,8,10,12,14,16}$% 最高的示例作为不同意的子集 。 图10(b)显示了整个集和这些子集的模型与人类评级之间的相互关系。 对于 * 偏好评分 * 设置, 我们用$P_{\text{disagree}}=1-\frac{\max(C_{\text{prefer 1}},C_{\text{prefer 2}})+0.5\cdot C_{\text{tie}}}{C_{\text{total}}}$表示每个偏好类型的注解数, 对每个偏好( 查询, 反应 1, 反应 2) 三进制进行量化 。 我们保留了最大的60%至95%的不同意见,60%的子集代表了更强烈的分歧。 整个集的模型和人类得分差异之间的皮尔逊关联,每个子集如图11(b)所示.

Our results show that correlations with human ratings across models drop substantially for examples with high annotator disagreement, in both absolute and pairwise rating setups. We also report results using alternative subset selection methods in Table 20 (§ Appendix D.4). The findings remain consistent across methods, highlighting the need for more nuanced modeling of idiosyncratic human disagreement to better capture the broad spectrum of open-ended possibilities. For complete results, including alternative grouping methods and model-level breakdowns, see § Appendix D.4.

• 我们的结果显示,在绝对评级和对等评级设定方面,与不同模型的人类评级的关联性,对于有高度注释者分歧的例子,则大幅下降。 我们还使用表20中的替代子集选择方法报告结果(§附录D.4)。 各种方法的研究结果仍然一致,突出表明需要更细致地模拟人类的特异性分歧,以更好地抓住广泛的无限制可能性。 完整的结果,包括备选分组方法和模型层次的细分,见附录D.4。

5 Related Work

5 相关工作

The diversity collapse problem of LMs. Diversity collapse, characterized by the inability of LMs to generate diverse outputs, presents a significant challenge to pluralistic alignment research (Zhang et al., 2024; Shumailov et al., 2023; Dohmatob et al., 2024; Yang et al., 2024; Li et al., 2024b; West and Potts, 2025). Prior studies identify some key factors contributing to diversity collapse, including training on synthetic data (Havrilla et al., 2024; Wang et al., 2024; Yang et al., 2024; Shumailov et al., 2024), LM alignment (Murthy et al., 2024a; Kirk et al., 2024a; Kirk et al., 2023), and insufficient diversity in training data (Chen et al., 2024b). Potential consequences of diversity collapse include reduced creativity, loss of minority perspectives, spread of bias, and overall decline in model utility and trustworthiness (Anderson et al., 2024a; Kapania et al., 2024; Deshpande et al., 2025). In response, a range of mitigation strategies are proposed, such as training corpora diversification (Wang et al., 2025a; Pang et al., 2024; Wang et al., 2025b; Havrilla et al., 2025), training algorithm modifications (Zhang et al., 2025b; Li et al., 2025), alternative decoding (Vijayakumar et al., 2016; Nguyen et al., 2025) and prompting (Zhang et al., 2025a; Bradley et al., 2023) strategies.

** LMs.的多样性崩溃问题** 多样性崩溃,其特点是LMS无法产生不同的产出,对多元对齐研究提出了重大挑战(Zhang等人,2024年;Shumailov等人,2023年;Dohmatob等人,2024年;Yang等人,2024年;Li等人,2024年b年;West and Potts,2025年)。 先前的研究确定了造成多样性崩溃的一些关键因素,包括合成数据培训(Havrilla等人,2024年;Wang等人,2024年;Yang等人,2024年;Shumailov等人,2024年);LM校正(Murthy等人,2024年a;Kirk等人,2024年a;Kirk等人,2023年),以及培训数据多样性不足(Chen等人,2024年b)。 多样性崩溃的潜在后果包括创造力下降、少数群体观点丧失、偏见蔓延以及模型效用和可信赖性全面下降(Anderson等人,2024a;Kapania等人,2024;Deshpande等人,2025)。 对此,提出了一系列缓解战略,如培训公司多样化(Wang等人,2025a;Pang等人,2024b;Wang等人,2025b;Havrilla等人,2025),培训算法修改(Zhang等人,2025b;Li等人,2025),替代解码(Vijayakumar等人,2016年;Nguyen等人,2025年)和激励(Zhang等人,2025a;Bradley等人,2023年)。

Measuring the creativity and divergent thinking of language models. Recent efforts to measure the creativity and divergent thinking of LMs often adapt established psychometric tests. For example, (Chen and Ding, 2023) utilizes a divergent association task (DAT) by asking models to generate semantically distant or unrelated words (Olson et al., 2021). Similarly, tasks such as the Alternate Uses Test (AUT) (Rabeyah et al., 2024), the Torrance Tests of Creative Thinking (TTCT) (Alabbasi et al., 2022; Chakrabarty et al., 2024), Human Evaluation, and LLM-as-a-judge (Krumdick et al., 2025) are employed to assess dimensions like fluency, originality, complexity, and effective semantic diversity (Shypula et al., 2025) of LM responses. Despite these demonstrated capabilities, LLM-generated creative content tends towards homogeneity, even when individual outputs achieve high creativity scores (Wenger and Kenett, 2025). To address these evaluation complexities, some benchmarks, such as (Ruan et al., 2024; Lu et al., 2024; Padmakumar et al., 2025; Zhang et al., 2025b), focus on specific creative abilities like scientific idea and code generation. Other works propose new metrics (Namuduri et al., 2025). While LMs show promise in creative tasks, comprehensively evaluating their creativity remains an active and challenging research area (Ismayilzada et al., 2024). We conduct a large-scale systematic study of real-world open-ended user queries and provide a comprehensive taxonomy, query dataset, and dense human annotations to improve evaluation and model training for reducing mode collapse in language models.

** 衡量语言模式的创造性和不同思维。 ** 最近为衡量LMS的创造力和不同思维所做的努力,常常适应既定的心理测量测试。 例如,(Chen and Ding, 2023)使用不同的关联任务(DAT),要求模型生成模棱两可的遥远或不相关的单词(Olson等人, 2021). 同样,诸如替代用途测试(AUT)(Rabeyah等人,2024年)、创造性思维的托伦斯测试(TTCT)(Alabbasi等人,2022年;Chakrabarty等人,2024年)、人类评估以及LLM-as-a-judge(Krundick等人,2025年)等任务也被用来评估LM响应的流畅度、原创性、复杂性和有效语义多样性(Shypula等人,2025年)。 尽管有这些展示的能力,LLM生成的创作内容往往趋向于同质性,即使个人产出取得了较高的创作分数(Wenger and Kenett, 2025). 为解决这些评价的复杂性,一些基准,例如(Ruan等人,2024年;Lu等人,2024年;Padmakumar等人,2025年;Zhang等人,2025年b)侧重于科学思想和代码生成等具体创造性能力。 其他作品提出了新的衡量标准(Namuduri等人,2025年)。 虽然LMs在创造性任务中表现出希望,但全面评估其创造性仍然是一个积极和具有挑战性的研究领域(Ismayilzada等人,2024年)。 我们对现实世界的开放式用户查询进行大规模系统研究,并提供全面的分类学,查询数据集,以及密集的人类说明,以改进评价和模型培训,以减少语言模型模式崩塌.

Disagreement and pluralistic alignment of language models. Advances in AI value alignment research have substantially improved LM utility and safety, through enhanced training processes (Schulman et al., 2017; Ouyang et al., 2022; Rafailov et al., 2023), and the use of both synthetic data (Ge et al., 2024) and human datasets (Bai et al., 2022b; Ganguli et al., 2022). Yet, a significant challenge remains: the potential for monolithic value representation (Ryan et al., 2024b; Ryan et al., 2024a). In contrast, the emerging focus on pluralistic alignment emphasizes the need for AI to serve the varied demands of a wide population (Sorensen et al., 2024; Alamdari et al., 2024). This shift is driving innovation in methods (Chung et al., 2025; Nguyen et al., 2025; Lake et al., 2024; Chen et al., 2024a; Feng et al., 2024; Srewa et al., 2025), benchmarks (Castricato et al., 2024), and data collection strategies (Kirk et al., 2024b; Shi et al., 2025) to support this vision of diversity. Additionally, approaches leveraging multiple LMs interacting through system messages are also being explored to boost variety (Verga et al., 2024; Chen et al., 2024c; Murthy et al., 2024b). Parallel efforts are dedicated to quantifying and improving the cultural diversity exhibited by LLMs (Rao et al., 2025; Shi et al., 2024; Chiu et al., 2024; Myung et al., 2025; Li et al., 2024a). Yet, a common characteristic of many existing pluralistic alignment work is its reliance on predefined diversity dimensions, like demographics (Kwok et al., 2024; Moon et al., 2024), personality style (Lee et al., 2025; Jiang et al., 2023; Zhu et al., 2025; Padmakumar and He, 2024), and cultural background (Xu et al., 2024; Diamond, 2025; AlKhamissi et al., 2024). To enable models that genuinely cater to individuality without relying on stereotypes, individual-level alignment is needed (Jiang et al., 2025a; Zhu et al., 2025).

** 语文模式的分歧和多元统一。 ** 通过强化培训程序(Schulman等人,2017年;Ouyang等人,2022年;Rafailov等人,2023年),以及使用合成数据(Ge等人,2024年)和人类数据集(Bai等人,2022年b;Ganguli等人,2022年),AI价值校正研究的进展大大提高了LM的效用和安全性。 然而,仍然存在一项重大挑战:单一价值代表的潜力(Ryan等人,2024年b;Ryan等人,2024年a)。 与此相对照,对多元化调整的日益重视强调AI需要满足广大民众的各种需求(Sorensen等人,2024年;Alamdari等人,2024年)。 这一转变正在推动方法创新(Chung等人,2025年;Nguyen等人,2025年;Lake等人,2024年;Chen等人,2024年a;Feng等人,2024年;Srewa等人,2025年)、基准(Casticato等人,2024年)和数据收集战略(Kirk等人,2024年b;Shi等人,2025年),以支持这种多样性观点。 此外,还在探索利用多种LMS通过系统信息进行互动的方法,以促进多样性(Verga等人,2024年;Chen等人,2024年c;Murthy等人,2024年b)。 同时还致力于量化和改善LLMS所展现的文化多样性(Rao等人,2025年;Shi等人,2024年;Chiu等人,2024年;Myung等人,2025年;Li等人,2024年a)。 然而,许多现有多元调整工作的一个共同特点是依赖预先界定的多样性层面,如人口学(Kwok等人,2024年;Moon等人,2024年);人格风格(Lee等人,2025年;Jiang等人,2023年;Zhu等人,2025年;Padmakumar和He,2024年);文化背景(Xu等人,2024年;Diamond,2025年;Al Khamissi等人,2024年)。 为使模式真正适合个人特点而不依赖陈规定型观念,需要个人层面的一致(Jiang等人,2025a;Zhu等人,2025)。

6 Conclusion

6 结论

Our work introduces Infinity-Chat, a large-scale resource designed to evaluate LMs’ diversity in naturally occurring, open-ended settings. Through comprehensive analysis, we uncover the “Artificial Hivemind” effect, highlighting both intra-model repetition and inter-model homogeneity of current LMs. By coupling a diverse taxonomy of prompts with dense human preference annotations, Infinity-Chat provides a new foundation for diagnosing, benchmarking, and ultimately mitigating mode collapse in generative AI. We hope this resource catalyzes future efforts to foster genuine diversity in model outputs and guard against the homogenization of human expression.

本文提出 Infinity-Chat，面向自然发生的开放式设定评测 LM 多样性的大规模资源。经综合分析，我们揭示“人工蜂群心智”效应，凸显当前 LM 的模型内重复与模型间同质化。通过将多样提示分类体系与稠密人类偏好标注结合，Infinity-Chat 为诊断、基准化并最终缓解生成式 AI 的模式崩塌提供新基础。我们希望该资源催化未来工作，培育模型输出的真正多样性，并防范人类表达的同质化。

Acknowledgment

致谢

This work was supported by DARPA under the ITM program (FA8650-23-C-7316). This work was also supported in part by the Defense Advanced Research Projects Agency’s (DARPA) SciFy program (Agreement No. HR00112520300). The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

这项工作得到了DARPA在ITM方案(FA8650-23-C-7316)下的支持.的共同支助还有国防高级研究项目局的SciFy方案(第HR00112520300号协议)。 表达的观点是作者的观点,并不反映国防部或美国政府的官方政策或立场.

References

参考文献

Alabbasi et al. (2022) A. M. A. Alabbasi, S. H. Paek, D. Kim, and B. Cramond What do educators need to know about the torrance tests of creative thinking: a comprehensive review . Frontiers in psychology 13 , pp. 1000385 . Cited by: §5 .

Alamdari et al. (2024) P. A. Alamdari, T. Q. Klassen, R. T. Icarte, and S. A. McIlraith Being considerate as a pathway towards pluralistic alignment for agentic ai . External Links: 2411.10613 , Link Cited by: §5 .

Alemohammad et al. (2024) S. Alemohammad, J. Casco-Rodriguez, L. Luzi, A. I. Humayun, H. Babaei, D. LeJeune, A. Siahkoohi, and R. Baraniuk Self-consuming generative models go MAD . In The Twelfth International Conference on Learning Representations , External Links: Link Cited by: §A.3 .

AlKhamissi et al. (2024) B. AlKhamissi, M. ElNokrashy, M. AlKhamissi, and M. Diab Investigating cultural alignment of large language models . External Links: 2402.13231 , Link Cited by: §5 .

Anderson et al. (2024a) B. R. Anderson, J. H. Shah, and M. Kreminski Homogenization effects of large language models on human creative ideation . In Proceedings of the 16th conference on creativity & cognition , pp. 413–425 . Cited by: §5 .

Anderson et al. (2024b) B. R. Anderson, J. H. Shah, and M. Kreminski Homogenization effects of large language models on human creative ideation . In Creativity and Cognition , pp. 413–425 . External Links: Link , Document Cited by: §A.3 .

Ashkinaze et al. (2025) J. Ashkinaze, J. Mendelsohn, L. Qiwei, C. Budak, and E. Gilbert How ai ideas affect the creativity, diversity, and evolution of human ideas: evidence from a large, dynamic experiment . In Proceedings of the ACM Collective Intelligence Conference , CI 2025 , pp. 198–213 . External Links: Link , Document Cited by: §A.3 .

Bai et al. (2022a) Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, N. Joseph, S. Kadavath, J. Kernion, T. Conerly, S. El-Showk, N. Elhage, Z. Hatfield-Dodds, D. Hernandez, T. Hume, S. Johnston, S. Kravec, L. Lovitt, N. Nanda, C. Olsson, D. Amodei, T. Brown, J. Clark, S. McCandlish, C. Olah, B. Mann, and J. Kaplan Training a helpful and harmless assistant with reinforcement learning from human feedback . External Links: 2204.05862 , Link Cited by: §D.2 , §4.2 .

Bai et al. (2022b) Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback . arXiv preprint arXiv:2204.05862 . Cited by: §5 .

Bommasani et al. (2022) R. Bommasani, K. A. Creel, A. Kumar, D. Jurafsky, and P. S. Liang Picking on the same person: does algorithmic monoculture lead to outcome homogenization? . In Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.) , Vol. 35 , pp. 3663–3678 . External Links: Link Cited by: §1 .

Bradley et al. (2023) H. Bradley, A. Dai, H. Teufel, J. Zhang, K. Oostermeijer, M. Bellagente, J. Clune, K. Stanley, G. Schott, and J. Lehman Quality-diversity through ai feedback . External Links: 2310.13032 , Link Cited by: §5 .

Castricato et al. (2024) L. Castricato, N. Lile, R. Rafailov, J. Fränken, and C. Finn PERSONA: a reproducible testbed for pluralistic alignment . External Links: 2407.17387 , Link Cited by: §5 .

Chakrabarty et al. (2024) T. Chakrabarty, P. Laban, D. Agarwal, S. Muresan, and C. Wu Art or artifice? large language models and the false promise of creativity . In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems , CHI ’24 , New York, NY, USA . External Links: ISBN 9798400703300 , Link , Document Cited by: §1 , §5 .

Chen et al. (2024a) D. Chen, Y. Chen, A. Rege, and R. K. Vinayak PAL: pluralistic alignment framework for learning from heterogeneous preferences . External Links: 2406.08469 , Link Cited by: §5 .

Chen et al. (2024b) H. Chen, A. Waheed, X. Li, Y. Wang, J. Wang, B. Raj, and M. I. Abdin On the diversity of synthetic data and its impact on training large language models . External Links: 2410.15226 , Link Cited by: §5 .

Chen and Ding (2023) H. Chen and N. Ding Probing the creativity of large language models: can models produce divergent semantic association? . arXiv preprint arXiv:2310.11158 . Cited by: §5 .

Chen et al. (2024c) J. C. Chen, S. Saha, and M. Bansal ReConcile: round-table conference improves reasoning via consensus among diverse llms . External Links: 2309.13007 , Link Cited by: §5 .

Chiu et al. (2024) Y. Y. Chiu, L. Jiang, M. Antoniak, C. Y. Park, S. S. Li, M. Bhatia, S. Ravi, Y. Tsvetkov, V. Shwartz, and Y. Choi CulturalTeaming: ai-assisted interactive red-teaming for challenging llms’ (lack of) multicultural knowledge . External Links: 2404.06664 , Link Cited by: §5 .

Chung et al. (2025) J. J. Y. Chung, V. Padmakumar, M. Roemmele, Y. Sun, and M. Kreminski Modifying large language model post-training for diverse creative writing . External Links: 2503.17126 , Link Cited by: §5 .

Data et al. (2016) M. C. Data, M. Komorowski, D. C. Marshall, J. D. Salciccioli, and Y. Crutain Exploratory data analysis . Secondary analysis of electronic health records , pp. 185–203 . Cited by: §4.3 .

Deshpande et al. (2025) V. Deshpande, D. Ghose, J. D. Patterson, R. Beaty, and A. Rumshisky Diverse, not short: a length-controlled data selection strategy for improving response diversity of language models . External Links: 2505.16245 , Link Cited by: §5 .

Diamond (2025) A. Diamond PRISM: perspective reasoning for integrated synthesis and mediation as a multi-perspective framework for ai alignment . External Links: 2503.04740 , Link Cited by: §5 .

Dohmatob et al. (2024) E. Dohmatob, Y. Feng, A. Subramonian, and J. Kempe Strong model collapse . arXiv preprint arXiv:2410.04840 . Cited by: §5 .

Doshi and Hauser (2024) A. R. Doshi and O. P. Hauser Generative ai enhances individual creativity but reduces the collective diversity of novel content . Science Advances 10 ( 28 ), pp. eadn5290 . External Links: Document , Link , https://www.science.org/doi/pdf/10.1126/sciadv.adn5290 Cited by: §A.3 .

Feng et al. (2025a) S. Feng, W. Ding, A. Liu, Z. Wang, W. Shi, Y. Wang, Z. Shen, X. Han, H. Lang, C. Lee, T. Pfister, Y. Choi, and Y. Tsvetkov When one llm drools, multi-llm collaboration rules . External Links: 2502.04506 , Link Cited by: §1 .

Feng et al. (2024) S. Feng, T. Sorensen, Y. Liu, J. Fisher, C. Y. Park, Y. Choi, and Y. Tsvetkov Modular pluralism: pluralistic alignment via multi-llm collaboration . arXiv preprint arXiv:2406.15951 . Cited by: §5 .

Feng et al. (2025b) S. Feng, Z. Wang, P. Goyal, Y. Wang, W. Shi, H. Xia, H. Palangi, L. Zettlemoyer, Y. Tsvetkov, C. Lee, and T. Pfister Heterogeneous swarms: jointly optimizing model roles and weights for multi-llm systems . External Links: 2502.04510 , Link Cited by: §1 .

Feng et al. (2025c) S. Feng, Z. Wang, Y. Wang, S. Ebrahimi, H. Palangi, L. Miculicich, A. Kulshrestha, N. Rauschmayr, Y. Choi, Y. Tsvetkov, C. Lee, and T. Pfister Model swarms: collaborative search to adapt llm experts via swarm intelligence . External Links: 2410.11163 , Link Cited by: §A.3 .

Ganguli et al. (2022) D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, et al. Red teaming language models to reduce harms: methods, scaling behaviors, and lessons learned . arXiv preprint arXiv:2209.07858 . Cited by: §5 .

Ge et al. (2024) T. Ge, X. Chan, X. Wang, D. Yu, H. Mi, and D. Yu Scaling synthetic data creation with 1,000,000,000 personas . arXiv preprint arXiv:2406.20094 . Cited by: §1 , §5 .

Gerstgrasser et al. (2024) M. Gerstgrasser, R. Schaeffer, A. Dey, R. Rafailov, H. Sleight, J. Hughes, T. Korbak, R. Agrawal, D. Pai, A. Gromov, et al. Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data . arXiv preprint arXiv:2404.01413 . Cited by: §1 .

Havrilla et al. (2024) A. Havrilla, A. Dai, L. O’Mahony, K. Oostermeijer, V. Zisler, A. Albalak, F. Milo, S. C. Raparthy, K. Gandhi, B. Abbasi, D. Phung, M. Iyer, D. Mahan, C. Blagden, S. Gureja, M. Hamdy, W. Li, G. Paolini, P. S. Ammanamanchi, and E. Meyerson Surveying the effects of quality, diversity, and complexity in synthetic data from large language models . External Links: 2412.02980 , Link Cited by: §5 .

Havrilla et al. (2025) A. Havrilla, E. Hughes, M. Samvelyan, and J. Abernethy SPARQ: synthetic problem generation for reasoning via quality-diversity algorithms . External Links: 2506.06499 , Link Cited by: §5 .

Ismayilzada et al. (2024) M. Ismayilzada, D. Paul, A. Bosselut, and L. van der Plas Creativity in ai: progresses and challenges . arXiv preprint arXiv:2410.17218 . Cited by: §5 .

Jiang et al. (2023) G. Jiang, M. Xu, S. Zhu, W. Han, C. Zhang, and Y. Zhu Evaluating and inducing personality in pre-trained language models . External Links: 2206.07550 , Link Cited by: §5 .

Jiang et al. (2025a) L. Jiang, T. Sorensen, S. Levine, and Y. Choi Can language models reason about individualistic human values and preferences? . External Links: 2410.03868 , Link Cited by: §5 .

Jiang et al. (2025b) Y. Jiang, W. Ding, S. Feng, G. Durrett, and Y. Tsvetkov SPARTA alignment: collectively aligning multiple language models through combat . External Links: 2506.04721 , Link Cited by: §A.3 .

Kapania et al. (2024) S. Kapania, W. Agnew, M. Eslami, H. Heidari, and S. Fox ’Simulacrum of stories’: examining large language models as qualitative research participants . arXiv preprint arXiv:2409.19430 . Cited by: §5 .

Kazdan et al. (2025) J. Kazdan, R. Schaeffer, A. Dey, M. Gerstgrasser, R. Rafailov, D. L. Donoho, and S. Koyejo Collapse or thrive? perils and promises of synthetic data in a self-generating world . External Links: 2410.16713 , Link Cited by: §A.3 .

Kim et al. (2024) S. Kim, J. Suk, S. Longpre, B. Y. Lin, J. Shin, S. Welleck, G. Neubig, M. Lee, K. Lee, and M. Seo Prometheus 2: an open source language model specialized in evaluating other language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.) , Miami, Florida, USA , pp. 4334–4353 . External Links: Link , Document Cited by: §D.2 , §4.2 .

Kirk et al. (2024a) H. R. Kirk, B. Vidgen, P. Röttger, and S. A. Hale The benefits, risks and bounds of personalizing the alignment of large language models to individuals . Nature Machine Intelligence 6 ( 4 ), pp. 383–392 . Cited by: §5 .

Kirk et al. (2024b) H. R. Kirk, A. Whitefield, P. Röttger, A. Bean, K. Margatina, J. Ciro, R. Mosquera, M. Bartolo, A. Williams, H. He, B. Vidgen, and S. A. Hale The prism alignment dataset: what participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models . External Links: 2404.16019 , Link Cited by: §5 .

Kirk et al. (2023) R. Kirk, I. Mediratta, C. Nalmpantis, J. Luketina, E. Hambro, E. Grefenstette, and R. Raileanu Understanding the effects of rlhf on llm generalisation and diversity . arXiv preprint arXiv:2310.06452 . Cited by: §5 .

Kosmyna et al. (2025) N. Kosmyna, E. Hauptmann, Y. T. Yuan, J. Situ, X. Liao, A. V. Beresnitzky, I. Braunstein, and P. Maes Your brain on chatgpt: accumulation of cognitive debt when using an ai assistant for essay writing task . External Links: 2506.08872 , Link Cited by: §A.3 .

Krumdick et al. (2025) M. Krumdick, C. Lovering, V. Reddy, S. Ebner, and C. Tanner No free labels: limitations of llm-as-a-judge without human grounding . arXiv preprint arXiv:2503.05061 . Cited by: §5 .

Kumar et al. (2025) H. Kumar, J. Vincentius, E. Jordan, and A. Anderson Human creativity in the age of llms: randomized experiments on divergent and convergent thinking . In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems , CHI ’25 , pp. 1–18 . External Links: Link , Document Cited by: §A.3 .

Kwok et al. (2024) L. Kwok, M. Bravansky, and L. D. Griffin Evaluating cultural adaptability of a large language model via simulation of synthetic personas . External Links: 2408.06929 , Link Cited by: §5 .

Lake et al. (2024) T. Lake, E. Choi, and G. Durrett From distributional to overton pluralism: investigating large language model alignment . arXiv preprint arXiv:2406.17692 . Cited by: §5 .

Lambert et al. (2024) N. Lambert, V. Pyatkin, J. Morrison, L. Miranda, B. Y. Lin, K. Chandu, N. Dziri, S. Kumar, T. Zick, Y. Choi, N. A. Smith, and H. Hajishirzi RewardBench: evaluating reward models for language modeling . External Links: 2403.13787 , Link Cited by: §4.2 .

Lee et al. (2025) A. Lee, R. S. Kwon, P. Railton, and L. Wang CLASH: evaluating language models on judging high-stakes dilemmas from multiple perspectives . External Links: 2504.10823 , Link Cited by: §5 .

Li et al. (2024a) C. Li, M. Chen, J. Wang, S. Sitaram, and X. Xie CultureLLM: incorporating cultural differences into large language models . External Links: 2402.10946 , Link Cited by: §5 .

Li et al. (2024b) M. Li, W. Shi, A. Pagnoni, P. West, and A. Holtzman Predicting vs. acting: a trade-off between world modeling & agent modeling . arXiv preprint arXiv:2407.02446 . Cited by: §1 , §3 , §5 .

Li et al. (2025) T. Li, Y. Zhang, P. Yu, S. Saha, D. Khashabi, J. Weston, J. Lanchantin, and T. Wang Jointly reinforcing diversity and quality in language model generations . External Links: 2509.02534 , Link Cited by: §5 .

Lu et al. (2024) Y. Lu, D. Wang, T. Li, D. Jiang, S. Khudanpur, M. Jiang, and D. Khashabi Benchmarking language model creativity: a case study on code generation . arXiv preprint arXiv:2407.09007 . Cited by: §5 .

Moon et al. (2024) S. Moon, M. Abdulhai, M. Kang, J. Suh, W. Soedarmadji, E. K. Behar, and D. M. Chan Virtual personas for language models via an anthology of backstories . External Links: 2407.06576 , Link Cited by: §5 .

Murthy et al. (2024a) S. K. Murthy, T. Ullman, and J. Hu One fish, two fish, but not the whole sea: alignment reduces language models’ conceptual diversity . arXiv preprint arXiv:2411.04427 . Cited by: §5 .

Murthy et al. (2024b) S. K. Murthy, T. Ullman, and J. Hu One fish, two fish, but not the whole sea: alignment reduces language models’ conceptual diversity . External Links: 2411.04427 , Link Cited by: §5 .

Myung et al. (2025) J. Myung, N. Lee, Y. Zhou, J. Jin, R. A. Putri, D. Antypas, H. Borkakoty, E. Kim, C. Perez-Almendros, A. A. Ayele, V. Gutiérrez-Basulto, Y. Ibáñez-García, H. Lee, S. H. Muhammad, K. Park, A. S. Rzayev, N. White, S. M. Yimam, M. T. Pilehvar, N. Ousidhoum, J. Camacho-Collados, and A. Oh BLEnD: a benchmark for llms on everyday knowledge in diverse cultures and languages . External Links: 2406.09948 , Link Cited by: §5 .

Namuduri et al. (2025) R. Namuduri, Y. Wu, A. A. Zheng, M. Wadhwa, G. Durrett, and J. J. Li QUDsim: quantifying discourse similarities in llm-generated text . External Links: 2504.09373 , Link Cited by: §3 , §5 .

Nguyen et al. (2025) M. Nguyen, A. Baker, C. Neo, A. Roush, A. Kirsch, and R. Shwartz-Ziv Turning up the heat: min-p sampling for creative and coherent llm outputs . ICLR . Cited by: §3 , §3 , §5 , §5 .

Olson et al. (2021) J. A. Olson, J. Nahas, D. Chmoulevitch, S. J. Cropper, and M. E. Webb Naming unrelated words predicts creativity . Proceedings of the National Academy of Sciences 118 ( 25 ), pp. e2022340118 . Cited by: §5 .

Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback . Advances in neural information processing systems 35 , pp. 27730–27744 . Cited by: §5 .

Padmakumar and He (2024) V. Padmakumar and H. He Does writing with language models reduce content diversity? . External Links: 2309.05196 , Link Cited by: §5 .

Padmakumar et al. (2025) V. Padmakumar, C. Yueh-Han, J. Pan, V. Chen, and H. He Beyond memorization: mapping the originality-quality frontier of language models . External Links: 2504.09389 , Link Cited by: §1 , §5 .

Pang et al. (2024) J. Pang, J. Wei, A. P. Shah, Z. Zhu, Y. Wang, C. Qian, Y. Liu, Y. Bao, and W. Wei Improving data efficiency via curating llm-driven rating systems . arXiv preprint arXiv:2410.10877 . Cited by: §5 .

Rabeyah et al. (2024) A. A. Rabeyah, F. Góes, M. Volpe, and T. Medeiros Do llms agree on the creativity evaluation of alternative uses? . External Links: 2411.15560 , Link Cited by: §5 .

Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn Direct preference optimization: your language model is secretly a reward model . Advances in Neural Information Processing Systems 36 , pp. 53728–53741 . Cited by: §5 .

Rao et al. (2025) A. Rao, A. Yerukola, V. Shah, K. Reinecke, and M. Sap NormAd: a framework for measuring the cultural adaptability of large language models . External Links: 2404.12464 , Link Cited by: §5 .

Ruan et al. (2024) K. Ruan, X. Wang, J. Hong, P. Wang, Y. Liu, and H. Sun LiveIdeaBench: evaluating llms’ scientific creativity and idea generation with minimal context . arXiv preprint arXiv:2412.17596 . Cited by: §5 .

Ryan et al. (2024a) M. J. Ryan, W. Held, and D. Yang Unintended impacts of LLM alignment on global representation . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , L. Ku, A. Martins, and V. Srikumar (Eds.) , Bangkok, Thailand , pp. 16121–16140 . External Links: Link , Document Cited by: §5 .

Ryan et al. (2024b) M. J. Ryan, W. Held, and D. Yang Unintended impacts of llm alignment on global representation . arXiv preprint arXiv:2402.15018 . Cited by: §5 .

Schulman et al. (2017) J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov Proximal policy optimization algorithms . arXiv preprint arXiv:1707.06347 . Cited by: §5 .

Shi et al. (2025) T. Shi, Z. Wang, L. Yang, Y. Lin, Z. He, M. Wan, P. Zhou, S. Jauhar, S. Chen, S. Xia, H. Zhang, J. Zhao, X. Xu, X. Song, and J. Neville WildFeedback: aligning llms with in-situ user interactions and feedback . External Links: 2408.15549 , Link Cited by: §5 .

Shi et al. (2024) W. Shi, R. Li, Y. Zhang, C. Ziems, C. yu, R. Horesh, R. A. de Paula, and D. Yang CultureBank: an online community-driven knowledge base towards culturally aware language technologies . External Links: 2404.15238 , Link Cited by: §5 .

Shumailov et al. (2023) I. Shumailov, Z. Shumaylov, Y. Zhao, Y. Gal, N. Papernot, and R. Anderson The curse of recursion: training on generated data makes models forget . arXiv preprint arXiv:2305.17493 . Cited by: §5 .

Shumailov et al. (2024) I. Shumailov, Z. Shumaylov, Y. Zhao, N. Papernot, R. Anderson, and Y. Gal AI models collapse when trained on recursively generated data . Nature 631 ( 8022 ), pp. 755–759 . Cited by: §5 .

Shypula et al. (2025) A. Shypula, S. Li, B. Zhang, V. Padmakumar, K. Yin, and O. Bastani Evaluating the diversity and quality of llm generated content . External Links: 2504.12522 , Link Cited by: §5 .

Sorensen et al. (2024) T. Sorensen, J. Moore, J. Fisher, M. Gordon, N. Mireshghallah, C. M. Rytting, A. Ye, L. Jiang, X. Lu, N. Dziri, et al. A roadmap to pluralistic alignment . arXiv preprint arXiv:2402.05070 . Cited by: §5 .

Sourati et al. (2025) Z. Sourati, F. Karimi-Malekabadi, M. Ozcan, C. McDaniel, A. Ziabari, J. Trager, A. Tak, M. Chen, F. Morstatter, and M. Dehghani The shrinking landscape of linguistic diversity in the age of large language models . External Links: 2502.11266 , Link Cited by: §A.3 , §1 .

Srewa et al. (2025) M. Srewa, T. Zhao, and S. Elmalaki PluralLLM: pluralistic alignment in llms via federated learning . External Links: 2503.09925 , Link Cited by: §5 .

Verga et al. (2024) P. Verga, S. Hofstatter, S. Althammer, Y. Su, A. Piktus, A. Arkhangorodsky, M. Xu, N. White, and P. Lewis Replacing judges with juries: evaluating llm generations with a panel of diverse models . External Links: 2404.18796 , Link Cited by: §5 .

Vijayakumar et al. (2016) A. K. Vijayakumar, M. Cogswell, R. R. Selvaraju, Q. Sun, S. Lee, D. Crandall, and D. Batra Diverse beam search: decoding diverse solutions from neural sequence models . arXiv preprint arXiv:1610.02424 . Cited by: §5 .

Wang et al. (2024) L. Wang, X. Shi, G. Li, J. Li, Y. Dong, X. Zhang, W. Jiao, and H. Mei Why language models collapse when trained on recursively generated text . arXiv preprint arXiv:2412.14872 . Cited by: §5 .

Wang et al. (2025a) Z. Wang, J. Zhang, X. Zhang, K. Liu, P. Wang, and Y. Zhou Diversity-oriented data augmentation with large language models . arXiv preprint arXiv:2502.11671 . Cited by: §5 .

Wang et al. (2025b) Z. Wang, J. Zhang, X. Zhang, K. Liu, P. Wang, and Y. Zhou Diversity-oriented data augmentation with large language models . External Links: 2502.11671 , Link Cited by: §5 .

Wang et al. (2025c) Z. Wang, J. Zeng, O. Delalleau, D. Egert, E. Evans, H. Shin, F. Soares, Y. Dong, and O. Kuchaiev Dedicated feedback and edit models empower inference-time scaling for open-ended general-domain tasks . External Links: 2503.04378 , Link Cited by: §4.1 .

Wenger and Kenett (2025) E. Wenger and Y. Kenett We’re different, we’re the same: creative homogeneity across llms . External Links: 2501.19361 , Link Cited by: §5 .

West and Potts (2025) P. West and C. Potts Base models beat aligned models at randomness and creativity . External Links: 2505.00047 , Link Cited by: §1 , §3 , §5 .

Xu et al. (2024) S. Xu, Y. Leng, L. Yu, and D. Xiong Self-pluralising culture alignment for large language models . External Links: 2410.12971 , Link Cited by: §5 .

Yang et al. (2024) S. Yang, M. A. Ali, L. Yu, L. Hu, and D. Wang Model autophagy analysis to explicate self-consumption within human-AI interactions . In First Conference on Language Modeling , External Links: Link Cited by: §5 .

Zhang et al. (2025a) J. Zhang, S. Yu, D. Chong, A. Sicilia, M. R. Tomz, C. D. Manning, and W. Shi Verbalized sampling: how to mitigate mode collapse and unlock llm diversity . External Links: 2510.01171 , Link Cited by: §5 .

Zhang et al. (2025b) Y. Zhang, H. Diddee, S. Holm, H. Liu, X. Liu, V. Samuel, B. Wang, and D. Ippolito NoveltyBench: evaluating language models for humanlike diversity . External Links: 2504.05228 , Link Cited by: §1 , §5 , §5 .

Zhang et al. (2024) Y. Zhang, A. Schwarzschild, N. Carlini, Z. Kolter, and D. Ippolito Forcing diffuse distributions out of language models . arXiv preprint arXiv:2404.10859 . Cited by: §1 , §3 , §5 .

Zhao et al. (2024) W. Zhao, X. Ren, J. Hessel, C. Cardie, Y. Choi, and Y. Deng Wildchat: 1m chatgpt interaction logs in the wild . ICLR . Cited by: §B.1 , §1 , §2 .

Zhu et al. (2025) M. Zhu, Y. Weng, L. Yang, and Y. Zhang Personality alignment of large language models . External Links: 2408.11779 , Link Cited by: §5 .

Appendix A Discussions

附录 A 讨论

A.1 Limitations

A.1 局限

While comprehensive with 26K queries, Infinity-Chat represents only a snapshot of the vast space of possible open-ended queries and may not capture all forms of creative divergence across different contexts. Moreover, the focus on English-language prompts derived from WildChat potentially underrepresents linguistic, cultural, and regional diversity in user interactions with language models, limiting the generalizability of our findings to non-English contexts and diverse cultural perspectives. To construct our taxonomy of queries, we used GPT-4o to efficiently label user queries, and we found human annotators achieved 74.7% agreement with these automatic labels. While this accuracy could be improved, e.g., by labeling with stronger models or through model ensembling approaches, this performance is comparable to typical human annotation accuracy and we deem sufficient to represent the distribution of categories within our open-ended taxonomy. Finally, while our analysis successfully reveals clear patterns such as the “time is a river/weaver” dichotomy, it may oversimplify the multidimensional nature of creative expression, and relying on semantic similarity of text embeddings to quantify diversity may lack sufficient expressiveness to capture the full spectrum of creative variation in generated responses.

Infinity-Chat虽然包含26K的问询内容全面,但只是描绘了可能存在的开放式问询的广阔空间,可能无法捕捉不同背景之间一切形式的创造性分歧. 此外,关注源自WildChat的英语诱因可能低估了用户与语言模型互动中的语言、文化和区域多样性,将我们的结论局限于非英语背景和文化观点。 为了构建我们的查询分类学,我们使用GPT-4o来高效地给用户查询贴上标签,我们发现人类的注释员与这些自动标签达成了74.7%的协议. 虽然这种准确性可以改进,例如,通过贴上更强模型的标签或通过模型共聚法,但这种性能与典型的人类注释精确性相当,我们认为足以代表我们开放式分类学中类别分布情况。 最后,虽然我们的分析成功揭示了“时间是河流/织物”两极分化等明显模式,但它可能过度简化了创造性表达的多面性,依赖文字嵌入的语义相似性来量化多样性可能缺乏足够的表达力,无法在生成的响应中捕捉到各种创造性差异。

While extending Infinity-Chat to multilingual and multicultural settings is an important direction, we anticipate that similar homogenization issues will likely arise across languages and cultures, given the global overlap in pretraining data sources and alignment practices. We encourage future work to pursue such extensions, and we hope our benchmark provides a comprehensive foundation that multilingual research can build upon. Our taxonomy was designed to be language-agnostic and can support adaptation when appropriate resources and expertise are available. We’ll further strengthen this discussion in camera-ready.

虽然将Infinity-Chat扩展至多语种和多文化环境是一个重要方向,但我们预计,鉴于培训前数据来源和校正做法方面的全球重叠,各种语言和文化之间可能会出现类似的同质化问题。 我们鼓励今后开展这种扩展工作,我们希望我们的基准为多语种研究提供一个全面的基础。 我们的分类设计是语言不可知的,在有适当的资源和专门知识时可以支持适应。 我们将会进一步加强这场讨论,

Diversity and quality represent two key dimensions in evaluating responses to open-ended queries. In this work, we focused primarily on diversity, with fixed model decoding configurations, without studying quality. We selected a decoding configuration that generated empirically coherent text and investigated whether models can be guided to produce responses that maximize diversity while maintaining reasonable quality. Furthermore, we show that existing LMs, LM judges, and reward models struggle to reliably distinguish between equal-quality responses for open-ended queries. This motivates the need for further investigation into reliable automatic evaluation of equal-quality responses for open-ended queries, beyond human annotation. Finally, although our analysis identifies clear evidence of an “Artificial Hivemind” effect across models, it falls short of establishing the underlying mechanisms causing this homogenization. Future research can mechanistically disentangle causality, whether stemming from shared training data, memorization, generalization, or other factors.

多样性和质量是评价对不限成员名额询问的答复的两个关键方面。 在这部作品中,我们主要专注于多样性,固定的模型解码配置,而没有研究质量. 我们选择了一种编解码配置,生成了经验性一致的文本,并研究是否能够指导模型生成在保持合理质量的同时尽量扩大多样性的对策。 此外,我们显示,现有的LM、LM法官和奖励模式难以可靠地区分对无限制询问的同等质量的答复。 这促使有必要进一步调查除了人类注释之外,对无限制询问的同等质量答复进行可靠的自动评价。 最后,尽管我们的分析确定了各种模型的“人工蜂窝效应”的明确证据,但它未能建立造成这种同质化的基本机制。 未来的研究可以机械地分解因果关系,无论是来自共享的培训数据,记忆,概括还是其他因素.

Our work has significant implications for AI development and society, providing a valuable dataset for diagnosing and potentially mitigating LM homogenization and accelerating the development of more diverse, creative AI systems that align with diverse human needs and perspectives. However, this “Artificial Hivemind” effect also raises serious concerns about models’ long-term impact on human creativity. If users increasingly rely on such systems for creative tasks, exposure to homogenized outputs could subtly influence human thinking patterns and reduce overall cultural and intellectual diversity. Furthermore, it could exacerbate existing biases and limit representation of marginalized perspectives. When LMs converge on dominant cultural expressions—such as Western-centric metaphors like “time is a river"—they may inadvertently suppress alternative worldviews and traditions.

我们的工作对人工智能的发展和社会有重大影响,为诊断和可能减轻LM同质化提供了宝贵的数据集,并加快了符合人类不同需要和观点的更加多样化,创造性的人工智能系统的发展. 然而,这种 " 人工蜂巢 " 效应也使人们严重关切模型对人类创造力的长期影响。 如果用户越来越多地依赖这种系统来完成创造性的任务,接触同质化的产出可能潜伏地影响人类的思维模式,降低整体的文化和智力多样性。 此外,它可能加剧现有的偏见,限制边缘化观点的代表性。 当LMs聚集在主流文化表现形式上——如以西方为中心的比喻,如“时间是一条河流”,它们可能会无意中压制其他的世界观和传统。

The “Artificial Hivemind” effect also highlights fundamental questions about what values we want AI systems to embody. Should AI prioritize efficiency and consistency, or diversity and novelty? These choices reflect deeper societal values about creativity, culture, and human flourishing. The ethical implications of LM homogenization extend far beyond technical considerations, touching on fundamental questions about human agency, cultural preservation, and the kind of future we want to create with these technologies. Addressing these challenges requires not just technical innovation, but sustained ethical reflection and inclusive dialogue about the role of AI in human creative expression.

“人工蜂窝”效应也突出了我们希望人工智能系统体现哪些价值的基本问题。 大赦国际应把效率和一致性列为优先事项,还是多样性和新颖性? 这些选择反映了关于创造力、文化和人类繁荣的更深刻的社会价值。 LM同质化的道德影响远远超出了技术考虑,触及了人类机构,文化保护等根本问题,以及我们希望用这些技术创造的未来类型. 应对这些挑战不仅需要技术创新,还需要就AI在人类创造性表达中的作用进行持续的道德反思和包容性对话.

A.2 Future Directions

A.2 未来方向

Our findings open several promising directions for advancing the study and mitigation of behavioral convergence in large language models.

我们所面对的结论为在大语言模型中推进行为趋同的研究和缓解提供了几个有希望的方向。

Foundation and training analysis. We plan to extend the Artificial Hivemind testbed to foundation models without instruction-following capabilities to better disentangle the respective roles of pre-training and post-training in shaping convergent behaviors. Moreover, we aim to quantify the relative contributions of different post-training pipelines—such as supervised fine-tuning, RLHF/RLAIF, and constitutional training—to the emergence of homogenized responses.

** 基金会和培训分析。 ** 我们计划将人工蜂窝试验台扩大到基础模型,而不提供遵循教学的能力,以更好地将训练前和训练后各自在塑造趋同行为方面的作用分开。 此外,我们力求量化不同培训后管道的相对贡献,例如监督微调、RLHF/RLAIF和宪法培训,从而出现统一的反应。

Mitigation and alignment strategies. Future work will explore diversity-aware training objectives and alignment schemes that explicitly reward exploration of multiple valid modes while preserving response quality. We also intend to benchmark decoding strategies (e.g., diverse beam search, nucleus sampling variants) under the Artificial Hivemind metric to evaluate their effectiveness in counteracting homogenization.

** 缓解和调整战略。 ** 今后的工作将探索了解多样性的培训目标和调整计划,明确奖励探索多种有效模式,同时保持反应质量。 我们还打算在人工蜂窝测量标准下将解码策略(例如:多样的光束搜索,核取样变体)作为基准,以评估其对抗同质化的效果.

Practical integration. To make our framework actionable, we will integrate Artificial Hivemind into red-teaming workflows to stress-test model robustness and coverage under open-ended queries. Additionally, our dataset can serve as a training prompt resource for reinforcement learning methods that explicitly encourage diversity during alignment. Finally, Hivemind’s diagnostic signals can inform curriculum design—gradually exposing models to increasingly open-ended prompts that are most susceptible to mode collapse.

** 实际整合。 ** 为了使我们的框架能够实施,我们将将人工蜂窝式工作流程纳入红色团队工作流程,以测试模型的稳健性和在无限制的询问下的覆盖面。 此外,我们的数据集可作为加强学习方法的培训及时资源,在调整过程中明确鼓励多样性。 最后,蜂窝座的诊断信号可以为课程设计提供信息——逐步使模型暴露于越来越容易发生模式崩塌的无限制的提示。

Together, these directions position Artificial Hivemind as not only a descriptive diagnostic tool, but also a foundation for developing training, decoding, and evaluation practices that foster greater diversity and individuality in language models.

这些方向共同将人工蜂窝定位为不仅一种描述性的诊断工具,而且也是发展培训、编解码和评价做法的基础,这些实践可以促进语言模式的更大多样性和个性。

A.3 Broader Implications

A.3 更广含义

Societal Implications.

While some degree of convergence among large language models (LLMs) is statistically expected, its societal consequences warrant close scrutiny. As billions of users increasingly depend on LLMs for creative, educational, and decision-making purposes, understanding and quantifying behavioral homogenization becomes critical. Emerging evidence shows measurable shifts in human writing styles, creative ideation, and divergent thinking following the widespread adoption of systems like ChatGPT [79, 80, 81, 82, 83, 76]. These findings suggest that model-level convergence may propagate into human expression, amplifying uniformity in linguistic and cognitive patterns at scale. By introducing quantitative tools to measure and track such convergence, our work provides an empirical foundation for assessing the long-term cultural and epistemic consequences of LLM-mediated communication.

社会影响。

虽然在统计学上预计大语言模型之间会在一定程度上趋同,但其社会后果需要仔细审查。 由于数十亿用户越来越依赖LLMs进行创造性、教育和决策,理解和量化行为趋同变得至关重要。 新出现的证据表明,在广泛采用ChatGPT等系统之后,人类写作风格、创造性思想和不同思维发生了可衡量的变化[79、80、81、82、83、76]。 这些研究结果表明,模型层面的趋同可能传播到人类的表达中,从而扩大语言和认知模式在规模上的统一性。 通过采用量化工具衡量和跟踪这种趋同,我们的工作为评估LLM媒介传播的长期文化和认知后果提供了经验基础。

Implications for Data Distillation and Model Training.

Our results also have immediate implications for synthetic data generation and model training practices. While distilling knowledge from LLMs has proven transformative for efficiency and scalability, it is well known that relying on a single model as a teacher can intensify mode collapse, reinforcing narrow response patterns, diminishing output diversity, and, in extreme cases, leading to degenerative feedback loops [84, 85]. To counter this, recent approaches employ model swarms and multi-agent frameworks that aggregate outputs from multiple models in pursuit of diversity [86, 87].

数据稀释和示范培训的影响。

我们的成果也立即影响到合成数据的生成和示范培训做法。 虽然从LLMs中提炼知识已证明对效率和可扩展性具有变革性,但众所周知,依靠单一模式作为教师可以强化模式崩塌,强化狭义响应模式,降低产出多样性,并在极端情况下导致退化反馈循环[84,85]. 为了应对这种情况,最近的办法采用模式组合和多剂框架,将多种模式的产出汇总起来,追求多样性[86、87]。

However, our findings reveal a fundamental limitation: even across distinct state-of-the-art models, diversity in open-ended tasks is far from guaranteed. Models often converge toward highly similar answers, undermining the assumed benefits of multi-model distillation. This insight carries significant implications for research on synthetic data curation, ensemble-based alignment, and constitutional AI. Without rigorous diagnostic frameworks such as Artificial Hivemind, practitioners risk overestimating cross-model diversity and unintentionally perpetuating homogenization across generations of language models.

然而,我们的调查结果揭示了一种根本的局限性:即使在不同的最先进的模式中,无限期任务的多样性也远未得到保障。 模型往往趋向高度相似的答案,破坏了多模型蒸馏的假定好处. 这种见解对合成数据解析、基于综艺的调合和宪法AI的研究具有重大影响。 没有诸如人工蜂窝等严格的诊断框架,从业人员有可能高估跨模型的多样性,无意中使几代语言模型的同质化永久化。

Appendix B Infinity-Chat: A Dataset of In-The-Wild Open-Ended User Queries

附录 B Infinity-Chat：野外开放式用户查询数据集

In this section, we describe the process of mining open-ended and closed-ended queries used to construct Infinity-Chat.

在本节中,我们描述了用于构建Infinity-Chat的不限成员名额和封闭式查询的开采过程。

B.1 Details of the Mining of In-The-Wild Open-Ended User Queries

B.1 野外开放式用户查询挖掘细节

To understand the types of real-world open-ended queries posed to LMs across diverse use cases, we carefully curate and select such queries from WildChat, a dataset containing a rich variety of real-world conversations from in-the-wild users [3].

为了了解在各种使用案件中向LMs提出的现实世界开放式询问的类型,我们仔细地整理和选择了来自WildChat的此类询问。 WildChat是一个数据集,包含了来自闪烁的用户的多种真实世界对话[3]。

To narrow the scope of queries for closer analysis, we first extract user queries from allenai/WildChat-1M that meet the following criteria: (1) written in English; (2) labeled as non-toxic and non-harmful according to the built-in flags in the original WildChat dataset; (3) directed to GPT-4 models; and (4) of moderate length (i.e.,, between 15 and 200 characters). This filtering process yields a total of 37,426 query candidates.

为了缩小查询范围以进行更紧密的分析,我们首先从Alenai/WildChat-1M中提取用户查询,这些查询符合以下标准:(1)用英文写成;(2)按照原WildChat数据集内置的旗帜标注为无毒和无害;(3)针对GBT-4模型;(4)长度中等(即15到200个字符之间). 这一过滤过程共产生37,426个查询候选人。

We then classify the remaining queries using gpt-4o-2024-11-20 along three dimensions: (1) Meaningful Information: whether the query poses a meaningful question or seeks substantive information; (2) Greeting/Model Inquiry: whether the query is a greeting or an inquiry about the model itself (e.g.,, “Are you an AI?”); and (3) Response Type (Single or Multiple): whether the query is expected to yield a single specific answer or multiple valid, diverse responses. During this classification process, we also revise queries in real-time if they are unclear or ambiguous.

然后,我们将使用gpt-4o-2024-11-20的其余查询按三个维度进行分类:(1)** 计量信息**:查询是提出有意义的问题还是寻求实质性信息;(2)** 格莱廷/模式调查**:查询是问候还是对模型本身的询问(例如,“你是一个AI吗?”);(3)** 答复类型(Single 或Multipl)**:查询是期望得出一个单一的具体答案还是多个有效的、多样的答复。 在这一分类过程中,如果询问不明确或含糊不清,我们也会实时修订.

After applying automatic filtering, we obtain 26,070 open-ended queries and 8,817 closed-ended queries (e.g.,, yes/no questions, queries with precise answers, or well-scoped writing tasks). Examples of open-ended and closed-ended queries are shown in Table 3 and Table 4, respectively. We refer to this dataset as Infinity-Chat, which consists of open-ended queries designed to elicit diverse and high-quality responses. The prompts used to classify user queries as either open-ended or closed-ended are shown in Figures 12 and 13.

在应用自动过滤后,我们获得26,070个开放式查询和8,817个封闭式查询(如是/否问题,有精确答案的查询,或范围明确的写作任务). 不限成员名额和非公开查询的例子分别见表3和表4。 我们把这个数据集称为Infinity-Chat,它由开放式的查询组成,旨在获得多样和高质量的答复。 用于将用户查询归类为无期限或无期限的提示见图12和13。

B.2 Details of Defining the Open-Ended Query Taxonomy

B.2 开放式查询分类体系定义细节

Table 2 presents additional examples illustrating the open-endedness taxonomy (Figure 2 in the main paper) in greater detail. Figure 14 shows the prompt used to annotate both new and existing open-ended categories of user queries. The automatic taxonomy mining is performed using gpt-4o-2024-11-20.

表2提供了其他例子,更详细地说明了开放式分类法(主要文件图2)。 图14显示了用于对新的和现有的开放式用户查询类别进行注释的提示。 使用gpt-4o-2024-11-20进行自动分类开采.

B.3 Human Validation of the Open-endedness of Queries in Infinity-Chat

B.3 Infinity-Chat 查询开放度的人类验证

To strengthen our dataset’s validity, we conduct a human study to verify the open-endedness of queries in Infinity-Chat. We sample 100 evaluation examples from the dataset and recruit 86 participants on Prolific, assigning three participants to each query. All participants meet the following qualifications: English as a primary, first, or fluent language; an approval rate above 99%; 500–10,000 prior submissions; and an education level beyond high school. Each participant is instructed to answer two questions for each query:

为了增强我们数据集的有效性,我们进行了一项人类研究,以核实无穷无尽的查询是否无限。 我们从数据集中抽取100个评价实例,并在Prolific上征聘86名参与者,为每次查询指派3名参与者。 所有参与者都符合以下资格:英语为小学、第一语言或流利语言;批准率超过99%;以前提交500至10 000份;中学以后的教育水平。 指示每个参与者回答每个查询的两个问题:

Q1: Is the user query open-ended? In other words, can it reasonably allow for several different, similarly valid answers, rather than just a single, fairly deterministic one?

QQ1 **: ** 用户查询是否开放? 换句话说,它能否合理地允许几个不同的、同样有效的答案,而不只是一个决定性的单一答案? **

• • A: Yes — this query is open-ended and can have multiple possible answers
• • B: No — this query is NOT open-ended and has a single, fairly deterministic answer

• A:是——这个查询是开放式的,可以有多种可能的答案
• B:不——这一询问不是没有时限的,只有一个决定性的答案。

We used majority vote among three annotators per question to assess query open-endedness. By this criterion, 89% of queries were judged open-ended, providing strong evidence of high open-endedness (per Q1). With a more inclusive criterion, counting a query as open-ended if at least one annotator agreed, 100% of queries qualified as open-ended.

我们利用每个问题在三个注释人中的多数票来评估问题是否开放。 根据这一标准,89%的询问被判定为没有时限,这有力地证明了高度开放性(每问题1)。 采用更具包容性将其视为不限名额的标准,如果至少有一名注释员同意,则100%的查询都符合不限名额的条件。

Q2: If the query is open-ended, how many different, reasonable alternative answers do you think could apply?

####2**:** 如果查询是无限制的, 你认为可以应用多少不同,合理的替代答案? **

• • A: Fewer than 3
• • B: 3 to 10
• • C: 10 to 20
• • D: More than 20

• A:少于3个
• B:3至10
• C:10至20
• D:超过20个

Beyond the binary classification of open-endedness, we assessed the breadth of reasonable answers. As shown in Table 1, annotators judged 81.27% of queries to allow 3+ alternatives, and 34.66% to permit over 20, showing that real-world queries are rarely narrow or deterministic. This underscores the need for diversity-aware LLM evaluation, as simplistic benchmarks with limited answer ranges fail to capture the true complexity of user queries.

除了开放式的二进制分类之外,我们还评估了合理答案的广度。 如表1所示,注释者判断81.27%的查询允许3+替代品,34.66%允许20多个替代品,表明现实世界的查询很少是狭隘的或决定性的。 这突出表明,需要开展具有多样性意识的LLM评价,因为答案范围有限的简单化基准无法反映用户查询的真正复杂性。

Answer	Percentage (%)
Fewer than 3	18.73
3 to 10	31.87
10 to 20	14.74
More than 20	34.66

Table 1: Human ratings of the degree of open-endedness of queries in Infinity-Chat.

** 答复**	** 百分比(%)**
少于3人	18.73
3 to 10	31.87
10 to 20	14.74
超过20人	34.66

表1:无穷无尽的查询程度的人评分.

We further examined how annotators perceived the degree of open-endedness across prompts. Interestingly, their judgments varied considerably across individuals. Below are sample annotations from three annotators per prompt:

我们进一步审查了注释者如何看待各种提示的开放式程度。 有趣的是,他们的判决在个人之间差别很大。 以下为每批3名注释员的样本说明:

• • [More than 20, 10 to 20, 3 to 10]
• • [More than 20, 3 to 10, More than 20]
• • [More than 20, More than 20, More than 20]
• • [Fewer than 3, More than 20, More than 20]
• • [Fewer than 3, 10 to 20, More than 20]

• [多于20、10至20、3至10]
  • [多于20、3至10、20]
  • [20人以上,20人以上]
  • [3级以上,20级以上,20级以上]
  • [温度超过3、10至20,超过20]

This variability indicates that assigning a definitive level of open-endedness to each query is inherently challenging. Nevertheless, we conducted a coarse-grained grouping based on the following rule: (1) High open-endedness, if any annotator labeled a prompt as “More than 20 responses.” (2) Low open-endedness otherwise.

这种变化表明,给每个查询确定一个开放性水平本身就具有挑战性。 尽管如此,我们还是根据以下规则进行了粗糙的分组:(1) 高度开放性*,如果* 任何* 注释者将一个即时标记为“超过20个答复”。 (2) 低开放性* 否则。

Using this classification, we analyzed model response diversity based on sentence similarity across 42 models. The average similarity scores were: High open-endedness $=0.800$, Low open-endedness $=0.837$. These results suggest that, even for prompts perceived as more open-ended, models do not necessarily produce more diverse responses.

我们利用这个分类方法,根据42个模型的句子相似性,分析了模型响应的多样性. 平均相似性分数是:高开放性$=0.800$,低开放性$=0.837$. 这些结果表明,即使对被视为更开放的提示来说,模型不一定产生更多样化的反应。

Prompt for Filtering Open-Ended User Queries (Part 1)

You are an expert in analyzing user queries to classify their intent, assess their clarity, and improve unclear language while maintaining the original meaning.

Your task is to:

• Determine whether the user query is a meaningful question or seeks meaningful information. If it is not meaningful or gibberish, classify it as such with reasoning.

• Only if the query is meaningful and not a greeting or model inquiry, determine whether the user query is a greeting or is inquiring about the model itself. Classify such queries appropriately with reasoning. Skip this classification for non-meaningful queries.

• Only if the query is meaningful and not a greeting or model inquiry, assess whether the query might result in diverse, equally valid alternative responses or has a single optimal response. Skip this classification for non-meaningful or greeting queries.

• Revise the query minimally to correct any grammatical, spelling, or language errors while retaining the original meaning and intent.

• Return your classifications, reasoning, and any necessary revisions in JSON format.

Classify each query into one or more of the following categories:

• Meaningful Information: The query is a meaningful question or seeks meaningful information. - Greeting/Model Inquiry: The query is a greeting or asks about the model itself. - Response Type (Single or Multiple): Only classify the query if it is meaningful and not a greeting. Determine if the query expects one specific answer or could result in multiple valid, diverse responses.

For each query, provide:

• The original query.

• The revised query (if applicable).

• Brief reasoning and classifications for each task in the JSON format.

Example 1:

Query: Wht is sqr root of 64?

Output:

Figure 12: Prompt for Filtering Open-Ended User Queries (Part 1)

过滤打开的用户查询提示( 第一部分)

您是分析用户查询的专家, 以分类其意图, 评估其清晰度,

你的任务是:

• 确定用户查询是“有意义的问题”还是“寻找有意义的信息”。 如果它没有意义或胡言乱语,就用推理来分类.

• *只有当查询是有意义的,而不是问候或模式查询 * 时,确定用户查询是否是 * greeting * 或 * 询问模式本身 * 。 将此类查询与推理适当分类。 * 将本分类用于无意义的查询。

• 只有在询问有意义而不是问候或示范询问时,评估询问是否可能导致 * 互不相同、同样有效的替代答复* 或有 * 单一最佳答复* 。 * 为非有意或问候性问询而删除这一分类。

• 修改查询 * 最小* 以纠正任何语法、拼写或语言错误,同时保留原始含义和意图。

• 归还你的分类,推理, 和任何必要的JSON格式修改。

将每次查询分为以下一个或多个类别:

• 有意义的信息:查询是一个有意义的问题或寻求有意义的信息。 - 问候/模式调查:查询是问候或询问模式本身。 - 反应类型(*单 * 或 * 多倍 * ): 只有当查询有意义而不是问候时才进行分类. 确定查询是否期望一个具体答案,或可能导致多个有效,多样的答复.

每项查询请提供:

-原始查询

• 订正查询(如适用)。

实例1:

查询:Wht是 64 的 sqr 根 ?

输出 :

图12:过滤开放端用户查询的提示(第一部分)

Prompt for Filtering Open-Ended User Queries (Part 2)

Example 2: Query: Hi, how are you?

Output:

Example 3:

Query: Tell me ideas on making work commun better.

Output:

Example 4:

Query: asdjklqwe?

Output:

Now classify this query and return the result in JSON format:

Query: {USER_QUERY}

Output:

Figure 13: Prompt for Filtering Open-Ended User Queries (Part 2)

过滤开放式用户查询的提示（第 2 部分）

示例 2：
Query: Hi, how are you?

Output:

示例 3：
Query: Tell me ideas on making work commun better.

Output:

示例 4：
Query: asdjklqwe?

Output:

现在对下面这条查询分类，并以 JSON 格式返回结果：

Query: {USER_QUERY}

Output:

图 13：过滤开放式用户查询的提示（第 2 部分）

Prompt for Classifying the Category of an Open-Ended User Query

You are an expert in analyzing user queries to determine their open-ended categories.

You will be provided with an open-ended user query directed at a language model. Such queries often allow for multiple valid responses. Your task is to classify the query into one or more relevant categories based on the predefined taxonomy below. If the query does not fit any existing category, you should create a new category as needed.

For each assigned category (whether predefined or newly created), provide a brief justification explaining why the query falls under that category.

Predefined Categories:

• Creative Content Generation

• Ideation and Brainstorming

• Philosophical Questions

• Abstract Conceptual Questions

• Ambiguous Everyday Questions

• Analytical and Interpretive Questions

• Speculative and Hypothetical Scenarios

• Value-Laden Questions with Alternative Perspectives

• Opinion-Based Questions with Alternative Perspectives

• Controversial Questions with Alternative Perspectives

• Alternative Communication Styles

• Alternative Writing Genres

• Information-Seeking about Problem Solving

• Information-Seeking about Decision Support

• Information-Seeking about Skill Development

• Information-Seeking about Recommendations

• Information-Seeking about Concept Explanations

• Information-Seeking about Personal Advice

Response Format:

Your response should follow the structured format below. Ensure that all relevant categories are included.

Please classify the following query accordingly:

Query: This is about the Monster Hunter series. Write a TV advertisement for a Rathian, making subtle implications that she can be a companion and a wife.

Output:

Figure 14: Prompt for Classifying the Category of an Open-Ended User Query

将开端用户查询类别分类的提示

您是分析用户查询以确定其开放式类别的专家。

将为您提供针对语言模型的开放式用户查询。 这类查询往往允许多个有效答复。 您的任务是根据下面 我们要预先定义的分类法将查询分类为一个或多个相关类别 。 如果查询不符合任何现有类别,您应当根据需要创建一个新类别。

对于每个指定类别(无论是预先界定的还是新创建的),提供简单的理由,解释为什么查询属于这一类别。

预定义类别 :

• 创造性内容生成

• 思想和头脑风暴

• 哲学问题

• 抽象概念问题

• 含糊不清的日常问题

• 分析和解释问题

• 投机和假象

• 有其他观点的价值观问题

• 有其他观点的基于意见的问题

• 有其他观点的有争议的问题

• 替代性交流方式

• 其他的三个写法

• 寻找解决问题的信息

• 寻找关于决策支助的信息

所提及的技能发展

• 关于建议的信息查询

• 了解概念解释

• 关于个人咨询的信息查询

响应格式 :

您的回应应该遵循下面的结构格式 。 确保列入所有相关类别。

请据此对下列查询进行分类:

腾讯:这是关于魔兽猎人系列. 为Rathian人写一个电视广告, 产生微妙的影响,她可以成为一个伴侣和妻子。

输出 :

图14:对不开放的用户查询类别进行分类的提示

Category	Sub Cat.	#	Representative Examples
Creative Content Generation	Creative	Content	Generation	-	58.0	Compose a short poem about the feeling of watching a sunset. Write an 800-word essay on why 6 is afraid of 7. Generate a joke about electric vehicles.	Compose a short poem about the feeling of watching a sunset.	Write an 800-word essay on why 6 is afraid of 7.	Generate a joke about electric vehicles.
Creative
Content
Generation
Compose a short poem about the feeling of watching a sunset.
Write an 800-word essay on why 6 is afraid of 7.
Generate a joke about electric vehicles.
Brainstorming & Ideation	Brainstorming	& Ideation	-	15.2	I am a graduate student in Marxist theory, and I want to write a thesis on Gorz. Can you help me think of some new ideas? Suggest a feature for a smartwatch designed specifically for senior citizens.	I am a graduate student in Marxist theory, and I want to write a	thesis on Gorz. Can you help me think of some new ideas?	Suggest a feature for a smartwatch designed specifically for senior citizens.
Brainstorming
& Ideation
I am a graduate student in Marxist theory, and I want to write a
thesis on Gorz. Can you help me think of some new ideas?
Suggest a feature for a smartwatch designed specifically for senior citizens.
Open -Endedness	Open	-Endedness	Philosophical Questions	Philosophical	Questions	3.5	How do I understand what I want? Is it impossible to know everything?	How do I understand what I want?	Is it impossible to know everything?
Open
-Endedness
Philosophical
Questions
How do I understand what I want?
Is it impossible to know everything?
	Abstract Conceptual Questions	Abstract	Conceptual	Questions	10.0	Is honey magical? Is there life on other planets? In humans, is there a correlation between intelligence and compassion?	Is honey magical?	Is there life on other planets?	In humans, is there a correlation between intelligence and compassion?
Abstract
Conceptual
Questions
Is honey magical?
Is there life on other planets?
In humans, is there a correlation between intelligence and compassion?
	Ambiguous Everyday Questions	Ambiguous	Everyday	Questions	2.6	I’m 60 years old and haven’t achieved anything. What have I missed out on? What are signs that show you were not loved enough as a child by your mom? I have an apartment that costs about $100 a month. How do I survive?	I’m 60 years old and haven’t achieved anything. What have I missed out on?	What are signs that show you were not loved enough as a child by your mom?	I have an apartment that costs about $100 a month. How do I survive?
Ambiguous
Everyday
Questions
I’m 60 years old and haven’t achieved anything. What have I missed out on?
What are signs that show you were not loved enough as a child by your mom?
I have an apartment that costs about $100 a month. How do I survive?
	Analytical & Interpretive Questions	Analytical	& Interpretive	Questions	22.6	What’s the consequence of algorithm-driven content for public discourse? How do global economic inequalities shape international relations? What is the future of work in an increasingly automated economy?	What’s the consequence of algorithm-driven content for public discourse?	How do global economic inequalities shape international relations?	What is the future of work in an increasingly automated economy?
Analytical
& Interpretive
Questions
What’s the consequence of algorithm-driven content for public discourse?
How do global economic inequalities shape international relations?
What is the future of work in an increasingly automated economy?
	Speculative & Hypothetical Scenarios	Speculative	& Hypothetical	Scenarios	22.2	How might society evolve if telepathy became possible? Create a short review of a movie that doesn’t exist yet. How will AI reshape the way humans interact with one another in 50 years?	How might society evolve if telepathy became possible?	Create a short review of a movie that doesn’t exist yet.	How will AI reshape the way humans interact with one another in 50 years?
Speculative
& Hypothetical
Scenarios
How might society evolve if telepathy became possible?
Create a short review of a movie that doesn’t exist yet.
How will AI reshape the way humans interact with one another in 50 years?
Alternative Perspectives	Alternative	Perspectives	Value-Laden Questions	Value-Laden	Questions	2.3	Why is equal pay for equal work not worth imposing on an economy? Give me strong arguments as to why some people allow the death penalty.	Why is equal pay for equal work not worth imposing on an economy?	Give me strong arguments as to why some people allow the death penalty.
Alternative
Perspectives
Value-Laden
Questions
Why is equal pay for equal work not worth imposing on an economy?
Give me strong arguments as to why some people allow the death penalty.
	Controversial Questions	Controversial	Questions	2.5	Different views on discipleship Is there such a thing as a paradox of free speech?	Different views on discipleship	Is there such a thing as a paradox of free speech?
Controversial
Questions
Different views on discipleship
Is there such a thing as a paradox of free speech?
Alternative Styles	Alternative	Styles	Communication Styles	Communication	Styles	3.2	Write an email to organize a catch-up with a referral partner. How to say a formal thank you for your time in the subject line of an email?	Write an email to organize a catch-up with a referral partner.	How to say a formal thank you for your time in the subject line of an email?
Alternative
Styles
Communication
Styles
Write an email to organize a catch-up with a referral partner.
How to say a formal thank you for your time in the subject line of an email?
	Writing Genres	Writing	Genres	38.5	Can you write five tweets in the style of Dril about El Salvador? Write a play script in the style of Dilbert as if a man in an apocalyptic world.	Can you write five tweets in the style of Dril about El Salvador?	Write a play script in the style of Dilbert as if a man in an apocalyptic world.
Writing
Genres
Can you write five tweets in the style of Dril about El Salvador?
Write a play script in the style of Dilbert as if a man in an apocalyptic world.
Information -Seeking	Information	-Seeking	Problem Solving	Problem	Solving	19.3	Give me a strategy to double the money in a month, starting with 1000 euros. How do I code an online forum using LAMP?	Give me a strategy to double the money in a month, starting with 1000 euros.	How do I code an online forum using LAMP?
Information
-Seeking
Problem
Solving
Give me a strategy to double the money in a month, starting with 1000 euros.
How do I code an online forum using LAMP?
	Decision Support	Decision	Support	2.2	Choose the right decision: buy a new Zara sneaker or a used Adidas sneaker? What is the best investment if I invest 60,000 euros?	Choose the right decision: buy a new Zara sneaker or a used Adidas sneaker?	What is the best investment if I invest 60,000 euros?
Decision
Support
Choose the right decision: buy a new Zara sneaker or a used Adidas sneaker?
What is the best investment if I invest 60,000 euros?
	Skill Development	Skill	Development	23.5	How can I make a zombie survivor game on Scratch? Help me use my Microsoft Surface touchpad and on-screen keyboard to run Dolphin Emulator for the Wii.	How can I make a zombie survivor game on Scratch?	Help me use my Microsoft Surface touchpad and on-screen keyboard to run	Dolphin Emulator for the Wii.
Skill
Development
How can I make a zombie survivor game on Scratch?
Help me use my Microsoft Surface touchpad and on-screen keyboard to run
Dolphin Emulator for the Wii.
	Recommen -dations	Recommen	-dations	11.0	What is a good secondhand market laptop for learning Python? Give me a 3-day plan for Osaka with a flight leaving on the 3rd day at 7pm. What is the best and most profitable day trading strategy of all time?	What is a good secondhand market laptop for learning Python?	Give me a 3-day plan for Osaka with a flight leaving on the 3rd day at 7pm.	What is the best and most profitable day trading strategy of all time?
Recommen
-dations
What is a good secondhand market laptop for learning Python?
Give me a 3-day plan for Osaka with a flight leaving on the 3rd day at 7pm.
What is the best and most profitable day trading strategy of all time?
	Concept Explanations	Concept	Explanations	23.6	What are the benefits of hardwood flooring over other types? Do you know the demon Morloch?	What are the benefits of hardwood flooring over other types?	Do you know the demon Morloch?
Concept
Explanations
What are the benefits of hardwood flooring over other types?
Do you know the demon Morloch?
	Personal Advice	Personal	Advice	4.1	What should I learn in software development to be relevant in the future? My girlfriend is buying a house. I plan to marry her in a year or so. I don’t know if it’s fair for me to pay for the house or not. What’s the best solution?	What should I learn in software development to be relevant in the future?	My girlfriend is buying a house. I plan to marry her in a year or so. I don’t	know if it’s fair for me to pay for the house or not. What’s the best solution?
Personal
Advice
What should I learn in software development to be relevant in the future?
My girlfriend is buying a house. I plan to marry her in a year or so. I don’t
know if it’s fair for me to pay for the house or not. What’s the best solution?

Creative
Content
Generation

Compose a short poem about the feeling of watching a sunset.
Write an 800-word essay on why 6 is afraid of 7.
Generate a joke about electric vehicles.

Brainstorming
& Ideation

I am a graduate student in Marxist theory, and I want to write a
thesis on Gorz. Can you help me think of some new ideas?
Suggest a feature for a smartwatch designed specifically for senior citizens.

Open
-Endedness

Philosophical
Questions

How do I understand what I want?
Is it impossible to know everything?

Abstract
Conceptual
Questions

Is honey magical?
Is there life on other planets?
In humans, is there a correlation between intelligence and compassion?

Ambiguous
Everyday
Questions

I’m 60 years old and haven’t achieved anything. What have I missed out on?
What are signs that show you were not loved enough as a child by your mom?
I have an apartment that costs about $100 a month. How do I survive?

Analytical
& Interpretive
Questions

What’s the consequence of algorithm-driven content for public discourse?
How do global economic inequalities shape international relations?
What is the future of work in an increasingly automated economy?

Speculative
& Hypothetical
Scenarios

How might society evolve if telepathy became possible?
Create a short review of a movie that doesn’t exist yet.
How will AI reshape the way humans interact with one another in 50 years?

Alternative
Perspectives

Value-Laden
Questions

Why is equal pay for equal work not worth imposing on an economy?
Give me strong arguments as to why some people allow the death penalty.

Controversial
Questions

Different views on discipleship
Is there such a thing as a paradox of free speech?

Alternative
Styles

Communication
Styles

Write an email to organize a catch-up with a referral partner.
How to say a formal thank you for your time in the subject line of an email?

Writing
Genres

Can you write five tweets in the style of Dril about El Salvador?
Write a play script in the style of Dilbert as if a man in an apocalyptic world.

Information
-Seeking

Problem
Solving

Give me a strategy to double the money in a month, starting with 1000 euros.
How do I code an online forum using LAMP?

Decision
Support

Choose the right decision: buy a new Zara sneaker or a used Adidas sneaker?
What is the best investment if I invest 60,000 euros?

Skill
Development

How can I make a zombie survivor game on Scratch?
Help me use my Microsoft Surface touchpad and on-screen keyboard to run
Dolphin Emulator for the Wii.

Recommen
-dations

What is a good secondhand market laptop for learning Python?
Give me a 3-day plan for Osaka with a flight leaving on the 3rd day at 7pm.
What is the best and most profitable day trading strategy of all time?

Concept
Explanations

What are the benefits of hardwood flooring over other types?
Do you know the demon Morloch?

Personal
Advice

What should I learn in software development to be relevant in the future?
My girlfriend is buying a house. I plan to marry her in a year or so. I don’t
know if it’s fair for me to pay for the house or not. What’s the best solution?

Table 2: Taxonomy of open-ended queries that can benefit from model generations with diversity.

** 类别**	** Sub Cat**.	#	** 代表实例**
创意内容做到生成	创意	内容	生成	-	58.0	写一首关于看日落的感觉的短诗. 写一篇800字的散文,说明6为何害怕7. 制造电动车辆的笑话.	写一首关于看日落的感觉的短诗.	写一篇800字的散文,说明6为何害怕7.	制造电动胥车的笑话.
创意
内容
生成
写一首 maximum. 短诗讲述看日落的感觉.
写一篇800字的散文,讲述6为何害怕7 10.
制造电动车辆的笑话。
集思广益	集思广益	构想( I)	-	15.2	我是马克思 哪门子的研究生 我想写一篇关于戈尔兹的论文 你能帮我想想新想法吗? 建议专门为老年公民设计的智能手表的功能.	我是马克思主义理论的研究生 我想写一本	关于戈尔兹的论文. 你能帮我想想新想法吗?	建议专门为老年公民设计的智能手表的功能.
集思广益
构想( I)
我是马克思主义理论的研究生 我想写一本
关于戈尔兹的论文. 你能帮我想想新想法吗?
建议专门为老年公民设计的智能手表的功能.
打开 - 结束	打开	- 结束	哲学问题	哲学	问 题	3.5	我该怎么理解我想要什么? 无法知道一切吗?	我该怎么理解我想要什么?	无法知道一切吗?
打开
- 结束
哲学
问 协商一致意见
我该怎么理解我想要什么?
无法知道一切吗?
	概念问题摘要	Abstract	概念	问 题	10.0	蜂蜜神奇吗? 其他星球有生命吗? 在人类中,智能和同情之间是否有关联?	蜂蜜神奇吗?	其他星球上ان生命吗?	Mustafa s s s s s s
Abstract
概念
问 题
蜂蜜神奇吗?
其他星球有生命吗?
在人类中,智能和同情之间是否有关联?
	每天的问题都含混不清	含蓄	每天都会	问 题	2.6	我已60岁, 我错过了什么? 有什么迹象显示 你小时候被你妈爱得不够? 我有一间每月约100美元的公寓 我怎么活下来?	我已60岁, 我错过了什么?	有什么迹象显示 你小时候被你妈妈爱得不够?	我有一间每月约100美元的公寓 我怎么活下来?
含蓄
每天都会
问 题
我已60岁, 我错过了什么?
有什么迹象显示 你小时候被你妈妈爱得不够?
我有一间每月约100美元的公寓 我怎么活下来?
	分析和解释问题	分析	解释( I)	问 题	22.6	由算法驱动的内容对公共言论有何影响? 全球经济不平等如何影响国际关系? 在日益自动化的经济中,工作的未来是什么?	由算法驱动的内容对公共言论有何影响?	全球经济不平等如何影响国际关系?	在日益自动化的经济中,工作的未来是什么?
分析
解释( I)
问 题
由算法驱动的内容对公共言论有何影响?
全球经济不平等如何影响国际关系?
在日益自动化的经济中,工作的未来是什么?
	投机和假象	投机	假设( H)	设想	22.2	如果心灵感应成为可能,社会将如何演变? 建立一部尚未存在的电影的短评。 AI如何重塑人类在50年中互动的方式?.	如果心灵感应成为可能,社会将如何演变?	建立一部尚未存在的电影的短评。	AI如何重塑人类在50年中互动的方式?.
投机
假设( H)
设想
如果心灵感应成为可能,社会将如何演变?
建立一部尚未存在的电影的短评。
AI如何重塑人类在50年中互动的方式?.
备选视角	备选案文	视角	价值问题	值 - 上限	问 题	2.3	为什么同工同酬不值得强加给经济? 给我一些有力的论据,说明为什么有些人允许死刑.	为什么同工同酬不值得强加给经济?	给我有力的论点,说明为什么有些人允许死刑.
备选案文
视角
值 - 上限
问 题
为什么同工同酬不值得强加给经济?
给我有力的论点,说明为什么有些人允许死刑.
	有争议的问题	有争议的	问 题	2.5	对门徒的不同看法 是否有言论自由的悖论?	对门徒的不同看法	是否有言论自由的悖论?
有争议的
问 题
对门徒的不同看法
是否有言论自由的悖论?
替代样式	备选案文	样式	通讯样式	通讯	样式	3.2	写邮件与转介伙伴组织补习. 24. 如何正式感谢您在电子邮件的主题栏上的发言?	龛写邮件与转介伙伴组织补习.	如何正式感谢您 在电子邮件的主题栏?
备选案文
样式
通讯
样式
劳资协议,
如何正式感谢您 在电子邮件的主题栏?
	写入流派	写入	流派	38.5	你能写五个关于萨尔瓦多的推文吗? 以迪尔伯特的风格写一个剧本,仿佛一个处于末日世界的人.	你能写五个关于萨尔瓦多的推文吗?	以迪尔伯特的风格写一个剧本,仿佛一个处于末日世界的人.
写入
流派
你能写五个关于萨尔瓦多的推文吗?
以迪尔伯特的风格写一个剧本,仿佛一个处于末日世界的人.
信息 - 正在寻找	资料	- 寻找	问题解决	问题	解决	19.3	给我一个策略,在一个月内把钱增加一倍,从1000欧元开始. 我如何用LAMP编码一个在线论坛?	给我一个策略,在一个月内把钱增加一倍,从1000欧元开始.	我如何用LAMP编码一个在线论坛?
资料
- 寻找
问题
解决
给我一个策略,在一个月内把钱增加一倍,从1000欧元开始.
我如何用LAMP编码一个在线论坛?
	决定支持	决定	支助	2.2	选择正确的决定 : 购买新的 Zara 运动鞋 或使用的 Adidas 运动鞋 ? 如果我投资6万欧元,那最好的投资是什么?	选择正确的决定 : 购买新的 Zara 运动鞋 或使用的 Adidas 运动鞋 ?	如果我投资6万欧元,那最好的投资是什么?
决定
支助
选择正确的决定 : 购买新的 Zara 运动鞋 或使用的 Adidas 运动鞋 ?
如果我投资6万欧元,那最好的投资是什么?
	技能发展	技能	联合国	23.5	我怎么能在Scratch上做一个僵尸幸存者游戏? 帮助我使用微软 Surface 触控ủ 和屏幕键盘运行海豚 模拟器的Wii.	我 đã在Scratch是通过僵尸生存者游戏	帮助我使用微软 Surface 触摸板和屏幕键盘运行	海豚模拟器为Wii.
技能
联合国
我怎么能在Scratch上做一个僵尸幸存者游戏?
帮助我使用微软表面触摸板和屏幕键盘运行
海豚模拟器为Wii.
	重新建议 - 配对	复职人员	- 标 题	11.0	什么是学习Python的二手电脑? 给大阪3天计划 3日晚上7点开班机. 什么是有史以来最好的和最有利可图的日交易策略?	将作为学习Python的二手电脑?	给我一个大阪的3天计划 3日晚上7点出发	龛中日交易策略最好,最有利可图的是什么呢?
图例
- 标 题
什么是学习Python的二手电脑?
给我一个大阪的3天计划 3日晚上7点出发
什么是有史以来最好的和最有利可图的日交易策略?
	概念解释	概念	解释	23.6	硬木地板对其他类型有什么好处? 你认识那个怪物吗?	硬木地板比其他类型有什么好处?	你认识恶魔莫洛奇吗?
概念
解释
硬木地板比其他类型有什么好处?
你认识恶魔莫洛奇吗?
	个人咨询	龛个人	咨询	4.1	在软件开发方面,我应该学习什么才能与未来相关? 我女朋友在买房子 我打算过一年就娶她 我不知道是否公平, 最好的解决办法是什么?	在软件开发方面,我应该学习什么才能与未来相关?	我女朋友在买房子 我打算过一年就娶她 我没有	我是否愿意为这房子 听着, 最好的解决办法是什么?
86. 个人
咨询
在软件开发方面,我应该学习什么才能与未来相关?
我女朋友在买房子 我打算过一年就娶她 我没有
我是否愿意支付房子费用, 最好的解决办法是什么?

创意
内容
生成

写一首关于看夕阳均属感的短诗.
写一篇800字的散文,说明6为何害怕7.
制造电动车辆的笑话.

集思广益
构想( I)

我是马克思主义理论的研究生 我想写一本
关于戈尔兹的论文. 你能帮我想想新想法吗?
建议专门为老年公民设计的智能手表的功能.

打开
- 结束

哲学
问 题

我该怎么理解我想要什么?
难道不可能知道一切对此表示怀疑吗?

Abstract
概念
问题

蜂蜜神奇吗?
其他星球有生命吗?
在人类中,智能和同情之间是否有关联?

含蓄
为支持一切
问 题

我已60岁, 我错过了什么?
有什么迹象显示 你小时候被你妈妈爱得不够?
我有一间每月约100美元的公寓 我怎么活下来?

分析
解释( I)
问 题

由算法驱动的内容对公共言论有何影响?
全球经济不平等如何影响国际关系?
在日益自动化的经济中,工作的未来是什么?

投机
假设( H)
设想

如果更加有效地实现心灵感应,社会将如何演变?
建立一部尚未存在的电影的短评。
AI如何重塑人类在50年中互动的方式?.

备选案文
视角

值 - 上限
问 题

为什么同工同酬不值得强加给经济?
给我有力的论点,说明为什么有些人允许死刑.

有争议的
问 题

对门徒的不同看法
是否有言论自由的悖论?

备选案文
你干嘛这样

通讯
样式

写邮件与转介伙伴组织补习.
如何正式感谢您 在电子邮件的主题栏?

写入
流派

你能噬钻五条关于萨尔瓦多的推文吗?
所作的剧本以迪尔伯特的风格写成,仿佛一个处于末日世界的人.

资料
- 寻找

问题
解决

给我一个策略,在一个月内把钱增加一倍,从1000欧元开始.
我如何用LAMP编码一个在线论坛?

决定
支助

选择正确的决定 : 购买新的 Zara 运动鞋 或使用的 Adidas 运动鞋 ?
这项活动最好投资60,000欧元?

技能
联合国

我怎么能在Scratch上做一个僵尸幸存者游戏?
帮助我使用微软 Surface 触摸板和屏幕键盘运行
海豚模拟器为Wii.

复职人员
- 标 题

什么是学习Python的二手电脑?
给我一个大阪的3天计划 3日晚上7点出发
性别平等问题,

概念
解释

硬木地板比其他类型有什么好处?
你认识恶魔莫洛奇吗?

个人
咨询

在软件开发方面,我应该学习什么才能与未来相关?
我女朋友在买房子 我打算过一年就娶她 我没有
我是否愿意支付房子费用, 最好的解决办法是什么?

表2:可受益于具有多样性的模范一代的开放式查询的赘述分类。

	Representative Open-Ended Queries from WildChat
	Write me 3 short tips for self-development.
	Create a title with the prefix ’best’, as a one-liner, using only strings, less than 100 characters.
	Paraphrase this: We’re checking if the domain ’aspris.ae’ is included in our scope.
	Name a hot English word below 10 letters.
	Create a sentence using a minimum of 2 R-colored vowels.
	Give an example of a linear graph in graph theory.
	Give me a trivia question about blue birds and its corresponding answer.
	Write me a 1-paragraph essay about the development of the economy during the Han Dynasty.
	In three sentences, describe a girl wandering around in Vietnam.
	Can you give an example of a life goal related to self-image?
	Rave about the significance of rivers in a paragraph.
	Write a sentence where the last word is ’apple’.
	Write a May the 4th joke.
	Write an essay on the importance of the Roman Empire and its impact on future generations. Max: 100 words.
	Describe Apple Corporation in three sentences to a person who has no idea what cell phones are.
	Name an economic value of an additional year of schooling.
	Name one meaning of life.
	Write an one-paragraph kid’s story with a prince, a princess, and a dragon. When all hope is lost, the prince orders a magic sword from Amazon and slays the dragon. The other parts of the story are up to you.
	Give me a tip to be more organized at work. I’m a high school teacher.
	Explain computational irreducibility like I’m 5.
	Make an analogy of the relationship between US and China.
	Provide a few sentences on Sisu Cinema Robotics.
	Give a numerical example to illustrate the concept of partial derivative.
	Help me draft a paragraph as an expert consultant explaining TOEFL vs IELTS for international students.
	Come up a short blurb to introduce a religion called The Next Exodus Society.
	Write a headline for a company called "USBC CONSTITUTION" that encourages companies to donate their waste for recycling in exchange for money for the donated waste.
	Write a Google ad with 2 sentences and a 30-character limit per sentence for mobile car detailing.
	Give me a tip for managing a team of coworkers.
	What is the difference between analysis and design? Can one begin to design without analysis? Why? Be concise.
	Output a hard question to humanity (super concise and short), independent of theme.
	Provide an example of the name of an optimization technique used in machine learning.
	Briefly explain the potential uses of biofuels in 2-3 sentences.
	Write a metaphor involving time.
	If there were double the amount of oxygen in the air, what would happen? Write in 100 words.
	Generate a paragraph on why introspection is very important for growth, and provide guidance on listening to yourself more than heeding other people’s opinions.
	Generate a one-liner title for ’Elephant’ and ’sticker’.
	Write a 30-word essay on global warming.
	Generate a motto for a social media page focused on success, wealth, and self-help.
	Can you give me an incredible STEM fair idea that is affordable and relate to issues in Vietnam?
	Write a tweet about: This is a video from this morning’s crazy sunrise at the beach.
	Write a funny two-sentence birthday card message for a teammate who is 50 years old and loves going to a Toby Carvery restaurant.
	Generate a description for ’Sticking to Cuteness: The Panda Way.’
	Give me a short phrase to put on my portfolio webpage about being an amateur data analyst, data scientist, and Next.js web page developer.
	Create a short 1-paragraph story about a boy running on a beach. He is Asian, 12 years old, and the time of day is 4 in the afternoon.
	Can you give me a powerful rhetorical question for an essay about the harms of social media on teens?
	Give me the names of 3 instrumental songs that best match the mood of a rainy night.
	Give me the name of a instrumental song that matches the mood of a rainy night.
	Describe Deadpool in a short paragraph. Make sure it’s accessible to children.
	Write a personal tweet about how I am walking right now at sunrise to the lighthouse; it’s spring but cold like winter.
	Write a fear-of-missing-out title including "You’ve Never Seen Anything Like This!"
	Write a short FOMO title for a video of a shell on the beach
	One other way to say: "Fingers crossed that everything goes well."
	Write 3 to 4 lines about India.
	Write 100-300 words on how stress affects the body and mind.

Table 3: Example open-ended queries where diversity is important, collected from WildChat.

	** 来自WildChat的不限名额查询**
	给我写3个关于自我发展的短提示
	创建前缀 ' best ' 的标题, 作为单行, 只使用字符串, 不到 100 个字符 。
	解释如下:我们正在检查是否将“aspris.
	在10个字母下面命名一个热的英文单词.
	使用至少2个R色元音创建句子.
	滔滔在图论中举一个线性图的例子.
	给我一个关于蓝鸟的三维问题 及其相应的答案
	写一篇关于汉朝时期经济发展的1段散文.
	在三句中,描述一个女孩在越南四处游荡.
	你能举个例子来说明与自我形象有关的人生目标吗?
	在一个段落中强调河流的重要性。
	写一句“苹果”这个词。
	写个5月4日的笑话
	写一篇关于罗马帝国向各代人的重要性及其影响的文章. 麦斯:100个字.
	在三句中描述苹果公司,给一个不知道手机是什么的人.
	指定额外学年的经济价值。
	说一个生命的意义
	用王子、公主和龙来写一个段落的故事。 当所有的希望都丧失时,王子命令亚马逊的魔剑并杀死龙. 故事的其他部分由你决定.
	给我个小费 工作更有条理 我是一个高中老师。
	如我 17. 解释计算不可减少性
	美中关系类推.
	提供几句西苏电影机器人。
	给出一个数字例子,以说明部分中对衍生物的概念.
	帮助我起草一个段落,作为专家顾问为国际学生解释TOEFL vs IELTS.
	来介绍一个名为"下一个排外学会"的宗教
	为一家名为"USBC Constitution"的公司写一条头条,鼓励公司捐赠垃圾进行回收,以换取捐赠垃圾的资金.
	写一个Google广告,每句有2句和30个特征限制,用于详细介绍移动汽车.
	给我一个管理团队的提示
	分析与设计有什么区别? 我们能开始设计而不进行分析吗? 为什么? 简言之.
	向人类提出一个困难的问题(非常简洁和简短),而不包括主题。
	提供在机器学习中使用的优化技术的名称的例子.
	在2-3句子中简要说明生物燃料的潜在用途.
	写个涉及时间的比喻
	如果空气中氧气量增加一倍, 大家都会怎样? 写一百字.
	产生一段关于内省对成长非常重要的段落,
	为`Elephant ' 和 ' sticker ' 产生一个单行标题。
	写一篇关于全球变暖的30字论文.
	为社交媒体的网页创造座右铭,
	你能给我一个令人难以置信的STEM公平的想法吗? 这个想法是买得起的,和越南的问题有关。
	写一则关于: 这是今早海滩疯狂日出时的录像。
	给一个50岁的队友写个有趣的两刑期生日卡信息 他喜欢去Toby Carvery餐厅
	制作一个描述,
	给我一个简短的句子来在我的组合网页上 介绍一个业余数据分析师,数据科学家,和Next.js网页灵活的开发者。
	制作一个短短的1段故事 一个男孩跑在海滩上。 他是亚洲人,12岁,白天是下午4点。
	能给我一个很强的词句问题吗?
	给我三首最符合雨夜心情的乐曲
	给我一个适合雨之夜心情的乐曲的名字
	在短段落中描述死池。 确保儿童能够使用。
	写个个人推特, 介绍我如何在日出时步行到灯塔;
	写一个害怕失落的标题,
	为海滩上的贝壳视频写一个简短的FOMO标题
	另一种说法是:"Fingers跨越了一切顺利".
	写三至四行关于印度.
	写100-300字,说明压力如何影响身体和心灵.

表3:从WildChat中收集的关于多样性十分重要的开放式查询实例。

Category	Representative Examples
Yes/No Questions	Yes/No	Questions	Is ’one’s lineage’ grammatically correct? Is a single cell visible under a microscope? Can humans have natural golden bronze skin? Can you access files on the internet directly? My friend from Russia said that in Russia there is no division into cities and towns, but only cities. Is that true?	Is ’one’s lineage’ grammatically correct?	Is a single cell visible under a microscope?	Can humans have natural golden bronze skin?	Can you access files on the internet directly?	My friend from Russia said that in Russia there is no division into	cities and towns, but only cities. Is that true?
Yes/No
Questions
Is ’one’s lineage’ grammatically correct?
Is a single cell visible under a microscope?
Can humans have natural golden bronze skin?
Can you access files on the internet directly?
My friend from Russia said that in Russia there is no division into
cities and towns, but only cities. Is that true?
Questions with Precise Answers	Questions	with	Precise	Answers	What is the plural form of the ancient Greek polis? When was marriage made into an institution in Europe? What is the ULA in space exploration? In a certain language, (A) ’hu ma sam’ means ’Water is life’. (B) ’sam na zo’ means ’Glass of water’. (C) ’chi zo ma’ means ’life of PI’. Which of the following represents ’PI’ in that language? How many atoms are contained in 6.71 grams of sulfur?	What is the plural form of the ancient Greek polis?	When was marriage made into an institution in Europe?	What is the ULA in space exploration?	In a certain language, (A) ’hu ma sam’ means ’Water is life’.	(B) ’sam na zo’ means ’Glass of water’. (C) ’chi zo ma’ means ’life of PI’.	Which of the following represents ’PI’ in that language?	How many atoms are contained in 6.71 grams of sulfur?
Questions
with
Precise
Answers
What is the plural form of the ancient Greek polis?
When was marriage made into an institution in Europe?
What is the ULA in space exploration?
In a certain language, (A) ’hu ma sam’ means ’Water is life’.
(B) ’sam na zo’ means ’Glass of water’. (C) ’chi zo ma’ means ’life of PI’.
Which of the following represents ’PI’ in that language?
How many atoms are contained in 6.71 grams of sulfur?
Well-Scoped Writing	Well-Scoped	Writing	Write an email to Kathy; thank her for her fast reply and also tell her that we received her invoice. Inform her that we are doing the paperwork for this invoice and will update her about any further developments.	Write an email to Kathy; thank her for her fast reply and also tell her that	we received her invoice. Inform her that we are doing the paperwork for	this invoice and will update her about any further developments.
Well-Scoped
Writing
Write an email to Kathy; thank her for her fast reply and also tell her that
we received her invoice. Inform her that we are doing the paperwork for
this invoice and will update her about any further developments.

Yes/No
Questions

Is ’one’s lineage’ grammatically correct?
Is a single cell visible under a microscope?
Can humans have natural golden bronze skin?
Can you access files on the internet directly?
My friend from Russia said that in Russia there is no division into
cities and towns, but only cities. Is that true?

Questions
with
Precise
Answers

What is the plural form of the ancient Greek polis?
When was marriage made into an institution in Europe?
What is the ULA in space exploration?
In a certain language, (A) ’hu ma sam’ means ’Water is life’.
(B) ’sam na zo’ means ’Glass of water’. (C) ’chi zo ma’ means ’life of PI’.
Which of the following represents ’PI’ in that language?
How many atoms are contained in 6.71 grams of sulfur?

Well-Scoped
Writing

Write an email to Kathy; thank her for her fast reply and also tell her that
we received her invoice. Inform her that we are doing the paperwork for
this invoice and will update her about any further developments.

Table 4: Example closed-ended queries where diversity is NOT as important.

** 类别**	** 代表实例**
是/无问题	有/无	问 题	一个家族的语法是否正确? 单细胞在显微镜下可见吗? 人类能有天然的金铜皮吗? 你能直接上网吗? 我来自俄罗斯的朋友说,在俄罗斯,没有划分城市和城镇,只有城市. 真的吗?	一个家族的语法是否正确?	单细胞在显微镜下可见吗?	人类能有天然的金铜皮吗?	你能直接上网吗?	我来自俄罗斯的朋友说 在俄罗斯没有分裂	城市和城镇,但只有城市。 真的吗?
有/无
问 题
一个家族的语法是否正确?
单所得到的细胞在显微镜下可见吗?
人类能有天然的金铜皮吗?
你能直接上网吗?
我来自俄罗斯的朋友说 在俄罗斯没有分裂
城市和城镇,但只有城市。 真的吗?
有精确答案的问题	问 题	与	精确度	答复	古希腊波利斯的复数形式是什么? 什么时候在欧洲结婚的? 太空探索中的ULA是什么? 水是生命。 (B) ' sam na zo ' 的意思是 ' Glass of water ' 。 'chi zo ma ' 的意思是 ' pI ' 的生命 ' 。 以下哪一种语言代表`PI'? 6.71克硫磺中含有多少原子?	古希腊波利斯的复数形式是什么?	什么时候在欧洲结婚的?	太空探索中的ULA是什么?	水是生命。	(B) ' sam na zo ' 的意思是 ' Glass of water ' 。 'chi zo ma ' 的意思是 ' pI ' 的生命 ' 。	以下哪一种语言代表`PI'?	6.71克硫磺中含有多少原子?
问 题
与
精确度
答复
古希腊波利斯的复数形式是什么?
什么时候在欧洲结婚的?
太空探索中的ULA是什么?
水是生命。
(B) ' sam na zo ' 的意思是 ' Glass of water ' 。 'chi zo ma ' 的意思是 ' pI ' 的生命 ' 。
以下哪些代表该语言的`PI'相互补充?
6.71克硫磺中含有多少原子?
覆盖的写法	范围很广	写入	写一封邮件给凯西,感谢她迅速回复,并告诉她我们收到了她的发票。 通知她我们正在为这张发票做文书工作 并将向她通报任何新的进展	pillow)写一封电子邮件给凯西;感谢她的快速回复,也告诉她	我们收到了她的发票。 告诉她我们正在做文件	该发票,将更新她的任何进一步发展。
范围很广
写入
给凯西写封邮件 感谢她迅速回复 并告诉她
我们收到了她的发票。 告诉她我们正在做文件
该发票,将更新她的任何进一步发展。

有/无
问 题

一个家族的语法是否正确?
19. 单细胞在显微镜下可见吗?
人类能有天然的金铜皮吗?
你能直接上网吗?
我来自俄罗斯的朋友说 在俄罗斯没有分裂
城市和城镇,但只有城市。 真的吗?

问岔
与
没问题
答复

古希腊波利斯的多元形式是什么?
什么时候在欧洲结婚的?
太空探索中的ULA是什么?
水是生命。
(B) ' sam na zo ' 的意思是 ' Glass of water ' 。 'chi zo ma ' 的意思是 ' pI ' 的生命 ' 。
以下哪一种语言代表`PI'?
6.71克硫磺含有多少原子?

范围很广
写入

给凯西写封邮件 感谢她迅速回复 并告诉她
我们收到了她的发票。 告诉她我们正在做文件
该发票,将更新她的任何进一步发展。

表4:多样性并不那么重要的封闭式查询实例。

Appendix C Artificial Hiveminds: Examining the Intra- and Inter- Model Homogeneity

附录 C 人工蜂群心智：考察模型内与模型间同质化

In this section, we present a detailed analysis of intra- and inter-model homogeneity across a broad range of open-source and closed-source language models.

在本节中,我们详细分析了各种开源和闭源语言模式的内同和间同源性。

C.1 Evaluation Setups

C.1 评测设置

For both intra- and inter-model analyses, we adopt a unified generation protocol and reuse the same model outputs across both settings. We use 100 open-ended prompts from Infinity-Chat100, a carefully selected subset of representative queries from Infinity-Chat, as the seed prompt set for generation. A comprehensive list of language models is curated for the analysis. The full list of models considered, as well as the subset presented in the main paper, is provided in Table 5. Due to space constraints, we select representative models for the main paper based on their scale or strength within each model family.

就模型内部和模型间分析而言,我们采用统一的生成协议,并在两种情况下重新使用相同的模型产出。 我们使用来自Infinity-Chat100的100个无限制提示,这是从Infinity-Chat中精心选择的代表性查询的子集,作为种子的生成提示. 综合语言模型清单已编入分析范围。 所审议模型的完整清单以及主要文件中的子集见表5。 由于空间的限制,我们根据每个模型家族内部的规模或强度来选择主要论文的代表性模型.

For all HuggingFace models, generations are performed on NVIDIA A100 or H100 GPUs, depending on availability. For closed-source models such as OpenAI, Anthropic, Gemini, and Qwen, we use their respective APIs to obtain responses. For a small number of the largest models (e.g., deepseek-ai/DeepSeek-V3) that exceed our local GPU capacity, we use TogetherAI to generate responses. Regardless of the generation method, all models follow the same decoding configurations. For the top-$p$ setup, we use $p=0.9$, temperature $=1.0$, and a maximum generation length of 2048 tokens. For the minimum-$p$ setup, we use $p=1.0$, min-$p=0.1$, temperature $=2.0$, and the same maximum generation length. Under each decoding configuration, each model independently generates 50 responses for every open-ended prompt in Infinity-Chat100. All similarities are computed using cosine similarity between sentence embeddings of the responses, generated by OpenAI’s embedding model text-embedding-3-small.

对于所有的HuggingFace模型,根据可用性的不同,在NVIDIA A100或H100 GPU上进行世代表演. 对于OpenAI,Anthropic,双子座,Quen等闭源模型,我们使用各自的API来获取响应. 对于少数超过我们本地GPU容量的最大型号(如深层搜索-ai/深层搜索-V3),我们使用UnitedAI生成响应. 无论生成方法如何,所有模型都遵循相同的解码配置. 对于顶端-$p$设置,我们使用$p=0.9$,温度$=1.0$,最大生成长度为2048令牌. 对于最小-$p$设置,我们使用$p=1.0$,min-$p=0.1$,温度$=2.0$,最大生成长度相同. 在每个解码配置下,每个模型独立生成50个响应,每个无限-Chat100的无限提示. 所有相似之处都是使用OpenAI嵌入模型文本嵌入-3-小所生成的答复嵌入句子的同位素相似性计算出来的。

For concrete examples of response pairs and their corresponding similarity scores, see Tables 12–16.

关于反应对及其相应的相似性分数的具体例子,见表12-16。

C.2 Intra-Model Homogeneity

C.2 模型内同质化

Table 6 presents the extended results of intra-model repetition for all models listed in Table 5, serving as a detailed counterpart to the results summarized in Figure 6.

表6列出表5所列所有模式内重复的扩展结果,作为图6汇总结果的详细对应结果。

C.3 Inter-Model Homogeneity

C.3 模型间同质化

Table 7-11 present the extended results of inter-model homogeneity for all models listed in Table 5, serving as a detailed counterpart to the results summarized in Figure 6.

表7-11列出表5所列所有模型的模型间同质性的扩展结果,作为图6汇总结果的详细对应结果。

C.4 Examining How Paraphrased Queries Influence Response Homogeneity

C.4 释义查询如何影响回答同质化

We conduct an experiment to examine how prompt paraphrasing affects response similarity across language models. Starting with 30 prompts from our evaluation set Infinity-Chat100, we use gpt-4.1-2025-04-14 to generate 4 paraphrases for each original prompt, ensuring the paraphrases maintain similar semantic meaning without drastic changes in connotation, with further LLM judge verification. For each original prompt and its paraphrases (30 queries $\times$ 5 variants = 150 prompts total), we generate 20 responses using 42 representative models from different model families and sizes. We use consistent sampling parameters from our previous experiments: top-p$=0.9$ and temperature$=1$. We then obtain sentence embeddings for all responses using OpenAI’s text-embedding-3-small model.

我们进行了一项试验,以审查迅速抛写如何影响不同语言模型的反应相似性。 从我们评价集Infinity-Chat100的30个提示开始,我们使用gpt-4.1-2025-04-14为每个原始提示生成4个解码,确保解码句保持类似的语义意义而不对内涵发生剧烈变化,进一步进行LLM法官验证. 对于每个原始的快报及其解码(30个查询$\times$ 5个变体=150个快报总数),我们利用来自不同模系和大小的42个代表性模型生成20个回复. 我们使用我们以往实验的一致取样参数:顶-p $=0.9$和温度 $=1$。 使用OpenAI的文本嵌入-3-小模型,

To measure response consistency, we compute two types of semantic similarity scores:

为了衡量响应的一致性,我们计算两类语义相似性分数:

• • Within-prompt similarity: Average pairwise similarity among the 20 responses generated from the same original prompt
• • Cross-paraphrase similarity: Average similarity between responses from the original prompt and responses from its paraphrases

• ** 瞬间类似**: 同一原始提示产生的20份答复的平均对等性

• ** 交叉语的相似性**: 原即时答复与原释答复的平均相似性

Our results across all 42 models show that the within-prompt similarity averaged 0.821, while the cross-paraphrase similarity averaged 0.781. Although responses to the original prompts showed slightly higher similarity scores, responses to paraphrased prompts also demonstrated high similarity (difference of only 0.04). This suggests that language models generate relatively consistent responses even when prompts are paraphrased.

我们对所有42个模型的结果显示,* 瞬间相似性* 平均为0.821,而* 跨参数相似性* 平均为0.781。 尽管对原始提示的答复显示的相似性分数略高,但对被解释的提示的答复也显示高度相似(差幅仅为0.04)。 这表明语言模型即使被解释的提示,也会产生相对一致的响应.

To better illustrate the observed similarities, we include concrete examples that highlight how model responses vary across paraphrased prompts and across different models in Table 17 and 18. In some cases, models exhibit high-level conceptual similarity, for instance, in Table 17, all responses frame the metaphor around the idea that “time is a river.” In other cases, the similarity is more surface-level; for example, in Table 18, many responses reuse the word “profound” in the opening sentence.

为了更好地说明所观察到的相似之处,我们列举了具体的例子,着重说明表17和表18中不同模型在解释性提示和不同模型中的不同反应方式。 在某些情况下,模型在概念上具有高度的相似性,例如,在表17中,所有答复都以“时间是一条河流”为隐喻。 在其他情况下,相似性更具有表面水平;例如,在表18中,许多答复在开头句中重复使用“问题”一词。

Type	In Main?	HuggingFace Model ID
Open-Source		meta-llama/Llama-3.1-8B-Instruct
x	meta-llama/Llama-3.1-70B-Instruct
x	meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo
	meta-llama/Llama-3.2-1B-Instruct
	meta-llama/Llama-3.2-3B-Instruct
x	meta-llama/Llama-3.3-70B-Instruct
	google/gemma-2-2b-it
	google/gemma-2-9b-it
x	google/gemma-2-27b-it
	google/gemma-1.1-2b-it
	google/gemma-1.1-7b-it
	Qwen/Qwen1.5-0.5B-Chat
	Qwen/Qwen1.5-1.8B-Chat
	Qwen/Qwen1.5-4B-Chat
	Qwen/Qwen1.5-7B-Chat
	Qwen/Qwen1.5-14B-Chat
	Qwen/Qwen1.5-32B-Chat
	Qwen/Qwen1.5-72B-Chat
x	Qwen/Qwen1.5-110B-Chat
	Qwen/Qwen2-0.5B-Instruct
	Qwen/Qwen2-1.5B-Instruct
x	Qwen/Qwen2-72B-Instruct
	Qwen/Qwen2.5-0.5B-Instruct
	Qwen/Qwen2.5-1.5B-Instruct
	Qwen/Qwen2.5-3B-Instruct
	Qwen/Qwen2.5-7B-Instruct
	Qwen/Qwen2.5-14B-Instruct
	Qwen/Qwen2.5-32B-Instruct
x	Qwen/Qwen2.5-72B-Instruct
	Qwen/Qwen2.5-7B-Instruct-1M
	Qwen/Qwen2.5-14B-Instruct-1M
	Qwen/Qwen3-0.6B
	Qwen/Qwen3-1.7B
	Qwen/Qwen3-4B
	Qwen/Qwen3-8B
	Qwen/Qwen3-14B
x	Qwen/Qwen3-32B
x	deepseek-ai/DeepSeek-V3
	mistralai/Mistral-Small-24B-Instruct-2501
	mistralai/Mistral-7B-Instruct-v0.1
	mistralai/Mistral-7B-Instruct-v0.2
	mistralai/Mistral-7B-Instruct-v0.3
	mistralai/Ministral-8B-Instruct-2410
	mistralai/Mistral-Nemo-Instruct-2407
	mistralai/Mistral-Small-Instruct-2409
x	mistralai/Mistral-Large-Instruct-2411
x	mistralai/Mixtral-8x7B-Instruct-v0.1
	microsoft/Phi-3-mini-128k-instruct
	microsoft/Phi-3.5-mini-instruct
x	microsoft/phi-4
	CohereForAI/aya-expanse-8b
x	CohereForAI/aya-expanse-32b
x	CohereForAI/c4ai-command-r-plus-08-2024
	CohereForAI/c4ai-command-r-08-2024
x	allenai/OLMo-2-1124-13B-Instruct
	allenai/OLMo-2-1124-7B-Instruct
	allenai/Llama-3.1-Tulu-3-8B
x	allenai/Llama-3.1-Tulu-3-70B
Closed-Source		gpt-4o-2024-11-20
x	gpt-4o-2024-08-06
	gpt-4o-2024-05-13
x	gpt-4o-mini-2024-07-18
	gpt-4-turbo-2024-04-09
x	claude-3-5-sonnet-20241022
	claude-3-5-haiku-20241022
x	claude-3-sonnet-20240229
	claude-3-haiku-20240307
x	claude-3-opus-20240229
	gemini-1.5-flash
x	gemini-1.5-pro
x	gemini-2.0-flash
	gemini-2.0-flash-lite-preview-02-05
x	qwen-max-2025-01-25
x	qwen-plus-2025-01-25
	qwen-turbo-2024-11-01

Table 5: The full list of models that we consider and models that are selected for the main paper.

**立足于此。	** 主机? **	** Hugging Face 型号ID**
开源		元拉玛/拉玛3.1-8B-指令
x	元拉玛/拉玛-3.1-70B-指令
x	mata-lama/Meta-Llama-3.1-405B-教程图尔博
	元拉玛/拉玛3.2-1B-指令
	元拉玛/拉玛3.2-3B-指令
x	元拉玛/拉玛3.3-70B-指令
	谷歌/宝石-2-2b-it
	谷歌/宝石-2-9b-it
x	谷歌/宝石-2-27b-it
	谷歌/宝石1.1-2b-it
	谷歌/宝石1.1-7b-it
	Quen/Qwen1.5-0.5B-Chat (英语).
	Quen/Quen1.5-1.8B-Chat (中文(简体) ).
	Quen/Quen1.5-4B-Chat (中文(简体) ).
	Quen/Quen1.5-7B-Chat (中文(简体) ).
	Quen/Qu 如果是的话 15-14B-Chart
	Quen/Quen1.5-32B-Chat (英语).
	Quen/Quen1.5-72B-Chat (中文(简体) ).
x	Quen/Quen1.5-110B-Chat (中文(简体) ).
	文/文2-0.5B-指示
	文/文2-1.5B-指示
x	文/文2-72B-指示
	文/文2.5-0.5B-指示
	文/文2.5-1.5B-指示
	文/文2.5-3B-指示
	文/文2.5-7B-指示
	文/文2.5-14B-指示
	Quen/Quen2.5-32B-指令
x	Quen/Quen2.5-72B-指示
	文/文2.5-7B-指令-1M
	文/文2.5-14B-指令-1M
	Quen/Quen3-0.6B (中文(简体) ).
	Quen/Quen3-1.7B (中文(简体) ).
	Quen/Quen3-4B (中文(简体) ).
	Quen/Quen3-8B (中文(简体) ).
	Quen/Quen3-14B (中文(简体) ).
x	Quen/Qwen3-32B (中文(简体) ).
x	深层/深层
	误差/Mistral-Small-24B-指令2501
	错特拉莱/Mistral-7B-Instruct-v0.1
	误差/Mistral-7B-Instruct-v0.2
	错特拉莱/Mistral-7B-Instruct-v0.3
	错误/部长 8B - 指令 2410
	错特拉伊/米斯特拉尔-尼莫-指令 2407
	mistral-Small-Instruct 2409 (英语).
x	错特拉伊/Mistral-Large-Instruct-2411
x	mistralai/Mixtral-8x7B-Instruct-v0.1 导弹发射系统
	微 我们正在使用微软/Phi-3-mini-128k-struct
	微软/Phi-3.5-微型指令
x	微软/菲-4
	协进会/亚-扩展-8b
x	合作论坛/亚-广32b
x	CohereForAI/c4ai-command-r-plus-08-2024 (韩语).
	CohereForAI/c4ai-command-r-08-2024 (韩语).
x	Alenai/OLMo-2-1124-13B-指令
	Alenai/OLMo-2-1124-7B-指示
	ổ娜/拉玛3.1-图鲁-3-8B
x	阿酝奈/拉玛-3.1-图鲁-3-70B
闭源		gpt-4o-2024-11-20 (韩语).
x	gpt-4o-2024-08-06 (韩语).
	gpt-4o-2024-05-13 (英语).
x	gpt-4o-mini-2024-07-18 (英语).
	gpt-4-涡轮-2024-04-09
x	克劳德-3-5-孙网-20241022
	克劳德-3-5-海库-20241022
x	克劳德-3-孙网-20240229
	克劳德-3-海库-20240307
x	克劳德-3-opus-20240229
	遗传1.5-闪光
x	遗传1.5-pro
x	双倍线
	格米尼2.0-闪电-立特-回顾-02-05
x	qwen-max-2025-01-25 (中文(简体) ).
x	qwen-plus-2025-01-25 (中文(简体) ).
	qwen-turbo-2024-11-01 (英语).

表5:我们所考虑的模型和为主要论文选定的模型的全部清单。

Similarity Range	0.9-1.0	0.8-0.9	0.7-0.8	0.6-0.7	0.5-0.6	0.4-0.5	0.3-0.4	0.2-0.3	0.1-0.2	0.0-0.1
Average	33.65	36.47	16.29	7.46	3.00	1.54	1.13	0.43	0.04	0.00
gpt-4o-2024-11-20	51.00	36.00	10.00	1.00	2.00	0.00	0.00	0.00	0.00	0.00
gpt-4o-2024-08-06	44.00	37.00	12.00	4.00	1.00	0.00	2.00	0.00	0.00	0.00
gpt-4o-2024-05-13	40.00	36.00	16.00	5.00	1.00	1.00	1.00	0.00	0.00	0.00
gpt-4o-mini-2024-07-18	53.00	34.00	9.00	4.00	0.00	0.00	0.00	0.00	0.00	0.00
gpt-4-turbo-2024-04-09	38.00	44.00	11.00	5.00	1.00	0.00	1.00	0.00	0.00	0.00
claude-3-5-sonnet-20241022	61.00	22.00	9.00	5.00	2.00	1.00	0.00	0.00	0.00	0.00
claude-3-5-haiku-20241022	56.00	33.00	7.00	3.00	1.00	0.00	0.00	0.00	0.00	0.00
claude-3-sonnet-20240229	48.00	36.00	10.00	5.00	1.00	0.00	0.00	0.00	0.00	0.00
claude-3-haiku-20240307	48.00	33.00	15.00	2.00	1.00	0.00	1.00	0.00	0.00	0.00
claude-3-opus-20240229	59.00	27.00	7.00	3.00	4.00	0.00	0.00	0.00	0.00	0.00
deepseek-ai/DeepSeek-V3	42.00	39.00	13.00	3.00	0.00	3.00	0.00	0.00	0.00	0.00
meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo	43.00	38.00	11.00	3.00	4.00	0.00	0.00	1.00	0.00	0.00
meta-llama/Llama-3.1-8B-Instruct	19.00	52.00	14.00	5.00	6.00	1.00	2.00	1.00	0.00	0.00
meta-llama/Llama-3.1-70B-Instruct	23.00	44.00	23.00	5.00	2.00	2.00	0.00	1.00	0.00	0.00
meta-llama/Llama-3.2-1B-Instruct	5.00	34.00	38.00	8.00	5.00	6.00	1.00	3.00	0.00	0.00
meta-llama/Llama-3.2-3B-Instruct	20.00	44.00	20.00	5.00	5.00	2.00	2.00	2.00	0.00	0.00
meta-llama/Llama-3.3-70B-Instruct	51.00	30.00	12.00	4.00	1.00	1.00	1.00	0.00	0.00	0.00
google/gemma-2-2b-it	19.00	46.00	19.00	9.00	3.00	2.00	2.00	0.00	0.00	0.00
google/gemma-2-9b-it	30.00	41.00	19.00	6.00	0.00	2.00	2.00	0.00	0.00	0.00
google/gemma-2-27b-it	33.00	43.00	14.00	7.00	0.00	2.00	1.00	0.00	0.00	0.00
google/gemma-1.1-2b-it	17.00	39.00	30.00	8.00	3.00	2.00	1.00	0.00	0.00	0.00
google/gemma-1.1-7b-it	22.00	45.00	18.00	12.00	2.00	1.00	0.00	0.00	0.00	0.00
Qwen/Qwen2.5-0.5B-Instruct	1.00	23.00	24.00	27.00	7.00	13.00	2.00	3.00	0.00	0.00
Qwen/Qwen2.5-1.5B-Instruct	3.00	39.00	20.00	16.00	11.00	8.00	2.00	1.00	0.00	0.00
Qwen/Qwen2.5-3B-Instruct	14.00	41.00	26.00	12.00	3.00	2.00	1.00	1.00	0.00	0.00
Qwen/Qwen2.5-7B-Instruct	31.00	35.00	23.00	5.00	4.00	1.00	0.00	1.00	0.00	0.00
Qwen/Qwen2.5-14B-Instruct	34.00	35.00	22.00	5.00	2.00	1.00	0.00	1.00	0.00	0.00
Qwen/Qwen2.5-32B-Instruct	36.00	44.00	16.00	2.00	1.00	0.00	0.00	1.00	0.00	0.00
Qwen/Qwen2.5-72B-Instruct	48.00	26.00	19.00	4.00	1.00	1.00	1.00	0.00	0.00	0.00
Qwen/Qwen2.5-7B-Instruct-1M	31.00	36.00	16.00	12.00	3.00	0.00	2.00	0.00	0.00	0.00
Qwen/Qwen2.5-14B-Instruct-1M	43.00	27.00	16.00	9.00	3.00	1.00	0.00	1.00	0.00	0.00
Qwen/Qwen2-0.5B-Instruct	0.00	20.00	21.00	23.00	20.00	9.00	4.00	2.00	1.00	0.00
Qwen/Qwen2-1.5B-Instruct	2.00	32.00	24.00	22.00	11.00	3.00	4.00	2.00	0.00	0.00
Qwen/Qwen2-72B-Instruct	35.00	35.00	24.00	5.00	0.00	0.00	1.00	0.00	0.00	0.00
Qwen/Qwen1.5-0.5B-Chat	0.00	23.00	27.00	24.00	12.00	6.00	6.00	2.00	0.00	0.00
Qwen/Qwen1.5-1.8B-Chat	12.00	41.00	27.00	9.00	4.00	3.00	3.00	1.00	0.00	0.00
Qwen/Qwen1.5-4B-Chat	6.00	46.00	24.00	14.00	4.00	2.00	3.00	1.00	0.00	0.00
Qwen/Qwen1.5-7B-Chat	26.00	40.00	16.00	10.00	4.00	2.00	1.00	1.00	0.00	0.00
Qwen/Qwen1.5-14B-Chat	37.00	33.00	22.00	3.00	5.00	0.00	0.00	0.00	0.00	0.00
Qwen/Qwen1.5-32B-Chat	37.00	37.00	17.00	7.00	1.00	1.00	0.00	0.00	0.00	0.00
Qwen/Qwen1.5-72B-Chat	50.00	35.00	9.00	3.00	1.00	1.00	1.00	0.00	0.00	0.00
Qwen/Qwen1.5-110B-Chat	48.00	36.00	10.00	2.00	3.00	0.00	1.00	0.00	0.00	0.00
mistralai/Mistral-Small-24B-Instruct-2501	31.00	39.00	17.00	4.00	4.00	2.00	2.00	1.00	0.00	0.00
mistralai/Mistral-7B-Instruct-v0.1	24.00	42.00	23.00	5.00	1.00	2.00	2.00	1.00	0.00	0.00
mistralai/Mistral-7B-Instruct-v0.2	38.00	40.00	13.00	4.00	3.00	1.00	1.00	0.00	0.00	0.00
mistralai/Mistral-7B-Instruct-v0.3	30.00	39.00	18.00	8.00	3.00	1.00	1.00	0.00	0.00	0.00
mistralai/Ministral-8B-Instruct-2410	10.00	38.00	25.00	14.00	7.00	2.00	3.00	1.00	0.00	0.00
mistralai/Mistral-Nemo-Instruct-2407	19.00	42.00	22.00	9.00	3.00	1.00	4.00	0.00	0.00	0.00
mistralai/Mistral-Small-Instruct-2409	23.00	42.00	20.00	6.00	6.00	3.00	0.00	0.00	0.00	0.00
mistralai/Mistral-Large-Instruct-2411	43.00	34.00	15.00	5.00	2.00	0.00	1.00	0.00	0.00	0.00
mistralai/Mixtral-8x7B-Instruct-v0.1	45.00	41.00	9.00	4.00	1.00	0.00	0.00	0.00	0.00	0.00
microsoft/phi-4	38.00	40.00	11.00	9.00	2.00	0.00	0.00	0.00	0.00	0.00
microsoft/Phi-3.5-mini-instruct	33.00	42.00	12.00	8.00	2.00	2.00	1.00	0.00	0.00	0.00
microsoft/Phi-3-mini-128k-instruct	15.00	40.00	22.00	13.00	5.00	2.00	2.00	1.00	0.00	0.00
o1-2024-12-17	27.00	37.00	20.00	9.00	4.00	0.00	3.00	0.00	0.00	0.00
o1-mini-2024-09-12	38.00	40.00	17.00	4.00	1.00	0.00	0.00	0.00	0.00	0.00
o1-preview-2024-09-12	40.00	33.00	16.00	8.00	1.00	2.00	0.00	0.00	0.00	0.00
o3-mini-2025-01-31	34.00	35.00	18.00	10.00	0.00	2.00	1.00	0.00	0.00	0.00
CohereForAI/aya-expanse-8b	40.00	44.00	7.00	7.00	1.00	0.00	1.00	0.00	0.00	0.00
CohereForAI/aya-expanse-32b	50.00	36.00	8.00	5.00	1.00	0.00	0.00	0.00	0.00	0.00
CohereForAI/c4ai-command-r-plus-08-2024	24.00	32.00	19.00	10.00	7.00	3.00	3.00	1.00	1.00	0.00
CohereForAI/c4ai-command-r-08-2024	27.00	35.00	14.00	16.00	2.00	4.00	1.00	1.00	0.00	0.00
allenai/OLMo-2-1124-13B-Instruct	29.00	39.00	12.00	9.00	7.00	2.00	1.00	1.00	0.00	0.00
allenai/OLMo-2-1124-7B-Instruct	30.00	38.00	15.00	10.00	3.00	3.00	1.00	0.00	0.00	0.00
allenai/Llama-3.1-Tulu-3-8B	27.00	36.00	22.00	8.00	3.00	1.00	2.00	0.00	1.00	0.00
allenai/Llama-3.1-Tulu-3-70B	28.00	39.00	18.00	11.00	1.00	2.00	0.00	1.00	0.00	0.00
qwen-max-2025-01-25	55.00	37.00	4.00	3.00	1.00	0.00	0.00	0.00	0.00	0.00
qwen-plus-2025-01-25	56.00	28.00	11.00	3.00	2.00	0.00	0.00	0.00	0.00	0.00
qwen-turbo-2024-11-01	37.00	33.00	20.00	8.00	1.00	1.00	0.00	0.00	0.00	0.00
Qwen/Qwen3-0.6B	13.00	43.00	27.00	8.00	6.00	1.00	2.00	0.00	0.00	0.00
Qwen/Qwen3-1.7B	44.00	30.00	14.00	11.00	1.00	0.00	0.00	0.00	0.00	0.00
Qwen/Qwen3-4B	51.00	37.00	4.00	4.00	2.00	2.00	0.00	0.00	0.00	0.00
Qwen/Qwen3-8B	56.00	26.00	15.00	1.00	1.00	0.00	1.00	0.00	0.00	0.00
Qwen/Qwen3-14B	52.00	31.00	10.00	4.00	0.00	1.00	2.00	0.00	0.00	0.00
Qwen/Qwen3-32B	40.00	36.00	14.00	6.00	1.00	1.00	2.00	0.00	0.00	0.00
gemini-1.5-flash	62.00	26.00	5.00	6.00	1.00	0.00	0.00	0.00	0.00	0.00
gemini-1.5-pro	53.00	32.00	11.00	3.00	1.00	0.00	0.00	0.00	0.00	0.00
gemini-2.0-flash	40.00	41.00	11.00	4.00	2.00	0.00	2.00	0.00	0.00	0.00
gemini-2.0-flash-lite-preview-02-05	40.00	41.00	8.00	6.00	3.00	1.00	1.00	0.00	0.00	0.00

Table 6: Full results of the intra-model repetition analysis in Figure 4 of the main paper.