The Value of Prediction in Identifying the Worst-Off

Unai Fischer-Abaigar    Christoph Kern    Juan Carlos Perdomo † † thanks: Supported by the Center for Research on Computation and Society (CRCS) at Harvard University.

Abstract

摘要

Machine learning is increasingly used in government programs to identify and support the most vulnerable individuals, prioritizing assistance for those at greatest risk over optimizing aggregate outcomes. This paper examines the welfare impacts of prediction in equity-driven contexts, and how they compare to other policy levers, such as expanding bureaucratic capacity. Through mathematical models and a real-world case study on long-term unemployment amongst German residents, we develop a comprehensive understanding of the relative effectiveness of prediction in surfacing the worst-off. Our findings provide clear analytical frameworks and practical, data-driven tools that empower policymakers to make principled decisions when designing these systems.

机器学习越来越多用于政府项目，以识别并支持最脆弱的个体，优先援助风险最高者，而非优化总体结果。本文考察公平驱动语境下预测的福利影响，以及它们与扩展官僚能力等其他政策杠杆相比如何。通过数学模型与德国居民长期失业的真实案例研究，我们系统理解预测在显现最弱势群体方面的相对有效性。我们的发现提供清晰的分析框架与实用的数据驱动工具，帮助政策制定者在设计这些系统时做出有原则的决策。

1LMU Munich      2Munich Center for Machine Learning      3Harvard University

1 Introduction

1 引言

Faced with pressure to modernize, large bureaucracies are increasingly adopting risk prediction tools to improve efficiency and better serve their populations. Beyond optimizing aggregate outcomes, investments in these programs often aim to address historical inequities and prioritize the needs of the worst-off. For instance, in 2012, Wisconsin launched a risk prediction system to explicitly address deep racial disparities in academic achievement and improve high school graduation rates amongst underserved students. More broadly, such systems are particularly relevant in settings where normative considerations demand prioritizing those at the greatest risk of adverse outcomes, and where well-established downstream interventions can meaningfully benefit these vulnerable individuals.

面对现代化压力，大型官僚机构正越来越多采用风险预测工具以提高效率、更好服务人群。除优化总体结果外，这些项目投资往往旨在纠正历史不公、优先照顾最弱势群体需求。例如 2012 年威斯康星州上线风险预测系统，明确应对学业成就上的深度种族差距，并提升服务不足学生的高中毕业率。更广泛地，当规范考量要求优先帮助不良结果风险最高者，且已有成熟下游干预能切实惠及这些脆弱个体时，此类系统尤为相关。

From a design perspective, these risk predictors are challenging to evaluate because their value cannot be assessed without reference to the broader social context. The value of a risk predictor is ultimately determined by its impact on bottom-line welfare (e.g., graduation rates) and how these welfare impacts compare to those of other bureaucratic alternatives [Johnson and Zhang, 2022]. For example, to understand whether investments in prediction are truly valuable in Wisconsin, we need to assess how much better the risk predictor is at identifying at-risk students relative to existing policies. We also need to understand whether sophisticated prediction systems yield higher graduation rates amongst the underserved than structural investments in teacher training or better facilities.

从设计视角看，这些风险预测器难评，因为其价值不能脱离更广社会语境。风险预测器的价值最终由其底部线福利影响（如毕业率）决定，以及这些福利影响与其他官僚替代方案相比如何 [Johnson and Zhang, 2022]。例如，要理解威斯康星州对预测的投资是否真正有价值，需评估相对现有政策，风险预测器识别风险学生好多少；也需理解相对教师培训或更好设施等结构性投资，复杂预测系统是否为服务不足群体带来更高毕业率。

Equity-driven programs are pervasive in applications like social housing, poverty targeting, and unemployment assistance. In these contexts, many government agencies are exploring how algorithmic prediction systems may be an improvement over their current profiling processes [Körtner and Bonoli, 2023]. Yet, due to the absence of an overarching framework that allows the systematic assessment of the relative impacts of different design decisions, efforts to improve predictive accuracy are rarely studied in concert with other policy levers such as expanding screening capacity.

公平驱动项目在社会住房、贫困瞄准与失业援助等应用中普遍存在。在这些语境下，许多政府机构正在探索算法预测系统如何改进当前画像流程 [Körtner and Bonoli, 2023]。然而，因缺乏可系统评估不同设计决策相对影响的总体框架，提升预测精度的努力很少与扩展筛查能力等其他政策杠杆一并研究。

Building on recent work in a budding area of learning in resource allocation contexts, we develop tools to evaluate the design and broader impact of prediction systems that aim to identify the worst-off members of a population. We develop a holistic understanding of the value of statistical prediction in these contexts through theoretical insights into foundational statistical models and a real-world case study on identifying long-term unemployment. Our results establish clear theoretical and empirical criteria characterizing the relative value of core design decisions within these problems. Specifically, we identify when improving prediction provides a higher marginal benefit in helping an institution identify the worst-off. This is compared to alternative strategies, such as keeping prediction accuracy fixed, expanding bureaucratic capacity and screening a larger population.

在资源配置语境下学习这一新兴领域的近期工作基础上，我们开发工具以评估旨在识别人群中最弱势群体的预测系统的设计与更广影响。通过基础统计模型上的理论洞见，以及识别长期失业的真实案例研究，我们建立对这些语境中统计预测价值的整体理解。结果给出刻画这些核心设计决策相对价值的清晰理论与经验标准。具体地，我们识别何时改进预测在帮助机构识别最弱势群体上提供更高边际收益，并与保持预测精度不变、扩展官僚能力并筛查更大人群等替代策略对比。

Interestingly, we show that prediction is a first and last-mile effort. The impacts of improving prediction are always outweighed by those of expanded screening capacity, except for when the system explains either none or almost all of the variance in outcomes. While this relationship is moderated by costs, it still largely holds when prediction improvements are more cost-efficient than measures that expand access.

有趣的是，我们表明预测是「第一英里」与「最后一英里」努力。除系统几乎解释不了或几乎解释全部结果方差外，改进预测的影响总被扩展筛查能力的影响压过。虽受成本调节，但当预测改进比扩展接入的措施更划算时，该关系在很大程度上仍成立。

These results are counternarrative to current efforts in empirical public policy where agencies focus on incremental improvements within complex prediction systems, starting from the solid baseline performances of their current processes [Desiere et al., 2019, Desiere and Struyven, 2021]. Furthermore, implementing more complex profiling systems at scale comes with operational costs (such as staff training and data collection) which need to be contextualized by the cost-benefit ratio of expanding access. Our empirical case study explicates how to systematically assess the relative gains of these design components in a real-world application setting, translating formal insights into critical guidance for designers of these systems.

这些结果与当前实证公共政策中的努力形成对照——机构往往从现有流程的扎实基线出发，专注复杂预测系统内的增量改进 [Desiere et al., 2019, Desiere and Struyven, 2021]。此外，规模化部署更复杂画像系统伴随运营成本（如员工培训与数据采集），需用扩展接入的成本收益比来语境化。我们的实证案例阐明如何在真实应用设定中系统评估这些设计组件的相对收益，把形式洞见转化为系统设计者的关键指导。

Our results provide theoretically principled and empirically grounded tools for policymakers to make informed decisions when designing prediction systems to identify the worst-off. They also offer a practical framework to help determine how much should be invested in prediction relative to other interventions and how to decide when prediction systems are “good enough" for deployment.

我们的结果为政策制定者设计识别最弱势群体的预测系统提供有理论原则且有经验依据的工具；也给出实践框架，帮助决定相对其他干预应向预测投资多少，以及何时预测系统「足够好」可部署。

1.1 Overview of Framework and Contributions

1.1 框架与贡献概览

Setup.

We consider a scenario where a decision-maker seeks to identify worst-off members of a population, as determined by a real-valued welfare metric $Y\in\mathbb{R}$, with the goal of prioritizing them for further screening and support. The population is represented by a distribution ${\cal D}$ over features $X$ and outcomes $Y$. The planner aims to identify all individuals whose outcomes $Y$ fall below some threshold $t(\beta)$, $Y\leqslant t(\beta)$. Here, $\beta\in[0,1]$ is a parameter (quantile) that determines the size of the population that is at risk, $\mathrm{Pr}[Y\leqslant t(\beta)]=\beta$. For instance, in poverty prediction, $Y$ is income, and the goal is to identify everyone whose income is below some value.

设定。

我们考虑决策者寻求识别人群中最弱势群体的场景，由实值福利度量 $Y\in\mathbb{R}$ 决定，目标是优先对他们做进一步筛查与支持。人群由特征 $X$ 与结果 $Y$ 上的分布 ${\cal D}$ 表示。规划者旨在识别所有结果 $Y$ 低于某阈值 $t(\beta)$ 的个体，即 $Y\leqslant t(\beta)$。这里 $\beta\in[0,1]$ 是决定风险人群规模的参数（分位数），$\mathrm{Pr}[Y\leqslant t(\beta)]=\beta$。例如在贫困预测中，$Y$ 是收入，目标是识别收入低于某值的所有人。

To solve this problem, the social planner has access to data $(X,Y)\sim{\cal D}$ and builds a screening policy $\pi:\mathcal{X}\rightarrow\{0,1\}$ that determines whether an individual with features $x$ is screened from the broader population to see if they belong to the worst-off group. Learning plays a fundamental role since the optimal policy is to predict each person’s expected outcome, $f(x)=\hat{Y}\approx\E\left[Y\mid X=x\right]$ and screen those in the bottom fraction, $\pi_{f}(x)=1\{f(x)\leqslant t(\alpha)\}$.

为解决该问题，社会规划者可及数据 $(X,Y)\sim{\cal D}$，并构建筛查策略 $\pi:\mathcal{X}\rightarrow\{0,1\}$，决定具有特征 $x$ 的个体是否从更广人群中被筛出，以查看其是否属于最弱势群体。学习起基础作用，因为最优策略是预测每人期望结果 $f(x)=\hat{Y}\approx\E\left[Y\mid X=x\right]$，并筛查底部比例 $\pi_{f}(x)=1\{f(x)\leqslant t(\alpha)\}$。

Unpacking this further, $\alpha\in[0,1]$, is a design parameter that determines how many people the planner can screen, $\mathrm{Pr}[f(x)\leqslant t(\alpha)]=\alpha$. The amount of resources $\alpha$ need not be equal to the size of the target population $\beta$. For instance, an organization might have normative goal of identifying the poorest 5% of individuals, but only have the resource to screen $1\%$ of the population. Conversely, they might realize that predictions are not perfect, and that to identify the bottom $5\%$, they might have to screen $10\%$ of the population.

进一步拆开，$\alpha\in[0,1]$ 是决定规划者可筛查人数的设计参数，$\mathrm{Pr}[f(x)\leqslant t(\alpha)]=\alpha$。资源量 $\alpha$ 不必等于目标人群规模 $\beta$。例如，组织可能有识别最穷 5% 个体的规范目标，但只有筛查 1% 人口的资源；反过来，他们可能意识到预测不完美，为识别底部 5% 可能需筛查 10% 人口。

Given a predictor $f$, a screening budget of $\alpha$, and a target parameter $\beta$, the value of a prediction system is equal to the fraction of the at-risk population that it identifies,

给定预测器 $f$、筛查预算 $\alpha$ 与目标参数 $\beta$，预测系统的价值等于其所识别的风险人群比例，

$$\displaystyle V(\alpha,f;\beta)=\mathrm{Pr}_{{\cal D}}[f(x)\leqslant t(\alpha)\mid Y\leqslant t(\beta)],$$

where again $t(\alpha),t(\beta)$ are chosen to respect the design constraints. We focus on this notion of value since our driving motivation is to analyze domains like unemployment assistance, or poverty prediction, where there is no harm in the prediction system raising a false positive $(\pi(x)=1,Y>t(\beta))$. By and large, the true value of the system is equal to the extent that it helps an institution efficiently identify the needy amongst a large, diverse population.

其中 $t(\alpha),t(\beta)$ 再次按设计约束选取。我们聚焦该价值概念，因为驱动力是分析失业援助或贫困预测等领域——预测系统产生假阳性（$\pi(x)=1,Y>t(\beta)$）并无伤害。大体上，系统的真正价值等于它帮助机构在庞大多样人群中高效识别需助者的程度。

The focus of our work is to build a holistic understanding of prediction in these contexts by evaluating the relative impacts of different design parameter, such as expanding screening capacity or improving prediction, on this notion of bottom-line value $V(\alpha,f;\beta)$. We develop these insights through theoretical investigations as well as in-depth empirical case study.

工作重点是通过评估扩展筛查能力或改进预测等不同设计参数对底部线价值 $V(\alpha,f;\beta)$ 的相对影响，建立对这些语境中预测的整体理解。我们通过理论考察与深入实证案例研究发展这些洞见。

Mathematical Results.

Following Perdomo [2024], we formalize the relative value of prediction for the worst-off by studying the prediction-access ratio or PAR. Intuitively, the PAR measures the relative change in value achieved by optimizing different policy levers.

数学结果。

遵循 Perdomo [2024]，我们通过研究预测–接入比（prediction-access ratio，PAR）形式化对最弱势群体的预测相对价值。直观上，PAR 度量优化不同政策杠杆所实现的价值相对变化。

$$\displaystyle\text{PAR}=\frac{\text{ Marginal Value of Expanding Access}}{\text{Marginal Value of Better Prediction}}.$$

We formally define this quantity in Equation 3. While initially developed to specifically study the value of prediction in allocation problems where allocating goods to individuals had heterogeneous effects, here we extend this concept to analyze the value of prediction in a related, but distinct, setting where we aim to identify the worst-off.

我们在公式 3 中形式定义该量。它最初专门研究分配问题中预测价值——向个体分配物品有异质效应；这里我们把概念扩展到相关但不同的设定：旨在识别最弱势群体。

Small values of the PAR (i.e. PAR $<1$) indicate that small improvements in prediction yield a much larger (relative) impact in the ability to target the worst-off than a small expansion in screening capacity. The opposite is true if the PAR is greater than 1. Calculating this quantity is a fundamental step in deciding which policy lever makes economic sense.

PAR 较小（即 PAR $<1$）表明：预测的小改进对瞄准最弱势群体的能力，相对筛查能力的小幅扩展，有大得多的（相对）影响。若 PAR 大于 1 则相反。计算该量是决定哪个政策杠杆在经济上说得通的基础步骤。

Costs.

A full cost-benefit analysis requires combining the prediction-access ratio with the (marginal) costs of improvements in capacity $C_{\text{Access}}$ and prediction $C_{\text{Pred}}$. Once we factor in costs, it is easy to decide what to focus on. A social planner should expand access whenever

成本。

完整成本收益分析需把预测–接入比与能力改进边际成本 $C_{\text{Access}}$、预测改进边际成本 $C_{\text{Pred}}$ 结合。纳入成本后，易于决定聚焦何处。社会规划者应在以下条件扩展接入

$$\displaystyle\frac{C_{\text{Access}}}{C_{\text{Pred}}}<\text{PAR}$$

and invest in better prediction otherwise. The (marginal) costs that competing policy levers carry are inherently context-dependent and will vary across application domains. In many applied settings the cost ratio is comparatively well understood, for example, the salary of an additional case-worker or the cost of a household survey. By presenting PAR separately from costs, we isolate the welfare side of the equation; domain experts can then plug in their own cost estimates to reach a policy decision. In particular, the PAR tells us how much we should be willing to pay for improvements in prediction versus expanding access.

否则投资更好预测。竞争政策杠杆的（边际）成本固有地依赖语境，并跨应用领域变化。许多应用设定中成本比相对清楚，例如额外个案工作者薪资或入户调查成本。把 PAR 与成本分开呈现，我们隔离方程的福利一侧；领域专家再代入自有成本估计做政策决定。特别地，PAR 告诉我们相对扩展接入，应为预测改进愿意支付多少。

We encourage future work to explore scenarios with more complex or less clearly defined cost structures. For instance, many practical applications involve recurring costs, such as ongoing staff salaries or periodic data collection, and fixed costs, such as initial investments in infrastructure or predictive model development. Analyzing how these cost structures affect welfare decisions over time, including amortization of fixed investments or identifying the point at which specific improvements become cost-effective, would significantly enhance our understanding of the relative value of prediction. To build intuition for the value of prediction in identifying the worst-off, we examine the prediction access ratio in one of the most basic statistical models. The outcomes $Y$ are Gaussian, and the learner has access to a predictor $f(x)=\hat{Y}$ such that the errors $Y-\hat{Y}$ are also Gaussian and independent of $\hat{Y}$. While extremely simple, the model yields surprisingly precise numerical insights that exactly match up in our real-world case study, where, of course, none of these assumptions hold. In this setting the quality of $\hat{Y}$ is fully summarized by the coefficient of determination $R^{2}=\operatorname{corr}(Y,\hat{Y})^{2}$.

我们鼓励未来工作探索成本结构更复杂或不甚清晰的场景。例如，许多实践应用涉及经常性成本（持续员工薪资或周期性数据采集）与固定成本（基础设施或预测模型开发的初始投资）。分析这些成本结构如何随时间影响福利决策——包括固定投资摊销或识别特定改进变得划算的时点——将显著增强对预测相对价值的理解。为建立识别最弱势群体时预测价值的直觉，我们在最基础统计模型之一中考察预测–接入比：结果 $Y$ 为高斯，学习者可及预测器 $f(x)=\hat{Y}$，误差 $Y-\hat{Y}$ 亦为高斯且独立于 $\hat{Y}$。虽极简，该模型给出惊人精确的数值洞见，并与真实案例研究精确吻合——而案例中这些假设当然都不成立。在该设定中，$\hat{Y}$ 的质量完全由决定系数 $R^{2}=\operatorname{corr}(Y,\hat{Y})^{2}$ 概括。

Our first result identifies when local improvements in prediction have the highest impact:

我们的第一个结果识别局部预测改进何时影响最大：

Theorem 1.1(Informal).

If $\alpha$ is at least a constant, the local improvements in $V$ with respect to $R^{2}$ diverge in two regimes: (1) $R^{2}\to 1$ and $\alpha=\beta$, or (2) $R^{2}\to 0$. In both cases, the prediction-access ratio satisfies $\textup{PAR}(\alpha,\beta)=0$.

定理 1.1（非正式）。

若 $\alpha$ 至少为常数，则 $V$ 对 $R^{2}$ 的局部改进在两体制下发散：(1) $R^{2}\to 1$ 且 $\alpha=\beta$，或 (2) $R^{2}\to 0$。两种情形下，预测–接入比满足 $\textup{PAR}(\alpha,\beta)=0$.

Predictions have the highest marginal impact at low and high $R^{2}$-values, making them a first- and last-mile effort. Our second result characterizes when the opposite is true. We prove that whenever screening capacities are severely limited relative to the size of the population one aims to identify $\alpha\ll\beta$ , the benefits of increasing $\alpha$ are overwhelming. Furthermore, it shows that the impacts of improving access are still relatively larger exactly in the regime where most current systems operate: $f$ explains $\approx 20\%$ of the variance and $\alpha$ is equal to, or even slightly larger, than $\beta$.

预测在低与高 $R^{2}$ 值处边际影响最高，使其成为第一与最后一英里努力。第二个结果刻画何时相反。我们证明：当筛查能力相对欲识别人群规模严重受限 $\alpha\ll\beta$ 时，增大 $\alpha$ 的收益压倒性。此外，它表明恰在多数当前系统运作的体制——$f$ 解释约 20% 方差且 $\alpha$ 等于甚至略大于 $\beta$——扩展接入的影响仍相对更大。

Theorem 1.2(Informal).

If the predictor $f$ explains an $R^{2}$ fraction of the variance, where $R^{2}$ is at least a constant, then the prediction access ratio is at least $\Omega(\alpha^{-1/(1-R^{2})})$. Furthermore, if $0.15\leqslant R^{2}\leqslant 0.85$ and $\alpha\leqslant\beta$ or $0.2\leqslant R^{2}\leqslant 0.5$, $\beta\geqslant 0.15$, and $\alpha\leqslant 0.5$ then the local prediction-access ratio is at least 1.

定理 1.2（非正式）。

若预测器 $f$ 解释 $R^{2}$ 比例的方差，且 $R^{2}$ 至少为常数，则预测–接入比至少为 $\Omega(\alpha^{-1/(1-R^{2})})$。此外，若 $0.15\leqslant R^{2}\leqslant 0.85$ 且 $\alpha\leqslant\beta$，或 $0.2\leqslant R^{2}\leqslant 0.5$、$\beta\geqslant 0.15$ 且 $\alpha\leqslant 0.5$，则局部预测–接入比至少为 1.

Empirical Results.

We complement our theoretical discussion by presenting a methodology for policymakers to evaluate the prediction-access ratio in practice. Using a real-world administrative dataset on hundreds of thousands of jobseekers in Germany, we show that our theoretical findings generalize to a more complex, real-world context that closely resembles algorithmic profiling systems widely implemented in many countries. Notably, our results reveal that when considering non-local improvements, expanding screening capacity has an even greater impact compared to enhancing prediction accuracy.

实证结果。

我们补充理论讨论，给出政策制定者在实践中评估预测–接入比的方法。利用覆盖数十万德国求职者的真实行政数据集，我们表明理论发现可推广到更复杂、更接近多国广泛部署的算法画像系统的真实语境。值得注意的是，在考虑非局部改进时，扩展筛查能力相对提升预测精度有更大影响。

1.2 Related Work

1.2 相关工作

Machine learning is increasingly used in the public sector to allocate support by predicting individuals at risk of adverse outcomes [Fischer-Abaigar et al., 2024], with applications spanning a wide range of problem domains [Desiere et al., 2019, Blumenstock, 2016, Perdomo et al., 2023, Chan et al., 2012, Potash et al., 2015, Chouldechova et al., 2018]. A large methodological literature draws on decision theory, operations research, economics, and machine learning to learn allocation rules from data [Elmachtoub and Grigas, 2022, Kitagawa and Tetenov, 2018, Manski, 2004, Fernández-Loría and Provost, 2022], with recent work in causal inference focusing on learning treatment policies from observational data [Athey and Wager, 2021, Kallus, 2021]. However, many decision-makers rely on separately trained predictive risk scoring-systems to solve “prediction policy problems” [Kleinberg et al., 2015]. Recently, this work has been extended using causal inference to train and evaluate these systems [Coston et al., 2023, Guerdan et al., 2023, Boehmer et al., 2024].

机器学习越来越多用于公共部门，通过预测不良结果风险个体来分配支持 [Fischer-Abaigar et al., 2024]，应用覆盖广泛问题域 [Desiere et al., 2019, Blumenstock, 2016, Perdomo et al., 2023, Chan et al., 2012, Potash et al., 2015, Chouldechova et al., 2018]。大量方法论文献从决策理论、运筹、经济学与机器学习从数据学习分配规则 [Elmachtoub and Grigas, 2022, Kitagawa and Tetenov, 2018, Manski, 2004, Fernández-Loría and Provost, 2022]，近期因果推断工作聚焦从观测数据学习处理策略 [Athey and Wager, 2021, Kallus, 2021]。但许多决策者依赖单独训练的预测风险评分系统来解决「预测政策问题」[Kleinberg et al., 2015]。近期该工作已用因果推断扩展以训练与评估这些系统 [Coston et al., 2023, Guerdan et al., 2023, Boehmer et al., 2024]。

The widespread use of risk-scoring systems has raised concerns regarding their tradeoffs, pitfalls, and validity [Wang et al., 2024, Coston et al., 2023, Fischer-Abaigar et al., 2024]. These concerns include not only questions of empirical performance but also of fairness and equity in how predictive systems shape access to public services [Barocas et al., 2023]. Recent work explores alternative design choices—such as employing aggregate rather than individual-level predictions [Shirali et al., 2024], balancing immediate needs with information-gathering [Wilder and Welle, 2024], and introducing randomization [Jain et al., 2024]—to improve downstream outcomes.

风险评分系统的广泛使用引发对其权衡、陷阱与有效性的关切 [Wang et al., 2024, Coston et al., 2023, Fischer-Abaigar et al., 2024]。这些关切不仅包括实证表现，也包括预测系统如何塑造公共服务接入中的公平与公正 [Barocas et al., 2023]。近期工作探索替代设计选择——如采用总体而非个体级预测 [Shirali et al., 2024]、平衡即时需求与信息收集 [Wilder and Welle, 2024]、引入随机化 [Jain et al., 2024]——以改进下游结果。

Perdomo [2024] studies the prediction-access ratio under both linear and probit models, with the latter closely related to our work. While they focus on binary welfare outcomes, we adopt a continuous welfare metric and a distinct policy objective: rather than evaluating changes in overall expected welfare, we measure the fraction of truly worst-off individuals who are identified. This captures a mathematically and conceptually distinct setting frequently encountered in the public sector. For instance, employment agencies often prioritize identifying and assisting individuals in greatest need, rather than optimizing average employment outcomes across all jobseekers. In addition, we introduce a set of empirical tools to analyze these tradeoffs in practice, while the work of Perdomo [2024] is purely theoretical.

Perdomo [2024] 在线性与 probit 模型下研究预测–接入比，后者与我们工作密切相关。他们聚焦二元福利结果，我们采用连续福利度量与不同政策目标：不是评估总体期望福利变化，而是度量真正最弱势群体中被识别的比例。这捕获公共部门常见的数学与概念上不同的设定。例如，就业机构往往优先识别并援助最需帮助者，而非优化全部求职者的平均就业结果。此外，我们引入一套实证工具在实践中分析这些权衡，而 Perdomo [2024] 纯属理论。

2 Formal Framework

2 形式框架

We start by formally defining our screening problem.

我们先形式定义筛查问题。

Definition 2.1(Screening Problem).

The screening problem seeks to identify a decision rule $\pi\colon\mathbb{R}\to\{0,1\}$ that maximizes the fraction of the worst-off individuals who are screened, subject to resource constraints $\alpha\in(0,1)$ that limit the proportion of the population the social planner can screen.

定义 2.1（筛查问题）。

筛查问题寻求识别决策规则 $\pi\colon\mathbb{R}\to\{0,1\}$，在资源约束 $\alpha\in(0,1)$（限制社会规划者可筛查的人口比例）下，最大化被筛查的最弱势群体比例。

$$\displaystyle\underset{\pi\colon\mathbb{R}\to\{0,1\}}{\max}\E\left[\pi(\hat{Y})=1\mid Y\leqslant F^{-1}_{Y}(\beta)\right]\ \text{s.t.}\ \E\left[\pi(\hat{Y})\right]\leqslant\alpha$$

The quantile $F^{-1}_{Y}(\beta)$ denotes the welfare cutoff that identifies the worst-off $\beta\in(0,1)$ fraction of the population.

分位数 $F^{-1}_{Y}(\beta)$ 表示识别人口中最弱势 $\beta\in(0,1)$ 比例的福利截止点。

Given perfect knowledge of the welfare outcomes $\hat{Y}=Y$, the optimal decision policy is simple: rank individuals based on their outcomes $Y$ and intervene in the bottom $\alpha$-fraction of the population. In the general case, we have:

若完美知道福利结果 $\hat{Y}=Y$，最优决策策略很简单：按结果 $Y$ 排序个体，干预人口底部 $\alpha$ 比例。一般情形下，有：

Proposition 1.

The optimal policy $\pi^{}\colon\mathbb{R}\to{0,1}$to solve the screening problem (Definition 2.1) is equal to$\pi^{}(\hat{Y}{i})=1{{s(\hat{Y}{i})\geqslant F^{-1}{s}(1-\alpha)}}$where$F^{-1}{s}(1-\alpha)$is the$(1-\alpha)$-quantile of$s(\hat{Y})=\mathrm{Pr}[Y\leqslant F^{-1}_{Y}(\beta)\mid\hat{Y}]$.

命题 1。

解决筛查问题（定义 2.1）的最优策略 $\pi^{}\colon\mathbb{R}\to{0,1}$等于$\pi^{}(\hat{Y}{i})=1{{s(\hat{Y}{i})\geqslant F^{-1}{s}(1-\alpha)}}$，其中$F^{-1}{s}(1-\alpha)$是$s(\hat{Y})=\mathrm{Pr}[Y\leqslant F^{-1}_{Y}(\beta)\mid\hat{Y}]$的$(1-\alpha)$-分位数。

Policy Value in Gaussian Setting.

For the theoretical investigation, we assume independent, identically distributed errors $\varepsilon=Y-\hat{Y}\overset{iid}{\sim}{\cal N}(0,\gamma^{2})$ that are independent of $\hat{Y}$. In this setting, the screening problem can be solved by ranking individuals in ascending order of their predicted outcomes $\hat{Y}$ and screening the bottom $\alpha$-fraction (see Proposition 4), achieving the policy value:

高斯设定下的策略价值。

为理论考察，我们假定独立同分布误差 $\varepsilon=Y-\hat{Y}\overset{iid}{\sim}{\cal N}(0,\gamma^{2})$ 且独立于 $\hat{Y}$。在此设定下，筛查问题可通过按预测结果 $\hat{Y}$ 升序排序并筛查底部 $\alpha$ 比例来解决（见命题 4），达到策略价值：

$$\displaystyle V(\pi^{*})=\mathrm{Pr}[\hat{Y}\leqslant F^{-1}_{\hat{Y}}(\alpha)\mid Y\leqslant F^{-1}_{Y}(\beta)] \tag{1}$$

In addition, we assume welfare outcomes $Y\sim{\cal N}(\mu,\eta^{2})$. Because the errors $\varepsilon$ are independent of $\hat{Y}$, this implies that $Y$ and $\hat{Y}$ follow a bivariate normal distribution.

此外，我们假定福利结果 $Y\sim{\cal N}(\mu,\eta^{2})$。因误差 $\varepsilon$ 独立于 $\hat{Y}$，这意味着 $Y$ 与 $\hat{Y}$ 服从二元正态分布。

Proposition 2.

(Policy Value in Gaussian Setting) Let $Y-\hat{Y}\overset{iid}{\sim}{\cal N}(0,\gamma^{2})$ and $Y\sim{\cal N}(\mu,\eta^{2})$, then the value $V(\pi^{})$of the optimal screening policy$\pi^{}$ is given by

命题 2。

（高斯设定下的策略价值）令 $Y-\hat{Y}\overset{iid}{\sim}{\cal N}(0,\gamma^{2})$ 且 $Y\sim{\cal N}(\mu,\eta^{2})$，则最优筛查策略 $\pi^{}$的价值$V(\pi^{})$ 由下式给出

$$\displaystyle V(\pi^{*})=V(\alpha,\beta,R^{2})=\frac{\Phi_{2}\left(\Phi^{-1}\left(\alpha\right),\Phi^{-1}\left(\beta\right);\rho\right)}{\beta} \tag{2}$$

where $\Phi_{2}\left(\cdot\right)$ denotes the bivariate standard normal CDF with correlation $\rho=\nicefrac{{\sqrt{\eta^{2}-\gamma^{2}}}}{{\eta}}$ and $\Phi^{-1}\left(\cdot\right)$ is the quantile function of the standard normal distribution.

其中 $\Phi_{2}\left(\cdot\right)$ 表示相关为 $\rho=\nicefrac{{\sqrt{\eta^{2}-\gamma^{2}}}}{{\eta}}$ 的二元标准正态 CDF，$\Phi^{-1}\left(\cdot\right)$ 是标准正态分位函数。

In this model, the goodness of the predictions $\hat{Y}$ are entirely captured by the coefficient of determination $R^{2}$, which equals the squared correlation $\rho^{2}$ between $Y$ and $\hat{Y}$.

在该模型中，预测 $\hat{Y}$ 的好坏完全由决定系数 $R^{2}$ 捕获，它等于 $Y$ 与 $\hat{Y}$ 之间相关系数的平方 $\rho^{2}$。

Our analysis extends to the log-normal distribution $\log Y\sim{\cal N}(\mu,\eta^{2})$ under a a multiplicative error model $Y=\hat{Y}\cdot u$ with $\log u\sim{\cal N}(0,\gamma^{2})$. Taking logarithms, leads to $\log Y=\log\hat{Y}+\log u$. Since the logarithm is strictly increasing, the ordering of $Y$ and $\hat{Y}$ is preserved under transformation. This allows us to apply the same framework to the log-transformed variables $\log Y$ and $\log\hat{Y}$. This extension is particularly useful because many welfare outcomes, such as income distributions [Clementi and Gallegati, 2005], can be approximated by a log-normal distribution.

我们的分析扩展到对数正态 $\log Y\sim{\cal N}(\mu,\eta^{2})$，在乘性误差模型 $Y=\hat{Y}\cdot u$、$\log u\sim{\cal N}(0,\gamma^{2})$ 下。取对数得 $\log Y=\log\hat{Y}+\log u$。因对数严格递增，$Y$ 与 $\hat{Y}$ 的排序在变换下保持。这使我们能对变换后的 $\log Y$ 与 $\log\hat{Y}$ 应用同一框架。该扩展特别有用，因为许多福利结果（如收入分布 [Clementi and Gallegati, 2005]）可近似为对数正态。

(a) Screening Probability

(a) 筛查概率

Visualization.

For a given screening capacity $\alpha$ and $R^{2}$ value, we can illustrate the corresponding screening policy by plotting the probability $\mathrm{Pr}\big\{\hat{Y}\leqslant F^{-1}_{\hat{Y}}(\alpha)\mid Y=y\big\}$ that an individual with welfare outcome $Y=y$ is screened. As shown in Figure 1, lower values of $Y$ correspond to higher probabilities of being screened. We focus on evaluating how effectively the screening policy identifies individuals in the worst-off segment of the population (i.e., on the left side of the $\beta$ cutoff).

可视化。

对给定筛查能力 $\alpha$ 与 $R^{2}$ 值，可通过绘制个体福利结果 $Y=y$ 被筛查的概率 $\mathrm{Pr}\big\{\hat{Y}\leqslant F^{-1}_{\hat{Y}}(\alpha)\mid Y=y\big\}$ 来说明对应筛查策略。如图 1，$Y$ 越低被筛查概率越高。我们聚焦评估筛查策略识别人口最弱势段（即 $\beta$ 截止点左侧）的有效程度。

3 Theoretical Results

3 理论结果

The decision-maker has (at least) two pathways to raise the policy value, which we refer to as policy levers:

决策者至少有两条路径提高策略价值，我们称之为政策杠杆：

• – Expanding Access Increasing the screening threshold from $\alpha$ to $\alpha+\Delta_{\alpha}$. If full screening were possible ($\alpha=1$), the $\beta$-fraction would be fully identified, as $V(\pi^{*})=\tfrac{\Phi_{2}\left(\Phi^{-1}\left(\alpha\right),\Phi^{-1}\left(\beta\right);\rho\right)}{\beta}=\tfrac{\Phi\left(\Phi^{-1}\left(\beta\right)\right)}{\beta}=1$.

• – 扩展接入 把筛查阈值从 $\alpha$ 增至 $\alpha+\Delta_{\alpha}$。若可完全筛查（$\alpha=1$），$\beta$ 比例会被完全识别，因为 $V(\pi^{*})=\tfrac{\Phi_{2}\left(\Phi^{-1}\left(\alpha\right),\Phi^{-1}\left(\beta\right);\rho\right)}{\beta}=\tfrac{\Phi\left(\Phi^{-1}\left(\beta\right)\right)}{\beta}=1$。

• – Improving Predictions Investing in better predictive models, modeled as increasing $R^{2}$ to $R^{2}+\Delta_{R^{2}}$. Perfect predictions ($R^{2}=1$) leads to optimal allocation of available capacities: $V(\pi^{*})=\tfrac{1}{\beta}\Phi\left(\text{min}\left(\Phi^{-1}\left(\alpha\right),{\Phi^{-1}\left(\beta\right)}\right)\right)$. If $\alpha\leqslant\beta$ then $V(\pi^{*})=\tfrac{\alpha}{\beta}$.

• – 改进预测 投资更好预测模型，建模为把 $R^{2}$ 增至 $R^{2}+\Delta_{R^{2}}$。完美预测（$R^{2}=1$）导致可用能力的最优分配：$V(\pi^{*})=\tfrac{1}{\beta}\Phi\left(\text{min}\left(\Phi^{-1}\left(\alpha\right),{\Phi^{-1}\left(\beta\right)}\right)\right)$。若 $\alpha\leqslant\beta$ 则 $V(\pi^{*})=\tfrac{\alpha}{\beta}$。

Figure 1 showcases improvements in access and prediction. Increasing capacity expands the fraction of the population screened, while improving $R^{2}$ shifts probability mass across the $\beta$ threshold, enhancing targeting accuracy.

图 1 展示接入与预测的改进。增大能力扩展被筛查人口比例，而改进 $R^{2}$ 使概率质量跨过 $\beta$ 阈值移动，提升瞄准精度。

Following Perdomo [2024], a key quantity of interest is the prediction-access ratio (PAR), which quantifies the relative improvements in policy value from enhancing predictions versus improving access to screening. Specifically, the PAR is defined as:

遵循 Perdomo [2024]，关键兴趣量是预测–接入比（PAR），量化相对改进预测与改进筛查接入时策略价值的相对提升。具体地，PAR 定义为：

$$\displaystyle\textup{PAR}=\frac{V(\alpha+\Delta_{\alpha},\beta,R^{2})-V(\alpha,\beta,R^{2})}{V(\alpha,\beta,R^{2}+\Delta_{R^{2}})-V(\alpha,\beta,R^{2})} \tag{3}$$

In other words, the PAR can inform a social planner how much more they should be willing to pay for improvements in screening capacity relative to prediction. For example, a PAR $>2$ implies that expanding the screening capacity by $\Delta_{\alpha}$ yields at least twice the increase in policy value compared to investing in improved predictions by $\Delta_{R^{2}}$. Consequently, the social planner should prioritize investments in screening capacity, provided the costs of doing so are not more than double those of improving predictions.

换言之，PAR 可告知社会规划者：相对预测，他们应为筛查能力改进愿意多付多少。例如，PAR $>2$ 意味着扩展筛查能力 $\Delta_{\alpha}$ 带来的策略价值增幅至少是投资改进预测 $\Delta_{R^{2}}$ 的两倍。因此，只要扩展筛查的成本不超过改进预测成本的两倍，规划者应优先投资筛查能力。

3.1 Theoretical Bounds for the Prediction-Access Ratio

3.1 预测–接入比的理论界

In our setting, direct calculation of the PAR is challenging due to the policy value being analytically intractable and the problem featuring strong non-linearities. We derive bounds for specific cases and regimes that we consider particularly insightful, with a focus on marginal local improvements. In our empirical investigation, we find that the main results generalize well to a more complex, real-world setting.

在我们的设定中，因策略价值解析难解且问题有强非线性，直接计算 PAR 具有挑战。我们为特别有洞见的特定情形与体制推导界，聚焦边际局部改进。在实证考察中，我们发现主要结果可良好推广到更复杂的真实设定。

Theorem 3.1(PAR for Small Screening Capacities).

For any $0<R^{2}<1$, $\Delta_{R^{2}},\Delta_{\alpha}>0$ and $0<\beta\leqslant 0.5$ there exists a threshold $t(\beta,R^{2},\Delta_{R^{2}})$ such that for any $\alpha+\Delta_{\alpha}\leqslant t$, $\textup{PAR}(\alpha,R^{2},\Delta_{\alpha},\Delta_{R^{2}})$ is at least

定理 3.1（小筛查能力下的 PAR）。

对任意 $0<R^{2}<1$，$\Delta_{R^{2}},\Delta_{\alpha}>0$ 与 $0<\beta\leqslant 0.5$，存在阈值 $t(\beta,R^{2},\Delta_{R^{2}})$，使得对任意 $\alpha+\Delta_{\alpha}\leqslant t$，$\textup{PAR}(\alpha,R^{2},\Delta_{\alpha},\Delta_{R^{2}})$ 至少为

$$\displaystyle\frac{\Delta_{\alpha}}{\Delta_{R^{2}}}\sqrt{R^{2}(1-R^{2})}\left(5.1\cdot\alpha\Phi^{-1}\left(1-\alpha\right)\right)^{-\frac{1}{1-R^{2}}+o(1)}$$

where $o(1)$ goes to zero as $\alpha$ approaches zero.

其中当 $\alpha$ 趋于零时 $o(1)$ 趋于零。

Suppose the available screening capacity $\alpha+\Delta_{\alpha}$ is very small ($\alpha+\Delta_{\alpha}\ll\beta$), and assume there is a baseline level of predictability (i.e., $R^{2}$ is bounded away from $0$). Then Theorem 3.1 implies that the PAR can become very large. Specifically, for small $\alpha$, $\Phi^{-1}\left(1-\alpha\right)$ grows asymptotically like $\sqrt{\log{(1/\alpha})}$. Consequently, the polynomial growth of $\alpha^{-1/(1-R^{2})}$ drives the PAR to increase rapidly as $\alpha$ decreases. It follows that in the scarce capacity regime, expanding the screening capacity has a far greater impact than improvements in prediction accuracy.

假定可用筛查能力 $\alpha+\Delta_{\alpha}$ 很小（$\alpha+\Delta_{\alpha}\ll\beta$），且存在基线可预测性（即 $R^{2}$ 远离 0）。则定理 3.1 意味着 PAR 可变得很大。具体地，对小 $\alpha$，$\Phi^{-1}\left(1-\alpha\right)$ 渐近像 $\sqrt{\log{(1/\alpha})}$ 增长。因而 $\alpha^{-1/(1-R^{2})}$ 的多项式增长驱动 PAR 随 $\alpha$ 减小而快速增大。于是在稀缺能力体制下，扩展筛查能力远比改进预测精度影响更大。

Theorem 3.2(Maximally Effective (Local) Prediction Improvements).

Let $0<\beta<1$ be fixed and $0<\alpha<1$. Consider the points that maximize the local rate of change in policy value $V$ with respect to improvements in $R^{2}$:

定理 3.2（最有效的（局部）预测改进）。

令 $0<\beta<1$ 固定且 $0<\alpha<1$。考虑使策略价值 $V$ 对 $R^{2}$ 改进的局部变化率最大的点：

$$\displaystyle(\alpha_{*},R^{2}_{*})=\underset{(\alpha,R^{2})\in(0,1)\times(0,1)}{\argmax}\lim_{\Delta\rightarrow 0}\frac{V(\alpha,\beta,R^{2}+\Delta)-V(\alpha,\beta,R^{2})}{\Delta}$$

The local improvements in $V$ diverge — and are maximized — in two regimes: (1) $R^{2}_{}\to 1$,$\alpha_{}=\beta$, and (2)$R^{2}_{}\to 0$. For both regimes, setting$\Delta_{R^{2}}=\Delta_{\alpha}=\Delta$, the local prediction-access ratio satisfies$\lim_{\Delta\to 0}\textup{PAR}(\alpha,\beta,\Delta)\rightarrow 0$.*

$V$ 的局部改进在两体制下发散——并达到最大：(1) $R^{2}_{}\to 1$，$\alpha_{}=\beta$，以及 (2)$R^{2}_{}\to 0$。对两体制，取$\Delta_{R^{2}}=\Delta_{\alpha}=\Delta$，局部预测–接入比满足$\lim_{\Delta\to 0}\textup{PAR}(\alpha,\beta,\Delta)\rightarrow 0$.*

According to Theorem 3.2, marginal improvements in prediction are most impactful in two distinct regimes. First, when predictive capacity is very low, even a small initial investment can lead to disproportionately large improvements, provided that a minimal baseline of screening capacity is present. Second, as $R^{2}$ approaches one, further marginal improvements can also have a significant relative impact, specifically around the point where the screening capacity $\alpha$ matches the requirements for screening the entire $\beta$-segment of the population. See Figure 2.

据定理 3.2，预测的边际改进在两个不同体制最有影响。第一，当预测能力很低时，只要有最低基线筛查能力，即便小额初始投资也可带来不成比例的大改进。第二，当 $R^{2}$ 接近一时，进一步边际改进也可有显著相对影响，特别是在筛查能力 $\alpha$ 恰好匹配筛查整个 $\beta$ 段人口需求的附近。见图 2。

Proposition 3(PAR for Local Improvements).

Let $R^{2}$, $\beta$, and $\alpha$ satisfy either $R^{2}\in(0.15,0.85)$, $\beta\in(0.03,0.5)$, and $\alpha\leqslant\beta$, or $R^{2}\in(0.2,0.5)$, $\beta\geqslant 0.15$, and $\alpha\leqslant 0.5$. If $\Delta_{R^{2}}=\Delta_{\alpha}=\Delta$, then $\lim_{\Delta\to 0}\textup{PAR}(\alpha,\beta,\Delta)\geqslant 1$.

命题 3（局部改进的 PAR）。

令 $R^{2}$、$\beta$ 与 $\alpha$ 满足 $R^{2}\in(0.15,0.85)$、$\beta\in(0.03,0.5)$ 且 $\alpha\leqslant\beta$，或 $R^{2}\in(0.2,0.5)$、$\beta\geqslant 0.15$ 且 $\alpha\leqslant 0.5$。若 $\Delta_{R^{2}}=\Delta_{\alpha}=\Delta$，则 $\lim_{\Delta\to 0}\textup{PAR}(\alpha,\beta,\Delta)\geqslant 1$.

We find that the PAR remains above one as long as $\alpha\leqslant\beta$ and $R^{2}$ is not too extreme. For larger $\beta$ values (i.e., $\beta\geqslant 0.15$) the PAR stays above one even for large $\alpha$ provided $R^{2}$ remains in a moderate range. Crucially, this represents the standard parameter regime in which most allocation programs operate, characterized by a moderate baseline of predictions and resource levels comparable to $\beta$.

我们发现：只要 $\alpha\leqslant\beta$ 且 $R^{2}$ 不太极端，PAR 保持大于一。对更大 $\beta$（即 $\beta\geqslant 0.15$），即便 $\alpha$ 较大，只要 $R^{2}$ 保持中等，PAR 仍高于一。关键的是，这代表多数分配项目运作的标准参数体制：中等基线预测，资源水平与 $\beta$ 可比。

Numerical Simulations.

We complement our theoretical investigation with numerical simulations of the PAR for different $\alpha$, $\beta$ and $R^{2}$ values (see Figure 2). Consistent with our theoretical results, the PAR becomes large for small screening capacities ($\alpha\ll\beta$) and remains above one for $\alpha\leqslant\beta$, provided a small baseline level of predictive performance has been established. The bounds in Proposition 3 are conservative, with PAR $>1$ observed for a broad range of $R^{2}$ values. Prediction improvements are particularly impactful when $R^{2}$ is small. Although the PAR falls below one in the high-$R^{2}$ and high-$\alpha$ regime, allocation is nearly perfect, making further improvements a “last mile” effort.

数值模拟。

我们用不同 $\alpha$、$\beta$ 与 $R^{2}$ 值的 PAR 数值模拟补充理论考察（见图 2）。与理论结果一致：对小筛查能力（$\alpha\ll\beta$）PAR 变大；在建立小基线预测表现后，对 $\alpha\leqslant\beta$ PAR 保持高于一。命题 3 的界偏保守，在广泛 $R^{2}$ 值上观察到 PAR $>1$。当 $R^{2}$ 很小时预测改进特别有影响。虽在高 $R^{2}$ 与高 $\alpha$ 体制 PAR 低于一，但分配已近乎完美，进一步改进成为「最后一英里」努力。

(a) PAR

Discussion.

We found several insights relevant to policymakers aiming to iteratively improve a screening system. First, establishing a baseline level of predictive performance is usually a good starting point. Once this is achieved, expanding the screening capacity becomes the next priority. For very small capacities, Theorem 3.1 tell us that the PAR can increase significantly, making investments in screening capacity highly impactful.

讨论。

我们发现若干对旨在迭代改进筛查系统的政策制定者相关的洞见。第一，建立基线级预测表现通常是好起点。一旦达成，扩展筛查能力成为下一优先。对很小能力，定理 3.1 告诉我们 PAR 可显著增大，使筛查能力投资极有影响。

Generally, expanding capacity to at least the level where everyone in need could hypothetically be screened ($\alpha\geqslant\beta$) is likely cost-efficient. Once both screening capacity and predictive accuracy are high and the allocation system is close to optimal, improvements in prediction become relatively more valuable again for perfecting the system. However, this regime may rarely be reached in practice.

一般而言，把能力至少扩展到理论上可筛查所有需助者的水平（$\alpha\geqslant\beta$）可能划算。一旦筛查能力与预测精度都高、分配系统接近最优，预测改进对完善系统再次相对更有价值。但该体制在实践中可能很少达到。

In Figure 2, we display the PAR for a cost ratio of $1/4$. As expected, the regions where investing in $R^{2}$ is more efficient expand, and some of the earlier nonquantitative bounds no longer apply. Nevertheless, the key insights remain consistent: when screening capacities are small, investments in expanding them are very effective, while improvements in $R^{2}$ are more important when predictive accuracy is low.

在图 2 中，我们展示成本比为 $1/4$ 时的 PAR。如预期，投资 $R^{2}$ 更高效的区域扩大，部分先前非定量界不再适用。然而关键洞见仍一致：筛查能力小时，扩展它们的投资非常有效；而 $R^{2}$ 改进在预测精度低时更重要。

4 Empirically Evaluating the PAR

4 实证评估 PAR

While our theory offers broad intuition when expanding screening capacity or improving predictions is most effective, policymakers need practical tools for their own systems. To support this, we develop a methodology to compute and interpret the prediction-access ratio, helping social planners identify the most efficient policy levers for their unique problem context.

虽理论给出何时扩展筛查能力或改进预测最有效的广泛直觉，政策制定者需要面向自有系统的实用工具。为此我们开发计算并解释预测–接入比的方法，帮助社会规划者为其独特问题语境识别最高效政策杠杆。

Policy Value.

As before, we define the allocation policy’s value as the probability that the worst-off individuals are successfully identified:, i.e. $V(\alpha,\beta)=\mathrm{Pr}[\hat{Y}\leqslant F^{-1}_{\hat{Y}}(\alpha)\mid Y\leqslant F^{-1}_{Y}(\beta)]$. In practice, this can be measured using a recall-like metric, capturing the proportion of truly at-risk individuals screened by the policy.

策略价值。

如前，我们把分配策略价值定义为最弱势群体被成功识别的概率，即 $V(\alpha,\beta)=\mathrm{Pr}[\hat{Y}\leqslant F^{-1}_{\hat{Y}}(\alpha)\mid Y\leqslant F^{-1}_{Y}(\beta)]$。实践中可用类似召回的指标度量，捕获策略筛查到的真正风险个体比例。

$$\displaystyle V(\alpha,\beta)\approx\frac{\sumop\displaylimits_{i=1}^{n}1\{\hat{Y}_{i}\leqslant F^{-1}_{\hat{Y},n}(\alpha)\}1\{Y_{i}\leqslant F^{-1}_{Y,n}(\beta)\}}{\sumop\displaylimits_{i=1}^{n}1\{Y_{i}\leqslant F^{-1}_{Y,n}(\beta)\}}$$

Increasing Screening Capacity. Given a chosen $\Delta_{\alpha}$ the policy improvement can be directly computed $V(\alpha+\Delta_{\alpha},\beta)-V(\alpha,\beta)$ by recalculating the empirical policy value at the new threshold. For example, in cash transfer programs [Blumenstock, 2016], a key question is how many resources $\alpha^{*}$ are required to reach a specified fraction $p$ of poor households, i.e. $\alpha^{*}=\inf_{\alpha\in(0,1)}\{\alpha\colon V(\alpha,\beta)\geqslant p\}$.

增大筛查能力。 给定所选 $\Delta_{\alpha}$，可通过在新阈值重算经验策略价值直接计算策略改进 $V(\alpha+\Delta_{\alpha},\beta)-V(\alpha,\beta)$。例如在现金转移项目中 [Blumenstock, 2016]，关键问题是需要多少资源 $\alpha^{*}$ 才能触及指定比例 $p$ 的贫困家庭，即 $\alpha^{*}=\inf_{\alpha\in(0,1)}\{\alpha\colon V(\alpha,\beta)\geqslant p\}$。

Improving Predictions.

A decision-maker can improve a model’s predictions through various pathways:

改进预测。

决策者可通过多种路径改进模型预测：

a) Data Collection Collect additional samples and increase the frequency of data collection. Social prediction systems are often vulnerable to distribution shifts over time in dynamic and evolving environments [Fischer-Abaigar et al., 2024, Aiken et al., 2023].

a) 数据采集 收集更多样本并提高采集频率。社会预测系统在动态演变环境中常易受随时间分布漂移影响 [Fischer-Abaigar et al., 2024, Aiken et al., 2023]。

b) Data Quality Improve data quality (i.e., reduce errors and missing data) by means such as standardizing data collection processes, implementing centralized data management systems, and offering targeted training programs for staff.

b) 数据质量 通过标准化采集流程、实施集中数据管理系统、为员工提供针对性培训等手段提升数据质量（即减少错误与缺失）。

c) Collect Additional Features In government, this may involve integrating separate data sources across institutions [Sun and Medaglia, 2019, Wirtz et al., 2019].

c) 采集额外特征 在政府中，这可能涉及跨机构整合分离数据源 [Sun and Medaglia, 2019, Wirtz et al., 2019]。

d) Advanced Modeling Techniques Utilize more sophisticated modeling techniques, which might capture more complex patterns in the data but are often more costly to operationalize.

d) 先进建模技术 利用更复杂建模技术，可能捕获数据中更复杂模式，但往往更难运营落地。

In resource-constrained settings, planners often focus on incremental improvements rather than rebuilding entire systems. For instance, collecting a small amount of additional data may boost $R^{2}$ by a few points, uniformly reducing errors. To simulate such minor gains, we scale the model’s residuals $\hat{Y}_{+}=\hat{Y}+\delta(Y-\hat{Y})$, choosing $\delta\in(0,1)$ so that $R^{2}$ increases by a target $\Delta_{R^{2}}$ (see Appendix B.3). This preserves the overall error structure, allowing us to gauge how a “similar but slightly better” model affects policy outcomes.

在资源受限设定中，规划者往往聚焦增量改进而非重建整套系统。例如，少量额外数据可能把 $R^{2}$ 抬高几个点，均匀降低误差。为模拟此类小增益，我们缩放模型残差 $\hat{Y}_{+}=\hat{Y}+\delta(Y-\hat{Y})$，选取 $\delta\in(0,1)$ 使 $R^{2}$ 增加目标 $\Delta_{R^{2}}$（见附录 B.3）。这保持总体误差结构，使我们能衡量「相似但略更好」的模型如何影响策略结果。

This approach can be extended in several ways. For example, residuals could be adjusted for specific subgroups to account for uneven prediction improvements (e.g., targeted data collection for rural or underrepresented populations). Alternatively, planners could retrain models under different conditions — such as sample size, feature set, or architecture — and compare the resulting policy value.

该方法可多方面扩展。例如，可对特定子群调整残差以计入不均衡的预测改进（如针对农村或代表性不足人群的定向数据采集）。或者，规划者可在样本量、特征集或架构等不同条件下重训模型，并比较所得策略价值。

5 Case Study: Identifying Long-Term Unemployment in Germany

5 案例研究：识别德国长期失业

Public employment services (PES) across the globe make use of profiling approaches to identify jobseekers at risk of long-term unemployment to target preventative measures [Loxha and Morgandi, 2014]. Starting from traditional rule-based approaches, many PES either test or already deploy algorithmic profiling to identify jobseekers in need of support [Desiere et al., 2019, Körtner and Bonoli, 2023]. While these profiling tools assist in allocating programs that account for large shares of PES spending — making design choices critical [Kern et al., 2024] — systematic assessments of their relative value compared to other measures for improving jobseekers’ outcomes remain absent.

全球公共就业服务（PES）使用画像方法识别有长期失业风险的求职者，以瞄准预防措施 [Loxha and Morgandi, 2014]。从传统基于规则的方法出发，许多 PES 或在测试或已部署算法画像以识别需支持的求职者 [Desiere et al., 2019, Körtner and Bonoli, 2023]。虽这些画像工具协助分配占 PES 支出大头的项目——使设计选择至关重要 [Kern et al., 2024]——但相对其他改进求职者结果的措施，对其相对价值的系统评估仍缺。

Figure 3: Unemployment duration The red line marks the 12 month threshold used to classify a jobseeking episode as long-term unemployment (LTU) in Germany.

图 3：失业时长。红线标出德国将求职期归类为长期失业（LTU）所用的 12 个月阈值。

We secured access to a dataset[^1] on German jobseekers derived from German administrative labor market records that cover a large portion of the German labor force. It covers a period from 1975 to 2017 and merges multiple administrative data sources, containing a wide spectrum of individual labor market information — including records on employment histories, received benefits, unemployment periods, participation in job training programs and demographic information. Such administrative records are the primary data source used by PES to build algorithmic profiling models [Bach et al., 2023].

我们获准使用来自德国行政劳动力市场记录、覆盖德国劳动力很大一部分的求职者数据集[^1]。覆盖 1975 至 2017 年，合并多源行政数据，含广泛个体劳动力市场信息——包括就业史、领取福利、失业期、参加就业培训与人口统计信息。此类行政记录是 PES 构建算法画像模型的主要数据源 [Bach et al., 2023]。

Experimental Setup.

We train a model to predict how long a newly registered jobseeker remains unemployed, defining the target $Y$ as unemployment duration in days (capped at $24$ months)[^2]. Following Bach et al. [2023], we use a set of covariates capturing demographic information, labor market history, and most recent job details. We focus on unemployment spells beginning between 2010 and 2015, resulting in data on 274,515 different jobseekers and 553,980 unemployment spells (see Figure 4).

实验设置。

我们训练模型预测新登记求职者保持失业多久，将目标 $Y$ 定义为失业天数（封顶 $24$ 个月）[^2]。遵循 Bach et al. [2023]，使用捕获人口统计、劳动力市场史与最近工作细节的协变量集。聚焦 2010 至 2015 年开始的失业期，得到 274,515 名不同求职者与 553,980 段失业期（见图 4）。

To avoid the impact of significant labor market reforms in Germany and to ensure full observation of unemployment durations up to 24 months, we restrict our analysis to unemployment episodes that began between 2010 and 2015. We use records from 2010 and 2011 to build the training dataset, records from 2012 for validation, and evaluate test performance on data from 2015 (see Figure 4). We left a gap between the training and test data periods to allow enough time for the outcomes in the training data to have been fully observed at test time, in order to mimic a realistic deployment scenario starting at the beginning of 2015. We refer to Appendix B.1 for additional information on the experimental setup and data.

为避免德国重大劳动力市场改革影响，并确保可完整观察至 24 个月的失业时长，我们把分析限制在 2010 至 2015 年开始的失业事件。用 2010 与 2011 年记录建训练数据，2012 年做验证，在 2015 年数据上评估测试表现（见图 4）。训练与测试期之间留间隙，以便测试时训练数据中的结果已充分被观察，模拟 2015 年初开始的真实部署场景。实验设置与数据更多信息见附录 B.1。

Figure 4: Stacked timeline diagram illustrating training (2010–2013), validation (2012–2014), and test (2015–2017) data periods. Red dashed boundaries within each colored box indicate the possible start dates of unemployment episodes, while the full colored boxes represent the entire observation phases for each dataset.

图 4：堆叠时间线图示，说明训练（2010–2013）、验证（2012–2014）与测试（2015–2017）数据时段。各色块内红色虚线边界表示失业事件可能开始日期，完整色块表示各数据集的整个观察期。

Guiding Questions

Our focus is the $\beta$-fraction of jobseekers with the longest expected unemployment durations, representing those most at risk. In Germany, being unemployed for over one year (about $15\%$ of cases in our data; see Figure 3) meets the legal definition of long-term unemployment [Bach et al., 2023], but some countries adopt different cutoffs [Desiere et al., 2019]. Taking the perspective of a social planner designing a profiling system in a public employment office, our analysis aims to answer the following questions:

引导问题

我们聚焦预期失业时长最长的 $\beta$ 比例求职者，代表风险最高者。在德国，失业超过一年（数据中约 $15\%$ 的情形；见图 3）符合长期失业的法定定义 [Bach et al., 2023]，但一些国家采用不同截止点 [Desiere et al., 2019]。站在公共就业办公室设计画像系统的社会规划者视角，我们的分析旨在回答以下问题：

• – How much does the screening capacity[^3] need to increase to ensure a significant portion of the high-risk jobseekers are screened, given the inaccuracies in the prediction system?

• – 在预测系统有误差的前提下，筛查能力[^3]需增大多少，才能确保相当比例的高风险求职者被筛查？

• – What is the real-world impact of improving screening capacity versus prediction errors?

• – 改进筛查能力相对预测误差的真实世界影响是什么？

• – When do small improvements in prediction error have the largest impact?

• – 预测误差的小改进何时影响最大？

• – What are the relative benefits and trade-offs of using a simpler vs more complex prediction model?

• – 使用更简单相对更复杂预测模型的相对收益与权衡是什么？

5.1 Results

5.1 结果

We train a CatBoost model (see Appendix B.2 for details), achieving an $R^{2}$ of $0.15$ on the test set. This level of predictive power aligns well with what is typically observed in social prediction tasks [Salganik et al., 2020] and similar applied settings [Desiere et al., 2019].

我们训练 CatBoost 模型（细节见附录 B.2），在测试集上达到 $R^{2}$ 为 $0.15$。该预测能力水平与社会预测任务中通常观察结果良好对齐 [Salganik et al., 2020]，也与类似应用设定一致 [Desiere et al., 2019]。

How much does the screening capacity need to increase to target a significant fraction of high-risk jobseekers?

As expected, larger screening capacities increase both the policy value and the number of high-risk jobseekers screened (see Figure 5). Focusing on the (German) LTU cutoff ($\beta\approx 0.15$), our policy value aligns well with findings of previous studies[^4] [Bach et al., 2023].

筛查能力需增大多少才能瞄准显著比例的高风险求职者？

如预期，更大筛查能力同时提高策略价值与被筛查的高风险求职者人数（见图 5）。聚焦（德国）LTU 截止点（$\beta\approx 0.15$），我们的策略价值与既往研究发现良好对齐[^4] [Bach et al., 2023]。

Figure 5: Policy value across different screening capacities ($\alpha$) and worst-off fractions $\beta$ evaluated on the test set using the CatBoost regression model. A $\beta$ value of $0.15$ corresponds to the $12$-month cutoff used to define long-term unemployment in Germany.

图 5：用 CatBoost 回归模型在测试集上评估的、跨不同筛查能力（$\alpha$）与最弱势比例 $\beta$ 的策略价值。$\beta$ 取值 $0.15$ 对应德国定义长期失业的 $12$ 个月截止点。

A planner might begin by setting $\alpha=\beta$, ensuring that, in theory, enough capacity is provided to screen and support every high-risk jobseeker. A natural question then arises: how much additional capacity $\Delta_{\alpha}$ would be required to screen at least a specified percentage of high-risk individuals? This additional capacity represents the overhead that must be invested to account for imperfect predictions. We observe that the $\Delta_{\alpha}$ required to ensure at least 75% of high-risk jobseekers are screened remains consistently around $0.25$ across different $\beta$ values. While the policy value increases as $\alpha=\beta$ rises, the marginal improvements gained from increasing access decrease for higher $\alpha$, resulting in a somewhat stable $\Delta_{\alpha}$ across $\beta$. In practice, this means we need to screen $25\%$ more of the population to ensure adequate coverage.

规划者可能先设 $\alpha=\beta$，理论上确保有足够能力筛查并支持每位高风险求职者。自然问题随之而来：为至少筛查指定百分比的高风险个体，还需多少额外能力 $\Delta_{\alpha}$？该额外能力代表为计入不完美预测必须投入的开销。我们观察到：为确保至少 75% 高风险求职者被筛查所需的 $\Delta_{\alpha}$ 跨不同 $\beta$ 值大致稳定在 $0.25$ 左右。虽策略价值随 $\alpha=\beta$ 上升而增大，但更高 $\alpha$ 下增大接入的边际改进下降，使跨 $\beta$ 的 $\Delta_{\alpha}$ 相对稳定。实践中这意味着需多筛查人口的 $25\%$ 才能确保充分覆盖。

(a) Constant Prediction ($R^{2}=0$)

(a) 常数预测（$R^{2}=0$）

What is the impact of improving screening capacity versus prediction errors?

We simulate small improvements in the $R^{2}$ value by uniformly scaling the residuals by a multiplicative factor.

改进筛查能力相对预测误差的影响是什么？

我们通过用乘性因子均匀缩放残差来模拟 $R^{2}$ 值的小改进。

To ensure that this approach approximates a realistic pathway of (marginally) improving the model, we train various models at different sample sizes. We then verify that as $R^{2}$ increases with the amount of training data, the variance of the residuals decreases, while the distribution remains largely unchanged in shape (see Figure 13). We then evaluate the prediction-access ratio for $\Delta_{R^{2}}=\Delta_{\alpha}=0.1$ in three scenarios : (1) the trained CatBoost model with $R^{2}=0.15$, (2) near-perfect predictions with $R^{2}=1-\Delta_{R^{2}}$ and (3) constant predictions ($R^{2}=0$), effectively randomizing screening decisions.

为确保该方法近似（边际）改进模型的现实路径，我们在不同样本量上训练多种模型。随后验证：随着训练数据量增加，$R^{2}$ 上升时残差方差下降，而分布形状大体不变（见图 13）。接着在三种情形下评估 $\Delta_{R^{2}}=\Delta_{\alpha}=0.1$ 的预测–接入比：(1) 训练得到的 CatBoost 模型，$R^{2}=0.15$；(2) 近乎完美预测，$R^{2}=1-\Delta_{R^{2}}$；(3) 常数预测（$R^{2}=0$），实质上随机化筛查决策。

We observe a rise in the PAR for small screening capacities $\alpha$ (see Figure 6), consistent with Theorem 3.1. Under random allocation ($R^{2}\!=\!0$), the PAR stays below one for $\alpha\geqslant 0.1$. This result aligns somewhat with Theorem 3.2, where we found that the (local) PAR approaches zero as $R^{2}\to 0$. Because we consider $\Delta=0.1$ (rather than an infinitesimal improvement, see Figure 14 for $\Delta=0.01$), the PAR remains large at small $\alpha$. For the CatBoost model ($R^{2}\!=\!0.15$), capacity improvements stay relatively more effective (i.e., PAR $>\!1$) for larger $\alpha$, matching Proposition 3, where we found that for moderate $R^{2}$ and $\alpha\leqslant\beta$, the local PAR remains above one. Meanwhile, near-perfect predictions ($R^{2}=0.9$) make capacity investments highly efficient, causing the PAR to diverge for $\alpha<\beta$, then drop sharply near $\alpha=\beta$ because the allocation becomes nearly optimal. When $\alpha\geqslant\beta$, the PAR stabilizes at one as numerator and denominator both approach zero.

我们观察到小筛查能力 $\alpha$ 下 PAR 上升（见图 6），与定理 3.1 一致。在随机分配（$R^{2}=0$）下，对 $\alpha\geqslant 0.1$，PAR 保持低于一。这与定理 3.2 部分对齐——我们曾发现当 $R^{2}\to 0$ 时（局部）PAR 趋于零。因我们取 $\Delta=0.1$（而非无穷小改进，见图 14 中 $\Delta=0.01$），小 $\alpha$ 处 PAR 仍较大。对 CatBoost 模型（$R^{2}=0.15$），在更大 $\alpha$ 下能力改进仍相对更有效（即 PAR $>1$），与命题 3 吻合——对中等 $R^{2}$ 且 $\alpha\leqslant\beta$，局部 PAR 保持高于一。同时，近乎完美预测（$R^{2}=0.9$）使能力投资高度高效，导致 $\alpha<\beta$ 时 PAR 发散，随后在 $\alpha=\beta$ 附近急剧下降，因为分配已近乎最优。当 $\alpha\geqslant\beta$ 时，分子分母都趋于零，PAR 稳定在一。

These observations broadly match our theoretical findings, despite the non-local improvements and more complex residual structure. Notably, the theory’s focus on local improvements offers a conservative perspective on capacity investments: even under random allocation ($R^{2}=0$), securing a modest screening capacity ($5\!-\!10\%$) is often the first priority, while at very high $R^{2}$, gains in policy value diminish so rapidly once $\alpha\geqslant\beta$ that the relative advantage of further prediction investments becomes negligible.

这些观察与理论发现大体吻合，尽管改进非局部、残差结构更复杂。值得注意的是，理论聚焦局部改进，对能力投资给出偏保守视角：即便在随机分配（$R^{2}=0$）下，先确保适度筛查能力（$5\!-\!10\%$）往往是第一优先；而在很高 $R^{2}$ 下，一旦 $\alpha\geqslant\beta$，策略价值增益迅速衰减，进一步预测投资的相对优势变得可忽略。

When do small improvements in prediction error have the largest impact?

From theory (Theorem 3.2), we expect local policy value improvements from better predictions to diverge as $R^{2}\!\rightarrow 0$ and $R^{2}\rightarrow 1$ when $\alpha=\beta$. This aligns with our results in Figure 7: for small $\Delta_{R^{2}}$, the rate of local improvements in $V(R^{2})$ with respect to $R^{2}$ diverges. The location of the maximum in $\alpha$ also follows from the theory: as $R^{2}\rightarrow 1$, the rate only diverges for $\alpha=\beta$, while for small $R^{2}$ the maximum is at $\alpha\approx 0.5$.

预测误差的小改进何时影响最大？

由理论（定理 3.2），我们期望更好预测带来的局部策略价值改进在 $R^{2}\!\rightarrow 0$ 以及当 $\alpha=\beta$ 时 $R^{2}\rightarrow 1$ 处发散。这与图 7 结果一致：对小 $\Delta_{R^{2}}$，$V(R^{2})$ 对 $R^{2}$ 的局部改进率发散。$\alpha$ 上最大值的位置也遵循理论：当 $R^{2}\rightarrow 1$ 时，该率仅在 $\alpha=\beta$ 发散；而对小 $R^{2}$，最大值约在 $\alpha\approx 0.5$。

(a) $R^{2}=0$

What are the relative benefits and trade-offs of using a simpler vs more complex prediction model?

We compare a shallow 4-depth decision tree with the CatBoost model. As expected, the simpler tree shows a small drop in predictive power ($5\%$ decrease in $R^{2}$) which translates into a $1$–$8\%$ reduction in policy value (see Figure 8(a)). Compared to a uniform $5\%$ increase in $R^{2}$ achieved by scaling the residuals (see Figure 15), the differences in policy value are only partially similar across $\alpha$. The CatBoost model does not provide a uniform improvement over the decision tree; for instance, it performs better at distinguishing longer unemployment spells.

使用更简单相对更复杂预测模型的相对收益与权衡是什么？

我们比较深度为 4 的浅决策树与 CatBoost 模型。如预期，更简单的树预测能力略降（$R^{2}$ 下降 $5\%$），转化为策略价值减少 $1$–$8\%$（见图 8(a)）。与通过缩放残差实现的均匀 $5\%$ $R^{2}$ 提升相比（见图 15），跨 $\alpha$ 的策略价值差异仅部分相似。CatBoost 相对决策树并非提供均匀改进；例如，它在区分更长失业期上表现更好。

Despite this performance gap, the simpler model offers potential advantages: it fits on a single sheet of paper, demands minimal computational infrastructure, can be easily explained to frontline case workers and resembles the categorical prioritization rules common in public institutions. [Johnson and Zhang, 2022]. Because more complex models incur higher costs, a planner might instead increase screening capacity. Formally, we define

尽管存在该表现差距，更简单模型仍有潜在优势：可写在一张纸上，几乎不需计算基础设施，易于向一线个案工作者解释，并类似公共机构常见的类别优先规则 [Johnson and Zhang, 2022]。因更复杂模型成本更高，规划者或可改为增大筛查能力。形式上，我们定义

$$\displaystyle\Delta_{\alpha}^{*}=\inf_{\Delta_{\alpha}\in(0,1-\beta)}\left\{\Delta_{\alpha}:\tfrac{V_{\text{TREE}}(\alpha+\Delta_{\alpha},\beta)-V_{\text{TREE}}(\alpha,\beta)}{V_{\text{CAT}}(\alpha,\beta)-V_{\text{TREE}}(\alpha,\beta)}\geqslant 1\right\}$$

the smallest $\Delta_{\alpha}^{*}$ that matches the policy-value gains of the CatBoost model. Empirically, $\Delta_{\alpha}^{*}$ mostly rises with $\alpha$ (see Figure 8(b)), consistent with our finding that the PAR decreases with $\alpha$. By framing the difference between models in terms of additional screenings, planners can directly compare the cost of increased capacity to that of deploying a more complex model.

能匹配 CatBoost 模型策略价值增益的最小 $\Delta_{\alpha}^{*}$。经验上，$\Delta_{\alpha}^{*}$ 大体随 $\alpha$ 上升（见图 8(b)），与我们关于 PAR 随 $\alpha$ 下降的发现一致。把模型差异表述为额外筛查次数，规划者可直接比较增大能力的成本与部署更复杂模型的成本。

(a) Difference in Policy Value

(a) 策略价值差异

6 Conclusion

6 结论

This paper develops a framework for quantifying the relative value of prediction in identifying the worst-off. We formalize tradeoffs between expanding screening capacity and improving predictive models, and show through both mathematical analysis and a real-world case study that prediction is not always the most important piece of the puzzle in social allocation systems. Future work could examine more specific application settings and cost structures, including distinctions between fixed and recurring costs, and explore policy levers that improve prediction unevenly, for example, by reducing errors in high-risk subgroups or by increasing robustness to distributional shifts. More broadly, we see a need for clearer theoretical foundations to understand the role of prediction in public-sector allocation, particularly in relation to the institutional and administrative systems in which it is embedded.

本文建立量化预测在识别最弱势群体中相对价值的框架。我们形式化扩展筛查能力与改进预测模型之间的权衡，并通过数学分析与真实案例研究表明：在社会分配系统中，预测并不总是拼图中最重要的一块。未来工作可考察更具体的应用设定与成本结构——包括固定与经常性成本的区分——并探索对预测改进不均的政策杠杆，例如降低高风险子群误差，或增强对分布漂移的稳健性。更广地看，我们需要更清晰的理论基础，以理解预测在公共部门分配中的角色，特别是相对其所嵌入的制度与行政系统。

Acknowledgements

This work is supported by the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research and by the Volkswagen Foundation, grant “Consequences of Artificial Intelligence for Urban Societies (CAIUS)”. Juan Carlos Perdomo is supported by the Center for Research on Computation and Society (CRCS) at Harvard University and by the Alfred P. Sloan Foundation grant G-2020-13941. We would like to thank the anonymous reviewers for their insightful comments, as well as Frauke Kreuter, Patrick Schenk and Moritz Hardt for their valuable feedback.

References

Aiken et al. [2023] E. Aiken, T. Ohlenburg, and J. Blumenstock. Moving targets: When does a poverty prediction model need to be updated? In Proceedings of the 6th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies , COMPASS ’23, page 117, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701498. doi: 10.1145/3588001.3609369 . URL https://doi.org/10.1145/3588001.3609369 .

Antoni et al. [2019a] M. Antoni, A. Ganzer, and P. vom Berge. Factually anonymous version of the Sample of Integrated Labour Market Biographies (SIAB-Regionalfile) – Version 7517 v1. Research Data Centre of the Federal Employment Agency (BA) at the Institute for Employment Research (IAB), 2019a. 10.5164/IAB.SIAB-R7517.de.en.v1.

Antoni et al. [2019b] M. Antoni, A. Ganzer, and P. vom Berge. Sample of Integrated Labour Market Biographies Regional File (SIAB-R) 1975 - 2027. FDZ-Datenreport 04/2019 (en), Research Data Centre of the Federal Employment Agency (BA) at the Institute for Employment Research (IAB), Nürnberg, 2019b. 10.5164/IAB.FDZD.1904.en.v1.

Athey and Wager [2021] S. Athey and S. Wager. Policy Learning with Observational Data. Econometrica , 89(1):133–161, 2021.

Bach et al. [2023] R. L. Bach, C. Kern, H. Mautner, and F. Kreuter. The impact of modeling decisions in statistical profiling. Data & Policy , 5:e32, 2023. doi: 10.1017/dap.2023.29 .

Barocas et al. [2023] S. Barocas, M. Hardt, and A. Narayanan. Fairness and Machine Learning: Limitations and Opportunities . MIT Press, 2023.

Blumenstock [2016] J. E. Blumenstock. Fighting Poverty with Data. Science , 2016.

Boehmer et al. [2024] N. Boehmer, Y. Nair, S. Shah, L. Janson, A. Taneja, and M. Tambe. Evaluating the Effectiveness of Index-Based Treatment Allocation. arXiv preprint arXiv:2402.11771 , 2024.

Chan et al. [2012] C. W. Chan, V. F. Farias, N. Bambos, and G. J. Escobar. Optimizing Intensive Care Unit Discharge Decisions with Patient Readmissions. Operations Research , 60(6):1323–1341, 2012. doi: 10.1287/opre.1120.1105 . URL https://doi.org/10.1287/opre.1120.1105 .

Chouldechova et al. [2018] A. Chouldechova, D. Benavides-Prado, O. Fialko, and R. Vaithianathan. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Conference on Fairness, Accountability and Transparency , pages 134–148. PMLR, 2018.

Clementi and Gallegati [2005] F. Clementi and M. Gallegati. Pareto’s law of income distribution: Evidence for Germany, the United Kingdom, and the United States. Econophysics of wealth distributions: Econophys-Kolkata I , pages 3–14, 2005.

Coston et al. [2023] A. Coston, A. Kawakami, H. Zhu, K. Holstein, and H. Heidari. A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) , pages 690–704, 2023. doi: 10.1109/SaTML54575.2023.00050 .

Desiere and Struyven [2021] S. Desiere and L. Struyven. Using Artificial Intelligence to classify Jobseekers: The Accuracy-Equity Trade-off. Journal of Social Policy , 50(2):367–385, Apr. 2021. ISSN 0047-2794, 1469-7823. doi: 10.1017/S0047279420000203 .

Desiere et al. [2019] S. Desiere, K. Langenbucher, and L. Struyven. Statistical Profiling in Public Employment Services: An International Comparison. Technical Report 224, OECD Publishing, 2019. URL https://doi.org/10.1787/b5e5f16e-en .

Drezner and Wesolowsky [1990] Z. Drezner and G. O. Wesolowsky. On the Computation of the Bivariate Normal Integral. Journal of Statistical Computation and Simulation , 1990.

Elmachtoub and Grigas [2022] A. N. Elmachtoub and P. Grigas. Smart “predict, then optimize”. Management Science , 68(1):9–26, 2022.

Fernández-Loría and Provost [2022] C. Fernández-Loría and F. Provost. Causal Decision Making and Causal Effect Estimation Are Not the Same…and Why It Matters. INFORMS Journal on Data Science , 1(1):4–16, 2022. doi: 10.1287/ijds.2021.0006 . URL https://doi.org/10.1287/ijds.2021.0006 .

Fischer-Abaigar et al. [2024] U. Fischer-Abaigar, C. Kern, N. Barda, and F. Kreuter. Bridging the Gap: Towards an Expanded Toolkit for Ai-driven Decision-making in the Public Sector. Government Information Quarterly , 41(4):101976, 2024. ISSN 0740-624X. doi: https://doi.org/10.1016/j.giq.2024.101976 . URL https://www.sciencedirect.com/science/article/pii/S0740624X24000686 .

Guerdan et al. [2023] L. Guerdan, A. Coston, K. Holstein, and Z. S. Wu. Counterfactual Prediction Under Outcome Measurement Error. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , FAccT ’23, page 1584–1598, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701924. doi: 10.1145/3593013.3594101 . URL https://doi.org/10.1145/3593013.3594101 .

Jain et al. [2024] S. Jain, K. Creel, and A. C. Wilson. Position: Scarce Resource Allocations That Rely On Machine Learning Should Be Randomized. In Forty-first International Conference on Machine Learning , 2024. URL https://openreview.net/forum?id=44qxX6Ty6F .

Johnson and Zhang [2022] R. A. Johnson and S. Zhang. What is the Bureaucratic Counterfactual? Categorical versus Algorithmic Prioritization in U.S. Social Policy. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , FAccT ’22, page 1671–1682, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393522. doi: 10.1145/3531146.3533223 . URL https://doi.org/10.1145/3531146.3533223 .

Kallus [2021] N. Kallus. More Efficient Policy Learning via Optimal Retargeting. Journal of the American Statistical Association , 116(534):646–658, 2021. doi: 10.1080/01621459.2020.1788948 . URL https://doi.org/10.1080/01621459.2020.1788948 .

Kern et al. [2024] C. Kern, R. Bach, H. Mautner, and F. Kreuter. When Small Decisions Have Big Impact: Fairness Implications of Algorithmic Profiling Schemes. ACM Journal on Responsible Computing , 1(4), Nov. 2024. doi: 10.1145/3689485 . URL https://doi.org/10.1145/3689485 .

Kitagawa and Tetenov [2018] T. Kitagawa and A. Tetenov. Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice. Econometrica , 86(2):591–616, 2018. doi: https://doi.org/10.3982/ECTA13288 . URL https://onlinelibrary.wiley.com/doi/abs/10.3982/ECTA13288 .

Kleinberg et al. [2015] J. Kleinberg, J. Ludwig, S. Mullainathan, and Z. Obermeyer. Prediction Policy Problems. American Economic Review , 105(5):491–495, 2015.

Körtner and Bonoli [2023] J. Körtner and G. Bonoli. Predictive Algorithms in the Delivery of Public Employment Services. In Handbook of Labour Market Policy in Advanced Democracies , pages 387–398. Edward Elgar Publishing, 2023.

Loxha and Morgandi [2014] A. Loxha and M. Morgandi. Profiling the unemployed: a review of OECD experiences and implications for emerging economics. Social protection discussion papers and notes , (91051), 2014.

Manski [2004] C. F. Manski. Statistical Treatment Rules for Heterogeneous Populations. Econometrica , 72(4):1221–1246, 2004. doi: https://doi.org/10.1111/j.1468-0262.2004.00530.x . URL https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1468-0262.2004.00530.x .

Perdomo [2024] J. C. Perdomo. The Relative Value of Prediction in Algorithmic Decision Making. In Proceedings of the 41st International Conference on Machine Learning , ICML’24. JMLR.org, 2024.

Perdomo et al. [2023] J. C. Perdomo, T. Britton, M. Hardt, and R. Abebe. Difficult Lessons on Social Prediction from Wisconsin Public Schools. arXiv preprint arXiv:2304.06205 , 2023.

Potash et al. [2015] E. Potash, J. Brew, A. Loewi, S. Majumdar, A. Reece, J. Walsh, E. Rozier, E. Jorgenson, R. Mansour, and R. Ghani. Predictive Modeling for Public Health: Preventing Childhood Lead Poisoning. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD ’15, page 2039–2047, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450336642. doi: 10.1145/2783258.2788629 . URL https://doi.org/10.1145/2783258.2788629 .

Salganik et al. [2020] M. J. Salganik, I. Lundberg, A. T. Kindel, C. E. Ahearn, K. Al-Ghoneim, A. Almaatouq, D. M. Altschul, J. E. Brand, N. B. Carnegie, R. J. Compton, D. Datta, T. Davidson, A. Filippova, C. Gilroy, B. J. Goode, E. Jahani, R. Kashyap, A. Kirchner, S. McKay, A. C. Morgan, A. Pentland, K. Polimis, L. Raes, D. E. Rigobon, C. V. Roberts, D. M. Stanescu, Y. Suhara, A. Usmani, E. H. Wang, M. Adem, A. Alhajri, B. AlShebli, R. Amin, R. B. Amos, L. P. Argyle, L. Baer-Bositis, M. Büchi, B.-R. Chung, W. Eggert, G. Faletto, Z. Fan, J. Freese, T. Gadgil, J. Gagné, Y. Gao, A. Halpern-Manners, S. P. Hashim, S. Hausen, G. He, K. Higuera, B. Hogan, I. M. Horwitz, L. M. Hummel, N. Jain, K. Jin, D. Jurgens, P. Kaminski, A. Karapetyan, E. H. Kim, B. Leizman, N. Liu, M. Möser, A. E. Mack, M. Mahajan, N. Mandell, H. Marahrens, D. Mercado-Garcia, V. Mocz, K. Mueller-Gastell, A. Musse, Q. Niu, W. Nowak, H. Omidvar, A. Or, K. Ouyang, K. M. Pinto, E. Porter, K. E. Porter, C. Qian, T. Rauf, A. Sargsyan, T. Schaffner, L. Schnabel, B. Schonfeld, B. Sender, J. D. Tang, E. Tsurkov, A. van Loon, O. Varol, X. Wang, Z. Wang, J. Wang, F. Wang, S. Weissman, K. Whitaker, M. K. Wolters, W. L. Woon, J. Wu, C. Wu, K. Yang, J. Yin, B. Zhao, C. Zhu, J. Brooks-Gunn, B. E. Engelhardt, M. Hardt, D. Knox, K. Levy, A. Narayanan, B. M. Stewart, D. J. Watts, and S. McLanahan. Measuring the Predictability of Life Outcomes with a Scientific Mass Collaboration. Proceedings of the National Academy of Sciences , 117(15):8398–8403, 2020. doi: 10.1073/pnas.1915006117 . URL https://www.pnas.org/doi/abs/10.1073/pnas.1915006117 .

Shirali et al. [2024] A. Shirali, R. Abebe, and M. Hardt. Allocation Requires Prediction Only if Inequality Is Low. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=WUicA0hOF9 .

Sun and Medaglia [2019] T. Q. Sun and R. Medaglia. Mapping the challenges of Artificial Intelligence in the public sector: Evidence from public healthcare. Government Information Quarterly , 36(2):368–383, 2019.

Wang et al. [2024] A. Wang, S. Kapoor, S. Barocas, and A. Narayanan. Against Predictive Optimization: On the Legitimacy of Decision-making Algorithms That Optimize Predictive Accuracy. ACM J. Responsib. Comput. , 1(1), Mar. 2024. doi: 10.1145/3636509 . URL https://doi.org/10.1145/3636509 .

Wilder and Welle [2024] B. Wilder and P. Welle. Learning treatment effects while treating those in need. arXiv preprint arXiv:2407.07596 , 2024.

Wirtz et al. [2019] B. W. Wirtz, J. C. Weyerer, and C. Geyer. Artificial Intelligence and the Public Sector—Applications and Challenges. International Journal of Public Administration , 42(7):596–615, 2019.

Appendix A Theoretical Investigation

附录 A 理论考察

(a) Normal Welfare Distribution

(a) 正态福利分布

(a) PAR

Appendix B Experiments

附录 B 实验

B.1 Experimental Setup and Labor Market Data

B.1 实验设置与劳动力市场数据

The dataset is provided via a Scientific Use File by the Research Data Centre (FDZ) of the German Federal Employment Agency (BA) at the Institute for Employment Research (IAB) [Antoni et al., 2019a, Antoni et al., 2019b]. It is a $2\%$ weakly anonymized random sample of the complete German labor market records from 1975 to 2017 and contains information on 1,827,903 individuals across 62,340,521 observations [Antoni et al., 2019b].

数据集由德国联邦就业局（BA）就业研究所（IAB）研究数据中心（FDZ）以科学使用文件形式提供 [Antoni et al., 2019a, Antoni et al., 2019b]。它是 1975 至 2017 年完整德国劳动力市场记录的 $2\%$ 弱匿名随机样本，覆盖 1,827,903 人、共 62,340,521 条观测 [Antoni et al., 2019b]。

We follow the same set of covariates and aggregation procedure for individual unemployment spells as described in Bach et al. [2023], incorporating demographic characteristics, labor market histories, and information about the most recent job. This results in 56 numerical variables and 24 categorical variables, which are one-hot encoded for model training. Figure 3 shows a histogram of individual unemployment durations, which we use as the basis for constructing the outcome variables. The distribution is right-skewed, with a concentration on short durations near zero and a long tail. Such a pattern is commonly observed in other welfare-related outcomes, such as health or income metrics. We define as prediction target the duration of the unemployment period in days $Y$, capped at $24$ months[^5]. Differentiating tail values is less important for decision-making, and capping also allows training across years with varying observation windows.

我们遵循 Bach et al. [2023] 所述协变量集与个体失业期聚合流程，纳入人口统计特征、劳动力市场史与最近工作信息。得到 56 个数值变量与 24 个类别变量，训练时做独热编码。图 3 给出个体失业时长直方图，作为构建结果变量的基础。分布右偏，短时长集中在零附近并有长尾。该模式在健康或收入等其他福利相关结果中也常见。我们将预测目标定义为失业天数 $Y$，封顶 $24$ 个月[^5]。区分尾部值对决策较不重要，封顶也便于在观察窗口不同的年份间训练。

B.2 Training Details

B.2 训练细节

We use CatBoost (https://catboost.ai) for model training. The model was trained for a maximum 5,000 iterations with an early stopping criterion (early_stopping_rounds = 20) based on validation performance. Additionally, we train a shallow Decision Tree (max_depth = 4) using the scikit-learn package. All hyperparameters are kept at their default settings unless otherwise specified.

我们用 CatBoost（https://catboost.ai）训练模型。最大迭代 5,000 次，并按验证表现设置早停（early_stopping_rounds = 20）。另外用 scikit-learn 训练浅决策树（max_depth = 4）。除非另说明，所有超参数保持默认。

B.3 Prediction Improvements

B.3 预测改进

To simulate an increase in predictive power by a specified amount $\Delta_{R^{2}}$, we adjust the model’s predictions $\hat{Y}$ using the residuals $Y-\hat{Y}$. Starting with the original predictions $\hat{Y}$ and true outcomes $Y$, we define the adjusted predictions as

为模拟预测能力增加指定量 $\Delta_{R^{2}}$，我们用残差 $Y-\hat{Y}$ 调整模型预测 $\hat{Y}$。从原始预测 $\hat{Y}$ 与真实结果 $Y$ 出发，定义调整后的预测为

$$\displaystyle\hat{Y}_{+}=\hat{Y}+\delta(Y-\hat{Y})$$

We can then determine the $\delta$ corresponding to an increase of $\Delta_{R^{2}}$ in the model’s $R^{2}$:

随后可确定使模型 $R^{2}$ 增加 $\Delta_{R^{2}}$ 所对应的 $\delta$：