AI evaluators favor job applications written by certain AI models, independent of candidate quality
作者: Nicolò Pagan, Michael Andreas Ebling, Stefania Ionescu, Anikó Hannák
分类: cs.CY, cs.AI, cs.MA
发布日期: 2026-10-30
💡 一句话要点
提出多模型审计框架以解决AI招聘筛选中的偏见问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: AI招聘 模型偏见 多模型审计 算法公平性 大语言模型
📋 核心要点
- 现有招聘筛选方法中，AI模型的选择可能导致候选人资格被误判，影响公平性。
- 本文提出了一种多模型审计框架，通过对比不同LLM写作代理和评估代理，分析模型选择对招聘结果的影响。
- 实验结果显示，模型不匹配可能导致约两名合格候选人被错误筛选，强调了模型选择的重要性。
📝 摘要（中文）
随着求职者越来越多地使用大型语言模型（LLMs）撰写求职信，雇主也开始利用LLMs进行筛选。本文探讨了模型选择是否独立于候选人资格影响招聘结果。通过将11个LLM写作代理与9个API托管评估代理配对，针对10个职位描述和每个职位50名候选人进行多模型审计。研究发现，评估者对写作模型的偏好存在一致的全球排名，且本地托管模型在评估中处于劣势。此外，评估者模型在引入偏见方面存在显著差异，影响候选人筛选结果。最后，研究讨论了算法公平性在就业中的影响及未来审计框架设计的潜在应用。
🔬 方法详解
问题定义：本文旨在解决在AI招聘筛选中，模型选择对候选人评估结果的影响。现有方法未能充分考虑模型偏见，导致合格候选人被误筛。
核心思路：通过构建多模型审计框架，结合不同的LLM写作代理与评估代理，系统性地分析模型选择对招聘结果的影响，旨在提高招聘筛选的公平性。
技术框架：整体框架包括11个LLM写作代理和9个评估代理，针对10个职位描述进行配对实验，评估每种组合对50名候选人的筛选效果。
关键创新：研究首次系统性地揭示了评估者对不同写作模型的偏好层级，指出本地模型在评估中处于劣势，并提出了自我偏好偏见的概念。
关键设计：在实验中，采用了多种评估标准，关注写作风格的特征以及评估者对这些特征的奖励或惩罚机制，确保结果的可靠性与有效性。
🖼️ 关键图片
📊 实验亮点
实验结果表明，在最不利的模型配对中，约有1.8名合格候选人因模型不匹配而被错误筛选，显示出模型选择对招聘结果的显著影响。这一发现强调了在AI招聘中设计公平筛选机制的重要性。
🎯 应用场景
该研究为招聘领域的AI应用提供了重要的理论基础，能够帮助企业在设计AI筛选系统时考虑模型选择的公平性。此外，研究结果可为政策制定者提供参考，以促进算法公平性和透明度的提升，确保招聘过程的公正性。
📄 摘要（原文）
Job applicants increasingly use large language models (LLMs) to write cover letters, while employers deploy LLMs to screen them. In this all-AI setting, we ask whether the \emph{choice of model} affects hiring outcomes independently of candidate qualifications. We introduce a multi-model auditing framework pairing eleven LLM writer agents with nine API-hosted evaluator agents across ten job descriptions and fifty candidates per job. Four findings emerge. First, we identify a generative preference hierarchy: a consistent global ranking of writer models by evaluator acceptance, in which locally hosted models are systematically disadvantaged regardless of their license, compounded by self-preference bias where evaluators favour their own outputs. Second, evaluator models differ substantially in how much bias they introduce based on applicants' tool choice, offering actionable guidance for fairer screening design. Third, this hierarchy has measurable competitive consequences: in the most adverse pairing, approximately two (1.8) out of twenty-five qualified candidates are filtered out purely due to model mismatch. Fourth, a common mechanism explains the patterns above: writer preference and self-preference trace to a combination of how much each writer leans on specific stylistic cues and how strongly evaluators, often unanimously, reward or penalise those same cues. A further check finds little evidence that an unmeasured "writer fingerprint" remains once these same features are accounted for. We discuss implications for algorithmic fairness in employment, regulation of AI-assisted screening, and the design of auditing frameworks that can be reapplied as the model landscape evolves.