Whose doctor does the AI recommend? An algorithm audit of reputation and demographic signals in large language model-assisted physician choice
作者: Syeda Anshrah Gillani, Mirza Samad Ahmed Baig
分类: cs.CY, cs.AI, cs.CL
发布日期: 2026-08-14
备注: 26 pages, 9 figures, 10 tables
💡 一句话要点
审计算法以评估大语言模型推荐医生的偏见与透明性
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 大语言模型 算法审计 医疗推荐 声誉信号 人口统计信号 透明性 偏见
📋 核心要点
- 现有的大语言模型在推荐医生时可能存在偏见，缺乏透明性，影响患者的选择。
- 通过随机算法审计，研究探讨了声誉和人口统计信号如何影响医生推荐，提出了重复审计的必要性。
- 实验结果显示，声誉信号对选择概率的影响显著，而模型在推荐理由中几乎不提及性别或种族，导致透明性不足。
📝 摘要（中文）
随着患者越来越多地询问大型语言模型助手推荐医生，这些系统成为了信息中介，影响着医生的可见性。本文报告了一项预设的随机算法审计，探讨了影响推荐的因素。研究发现，声誉信号在推荐中占主导地位，提升评分显著增加选择概率，而费用的提高则降低选择概率。尽管模型在推荐理由中几乎不提及性别或种族，但不同种族信号的名字在选择概率上存在显著差异。该研究强调了重复审计的重要性，以确保模型的透明性和可审计性。
🔬 方法详解
问题定义：本文旨在解决大型语言模型在推荐医生时可能存在的偏见和透明性不足的问题。现有方法缺乏对推荐因素的深入审计，导致患者选择受到不公平影响。
核心思路：研究通过预设的随机算法审计，系统性地分析影响医生推荐的因素，特别是声誉和人口统计信号的作用。设计上强调重复审计的重要性，以确保结果的可靠性和可验证性。
技术框架：整体架构包括七个模型（六个开放权重模型和一个gpt-4o-mini），在3,024个选择集上进行实验，涉及三种患者角色和九种提示改写，最终生成40,068个评分响应。
关键创新：最重要的创新在于采用随机算法审计方法，系统评估推荐的透明性和偏见，尤其是声誉信号的显著性与人口统计信号的微弱性。与现有方法相比，提供了更为严谨的审计框架。
关键设计：实验中采用了独立随机化的医生属性，使用了对应审计方法来信号化性别和种族。模型在推荐理由中提及性别或种族的比例极低，强调了透明性不足的问题。
🖼️ 关键图片
📊 实验亮点
实验结果显示，提升医生评分从3.9到4.7可使选择概率增加31.4个百分点，而费用从90美元提高到190美元则降低选择概率20个百分点。不同种族信号的名字在选择概率上存在显著差异，女性信号名字增加2.5个百分点，西班牙裔、南亚裔和黑人名字相较于白人名字也有显著提升。
🎯 应用场景
该研究的潜在应用领域包括医疗推荐系统和人工智能助手的设计，能够帮助开发更公平和透明的推荐算法。通过审计算法，医疗机构可以更好地理解和改善患者的选择过程，提升医疗服务的公平性和有效性。
📄 摘要（原文）
Patients increasingly ask large language model (LLM) assistants which doctor to see, making these systems AI infomediaries: algorithms that intermediate one person's choice among other people and thereby decide, silently and at scale, which physicians become visible. We report a prespecified randomized algorithm audit of what causally moves those recommendations. Seven models (six open-weight; gpt-4o-mini) each chose among five synthetic family-medicine physician cards whose attributes were independently randomized across 3,024 choice sets, three patient personas, nine prompt paraphrases and nine experimental arms, yielding 40,068 scored responses; gender and ethnicity were signaled through names following correspondence-audit methodology. Reputation signals dominate: raising a rating from 3.9 to 4.7 increases choice probability by 31.4 percentage points (pp), and raising the fee from $90 to $190 lowers it by 20.0 pp. Demographic parity is rejected, but not in the direction human audit studies predict: female-signaled names gain 2.5 pp, and Hispanic-, South-Asian- and Black-signaled names gain 1.3-2.9 pp over White-signaled names, tilts worth $7-$14 per visit in fee-equivalent terms, and a content-free first-listed position is worth $11. Yet models mentioned gender or ethnicity in at most 0.03% of their stated reasons and abstained in 0.39% of trials, so these effects are invisible in the models' own explanations, and transparency obligations relying on model self-report would not detect them. One reasoning model failed the prespecified auditability gate outright. The frozen design makes the audit repeatable: any new model can be assessed against identical stimuli, making recurring behavioural audit, rather than self-reported explanation, the monitoring technology fit for purpose.