Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generation
作者: Ryutaro Tanno, David G. T. Barrett, Andrew Sellergren, Sumedh Ghaisas, Sumanth Dathathri, Abigail See, Johannes Welbl, Karan Singhal, Shekoofeh Azizi, Tao Tu, Mike Schaekermann, Rhys May, Roy Lee, SiWai Man, Zahra Ahmed, Sara Mahdavi, Yossi Matias, Joelle Barral, Ali Eslami, Danielle Belgrave, Vivek Natarajan, Shravya Shetty, Pushmeet Kohli, Po-Sen Huang, Alan Karthikesalingam, Ira Ktena
分类: eess.IV, cs.CL, cs.CV, cs.LG
发布日期: 2023-11-30 (更新: 2023-12-20)
💡 一句话要点
提出Flamingo-CXR以解决放射科报告生成中的临床质量评估问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 放射科报告 自动报告生成 视觉-语言模型 AI与临床协作 报告质量评估
📋 核心要点
- 现有的放射科报告生成方法在临床质量评估上存在挑战，导致AI生成的报告难以被广泛接受。
- 本研究提出Flamingo-CXR，通过微调视觉-语言基础模型，生成高质量的胸部X光报告，并与临床医生进行协作。
- 实验结果表明，在多个案例中，AI生成的报告被放射科医生偏好，且AI与医生的协作提升了报告质量。
📝 摘要（中文）
放射科报告是现代医学的重要组成部分，影响诊断和治疗决策。然而，放射科医生的短缺限制了专家护理的可及性，并导致报告交付延误。尽管基于视觉-语言模型的自动报告生成显示出改善的潜力，但临床质量评估的挑战阻碍了其实际应用。本研究构建了一个先进的胸部X光报告生成系统Flamingo-CXR，通过对放射数据的微调，评估AI生成报告的质量。16名认证放射科医生对AI生成和人类撰写的报告进行了详细评估，结果显示在超过60%的案例中，至少一名放射科医生更倾向于AI报告。研究还提出了AI与临床医生协作的场景，AI生成初稿后由医生修订，结果显示在80%的住院病例和60%的重症监护病例中，AI辅助报告被认为等同或优于专家单独撰写的报告。
🔬 方法详解
问题定义：本研究旨在解决放射科报告生成中AI生成报告的临床质量评估问题。现有方法在实际应用中面临放射科医生短缺和报告质量不一致的挑战。
核心思路：论文提出的核心思路是通过微调一个已知的视觉-语言基础模型，Flamingo-CXR，来生成胸部X光报告，并与放射科医生进行协作，以提高报告的临床适用性和质量。
技术框架：整体架构包括数据收集、模型微调、报告生成和医生评估四个主要模块。首先，收集来自美国和印度的胸部X光数据，然后对模型进行微调以适应特定的放射科任务，最后生成报告并由医生进行评估。
关键创新：最重要的技术创新在于首次实现了AI与临床医生的协作，AI生成初稿后由医生进行修订，这种模式显著提升了报告的质量和接受度。
关键设计：在模型微调过程中，采用了特定的损失函数以优化报告的准确性，并设计了适合放射科特定需求的网络结构，确保生成的报告在临床上具有实用性。
🖼️ 关键图片
📊 实验亮点
实验结果显示，在超过60%的案例中，至少一名放射科医生更倾向于AI生成的报告。此外，在80%的住院病例和60%的重症监护病例中，AI辅助生成的报告被认为等同或优于专家单独撰写的报告，显示出显著的性能提升。
🎯 应用场景
该研究的潜在应用领域包括医院的放射科、急救中心和远程医疗服务。通过提高报告生成的效率和质量，Flamingo-CXR能够帮助缓解放射科医生的工作负担，改善患者的诊断和治疗体验，具有重要的实际价值和未来影响。
📄 摘要（原文）
Radiology reports are an instrumental part of modern medicine, informing key clinical decisions such as diagnosis and treatment. The worldwide shortage of radiologists, however, restricts access to expert care and imposes heavy workloads, contributing to avoidable errors and delays in report delivery. While recent progress in automated report generation with vision-language models offer clear potential in ameliorating the situation, the path to real-world adoption has been stymied by the challenge of evaluating the clinical quality of AI-generated reports. In this study, we build a state-of-the-art report generation system for chest radiographs, $\textit{Flamingo-CXR}$, by fine-tuning a well-known vision-language foundation model on radiology data. To evaluate the quality of the AI-generated reports, a group of 16 certified radiologists provide detailed evaluations of AI-generated and human written reports for chest X-rays from an intensive care setting in the United States and an inpatient setting in India. At least one radiologist (out of two per case) preferred the AI report to the ground truth report in over 60$\%$ of cases for both datasets. Amongst the subset of AI-generated reports that contain errors, the most frequently cited reasons were related to the location and finding, whereas for human written reports, most mistakes were related to severity and finding. This disparity suggested potential complementarity between our AI system and human experts, prompting us to develop an assistive scenario in which Flamingo-CXR generates a first-draft report, which is subsequently revised by a clinician. This is the first demonstration of clinician-AI collaboration for report writing, and the resultant reports are assessed to be equivalent or preferred by at least one radiologist to reports written by experts alone in 80$\%$ of in-patient cases and 60$\%$ of intensive care cases.