DocTalk: Enhancing Medical Knowledge Retrieval via Fine-tuned Large Language Models with Chain-of-Thought Reasoning for Robust Clinical Decision Support

📄 arXiv: 2610.34871v1 📥 PDF

作者: Asad Gohar, Md Ariful Islam Mozumder, Moon-Il Joo, Hee-Cheol Kim

分类: cs.AI, cs.CL, cs.CV

发布日期: 2026-10-26

💡 一句话要点

提出DocTalk以解决医疗知识检索中的安全性与有效性问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 医疗问答 临床决策支持 指令调优 链式思维推理 数据生成 低秩适应 电子健康记录

📋 核心要点

  1. 现有大型语言模型在医疗领域的应用面临幻觉和领域不对齐等挑战，影响临床决策支持的安全性和有效性。
  2. 论文提出DocTalk，一个基于去标识化出院总结的问答系统，采用链式思维推理和高保真数据生成管道来提升回答的准确性。
  3. 实验结果显示，经过指令调优后，模型在DocTalk-QA数据集上的表现显著提升，准确率在PubMedQA数据集上也得到了改善。

📝 摘要（中文）

本研究旨在解决大型语言模型在医疗领域应用中的安全性和有效性挑战，特别是幻觉和领域不对齐的问题。我们提出了DocTalk，一个医生辅助的问答系统，能够从去标识化的出院总结中生成简洁且基于病历的回答。通过高保真数据生成管道，我们生成了500对临床相关的问答对，并对六种开源大型语言模型进行了评估。结果表明，经过参数高效的指令调优后，模型在各项指标上均有显著提升，最终实现了临床决策支持的有效性。

🔬 方法详解

问题定义：本研究旨在解决大型语言模型在医疗知识检索中的幻觉和领域不对齐问题，这些问题会影响临床决策支持的安全性和有效性。

核心思路：论文提出DocTalk，通过生成基于病历的简洁回答，利用链式思维推理和高保真数据生成管道来提高问答系统的准确性和可靠性。

技术框架：整体架构包括数据生成、问答对生成、模型评估等模块。首先通过句子级分解生成临床相关问答对，然后对六种开源大型语言模型进行评估和调优。

关键创新：最重要的技术创新在于结合了链式思维推理和参数高效的指令调优，显著提升了模型在临床问答中的表现，与传统方法相比具有更高的准确性和可解释性。

关键设计：在模型调优过程中，采用了低秩适应（LoRA）技术，并进行了提示工程和上下文学习，以确保生成的回答在语义和临床上都具有高度的准确性。

📊 实验亮点

实验结果显示，经过指令调优后，DocTalk在DocTalk-QA数据集上的ROUGE-L达到了0.40，BERTScore-F1为0.76，UMLS-F1为0.49。在PubMedQA数据集上，Llama-3.1-8B-Instruct的准确率从0.931提升至0.977，Llama-3-Med42-8B的准确率从0.935提升至0.943，显示出显著的性能提升。

🎯 应用场景

DocTalk的潜在应用场景包括医院的电子健康记录（EHR）查询、临床决策支持系统以及医疗教育等领域。通过提供基于临床数据的准确回答，DocTalk能够帮助医生更快地做出决策，提高医疗服务的质量和效率。未来，该系统可能会在医疗AI助手中发挥更大的作用，推动智能医疗的发展。

📄 摘要（原文）

Objective: The rapid advancement of large language models (LLMs) in healthcare has opened new possibilities for AI-assisted clinical decision-making; however, safe and effective deployment remains challenging because of hallucination and domain misalignment. This study introduces DocTalk, a doctor-assistive question-answering (QA) system designed to generate concise, chart-grounded responses from de-identified discharge summaries. Methods: We developed a high-fidelity data generation pipeline that produces 500 clinically relevant QA pairs through sentence-level decomposition, chain-of-thought-guided question formulation, and expert review. Six open-source LLMs, including three general-purpose and three biomedical models, were evaluated before and after parameter-efficient instruction tuning using Low-Rank Adaptation (LoRA). The framework integrates prompt engineering, in-context learning, and medically aligned evaluation to support factual and interpretable responses. Results: On DocTalk-QA, instruction tuning produced consistent improvements across lexical, semantic, and clinical fidelity metrics. The best-performing tuned model, Llama-3-Med42-8B, achieved ROUGE-L = 0.40, BERTScore-F1 = 0.76, and UMLS-F1 = 0.49. To examine external generalization beyond the DocTalk-QA dataset, the two best-performing models, Llama-3.1-8B-Instruct and Llama-3-Med42-8B, were evaluated on PubMedQA before and after instruction tuning. Accuracy improved from 0.931 to 0.977 for Llama-3.1-8B-Instruct and from 0.935 to 0.943 for Llama-3-Med42-8B. Conclusion: DocTalk was deployed as a secure web-based platform for interactive EHR querying. These findings suggest that clinically grounded instruction tuning can improve medical QA performance and support practical clinical decision support.