Faithful Explanations under Missing Modalities in Multimodal Dementia Prognosis

📄 arXiv: 2610.35430v1 📥 PDF

作者: Firuz Kamalov

分类: cs.AI

发布日期: 2026-10-26

💡 一句话要点

提出多模态缺失情况下的可信解释方法以改善阿尔茨海默病预后

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多模态学习 阿尔茨海默病 缺失数据处理 语言模型 可信解释

📋 核心要点

  1. 现有多模态预后模型在缺失高成本模态时，可能产生幻影归因和不可信叙述，影响临床决策。
  2. 论文提出了一种新的评估框架，通过对缺失模态的处理，提升模型解释的可信性和准确性。
  3. 实验结果显示，缺失模态的命名显著提高了模型的承认率，并减少了幻影证据的出现，提升了预测准确性。

📝 摘要（中文）

随着大型语言模型在多模态预后模型中的应用日益增加，如何在缺失高成本模态（如淀粉样PET和脑脊液生物标志物）时保持解释的可信性尚未得到充分验证。本文通过对307名阿尔茨海默病神经影像倡议参与者的数据进行分析，探讨了模型在缺失证据下的表现，识别了“幻影归因”和“不可信叙述”两种失败点。研究表明，缺失数据策略在归因中赋予未观察模态12%至23%的权重，且在四个模态缺失时可高达45%。通过命名缺失模态，语言模型的承认率从48%提升至88%。

🔬 方法详解

问题定义：本文旨在解决多模态预后模型在缺失高成本模态时的解释可信性问题。现有方法在缺失证据时容易产生幻影归因和不可信叙述，影响临床应用。

核心思路：论文通过对307名参与者的数据进行分析，提出了针对缺失模态的处理策略，旨在提高模型的解释准确性和可信度。

技术框架：整体流程包括数据遮蔽、预测转换、恢复遮蔽数据以获得干预性真实值，以及对语言模型输出的评估。主要模块包括数据预处理、模型训练和结果分析。

关键创新：最重要的创新在于提出了缺失模态的命名策略，显著提高了模型对缺失证据的承认率，并减少了幻影证据的出现。与现有方法相比，该策略在保持准确性的同时，提升了模型的解释能力。

关键设计：在实验中，采用了多种缺失数据策略，评估了不同语言模型在缺失模态下的表现，关键参数设置包括模态遮蔽的数量和类型，以及对语言模型的提示设计。

📊 实验亮点

实验结果显示，缺失模态的命名策略使得语言模型的承认率从48%提升至88%，同时幻影证据减少了20倍。通过对执行恢复的检查，模型的预测准确性从27%-41%提升至69%（开放权重叙述者）和99.9%（专有模型）。

🎯 应用场景

该研究具有广泛的应用潜力，尤其在阿尔茨海默病等神经退行性疾病的早期诊断和预后评估中。通过提高模型解释的可信性，能够帮助临床医生做出更为准确的决策，进而改善患者的治疗效果和生活质量。

📄 摘要（原文）

Large language models are increasingly used to turn the feature attributions of multimodal prognostic models into narratives for clinicians, yet costly modalities such as amyloid PET and cerebrospinal fluid biomarkers are often missing, and whether these explanations stay faithful under missing evidence has not been tested. We separate two failure points: phantom attribution, in which the model credits modalities that were never observed, and unfaithful narration, in which the report omits, cites or misjudges missing modalities. Using 307 participants of the Alzheimer's Disease Neuroimaging Initiative with complete data, we masked every subset of four costly modalities, predicted 36-month conversion from mild cognitive impairment to Alzheimer's disease, and restored the masked data to obtain interventional ground truth. All five missing-data strategies assigned 12% to 23% of attribution to unobserved modalities, and up to 45% with four absent; missingness-aware strategies minimised it at no cost in accuracy. Across 51,200 reports from four language models and four prompting regimes, naming absent modalities in the prompt raised acknowledgement from 48% to 88% and cut phantom evidence twenty-fold, but advice on whether acquiring a missing test would change the prediction was correct in only 27% to 41% of cases until checked against executed restorations, which raised accuracy to 69% for open-weight narrators and 99.9% for a proprietary model. Higher phantom attribution was associated with more phantom evidence and fewer faithful direction claims, and direction fidelity, measured by a calibrated language-model judge, depended on narrator scale rather than prompting. Narrators told to name absent modalities sometimes declared observed ones absent, an error that a structured template eliminated. We release the masking protocol and evaluation code.