CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs

📄 arXiv: 2606.27264v1 📥 PDF

作者: Hashmat Shadab Malik, Anees Ur Rehman Hashmi, Numan Saeed, Muzammal Naseer, Salman Khan, Christoph Lippert

分类: cs.CV

发布日期: 2026-06-25

💡 一句话要点

提出CORTEX以解决3D胸部CT诊断推理不足问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 3D胸部CT 推理模型 多模态学习 医学影像 结构化推理 临床应用 数据集

📋 核心要点

  1. 现有的胸部CT问答数据集缺乏结构化推理，导致推理过程难以验证和追溯。
  2. CORTEX通过四个阶段的诊断轨迹恢复推理过程，模拟放射科医生的工作流程。
  3. CORTEX包含76,177个经过验证的推理轨迹，提供了必要的监督和评估协议，推动了3D胸部CT MLLMs的发展。

📝 摘要（中文）

在多模态大型语言模型（MLLMs）中，医学影像推理展现出强大的潜力。然而，现有方法往往仅通过最终答案进行评估，缺乏可追溯的推理过程，尤其是在3D放射学中。现有的胸部CT问答数据集将专家放射科报告简化为仅包含答案的对，忽略了将发现与结论联系起来的推理过程。为此，本文提出了CORTEX（临床组织推理与结构化解释），这是一个针对3D胸部CT的结构化推理基准，提供了76,177个经过验证的推理轨迹，旨在为构建和评估可信的推理模型提供结构化监督和阶段性评估协议。

🔬 方法详解

问题定义：本文旨在解决现有3D胸部CT问答系统中推理过程缺失的问题。现有方法仅提供答案，无法追溯推理过程，影响了临床应用的可信度。

核心思路：CORTEX通过构建一个四阶段的诊断轨迹，恢复推理过程，确保每个问题的回答都能追溯到具体的观察和推理，增强了模型的可解释性和可信度。

技术框架：CORTEX的整体架构包括任务理解、视觉观察、诊断推理和答案综合四个主要阶段。每个阶段都经过严格的评估和验证，以确保推理的准确性。

关键创新：CORTEX的最大创新在于其结构化的推理轨迹设计，结合了临床专家的反馈，确保推理过程不仅符合医学标准，还能被有效评估。

关键设计：在数据生成过程中，使用了前沿的大型语言模型，并通过阶段性评估协议结合自动评分和专家审核进行验证，确保推理轨迹的质量和可靠性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

CORTEX提供了76,177个经过验证的推理轨迹，涵盖开放式和封闭式问答以及报告生成，显著提升了3D胸部CT MLLMs的推理能力。与现有数据集相比，CORTEX在推理的结构化和可验证性方面具有明显优势。

🎯 应用场景

CORTEX的研究成果可广泛应用于医学影像分析、临床决策支持系统等领域，提升3D胸部CT的诊断准确性和效率。未来，该基准有望推动更多医学领域的推理模型发展，促进人工智能在医疗中的应用。

📄 摘要（原文）

Reasoning in multimodal large language models (MLLMs) has shown strong promise in medical imaging. However, this reasoning is usually free-form text judged only by its final answer, making it hard to interpret and verify, especially in 3D radiology, where a diagnosis should be traceable to evidence in the scan. Existing chest CT question-answering datasets compound this by reducing expert radiology reports to answer-only pairs, dropping the reasoning that links findings to conclusions and omitting the patient history clinicians rely on. As a result, reasoning-capable 3D chest CT MLLMs remain out of reach, as neither the structured supervision needed to train them nor the protocol needed to verify their reasoning yet exists. We introduce CORTEX (Clinically Organized Reasoning and sTructured EXplanation), a structured reasoning benchmark for 3D chest CT. For each question, CORTEX restores the missing reasoning as a four-stage diagnostic trace mirroring a radiologist's workflow: task understanding, visual observation, diagnostic reasoning, and answer synthesis. We generate these traces using frontier large language models with broad medical and general-domain knowledge, then filter and verify them with a stage-level evaluation protocol combining automated rubric scoring with expert radiologist review. Crucially, both the reasoning structure and evaluation rubrics are designed in close collaboration with clinicians. Built on CT-RATE, a large, publicly available chest CT dataset without reasoning annotations, CORTEX comprises 76,177 validated reasoning traces across open-ended VQA, closed-ended VQA, and report generation, providing both the structured supervision and the stage-level evaluation protocol needed to build and evaluate trustworthy reasoning models for 3D chest CT. Our dataset and evaluation code will be made publicly available upon acceptance.