Histo-Genomic Knowledge Distillation For Cancer Prognosis From Histopathology Whole Slide Images

📄 arXiv: 2403.10040v2 📥 PDF

作者: Zhikang Wang, Yumeng Zhang, Yingxue Xu, Seiya Imoto, Hao Chen, Jiangning Song

分类: eess.IV, cs.CV

发布日期: 2024-03-15 (更新: 2024-03-18)

💡 一句话要点

提出G-HANet以解决癌症预后中的单模态图像信息不足问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 癌症预后 histo-genomic 知识蒸馏 多模态学习 超注意力网络 全切片图像 精准肿瘤学

📋 核心要点

  1. 现有的histo-genomic多模态方法在癌症预后中面临基因组测序可及性不足的挑战，限制了其临床应用。
  2. 本文提出的G-HANet通过蒸馏histo-genomic知识，提升了单模态全切片图像的推断能力，解决了信息不足的问题。
  3. 在五个TCGA基准数据集上的实验结果显示，G-HANet在性能上显著优于现有的WSI方法，且与基因组和多模态方法具有竞争力。

📝 摘要（中文）

Histo-genomic多模态方法在癌症预后中展现出显著潜力，但基因组测序在欠发达地区的可及性有限，限制了其临床应用。为此，本文提出了一种新颖的基因信息超注意力网络G-HANet，首次有效地在训练过程中蒸馏histo-genomic知识，以提升单模态全切片图像的推断能力。与传统的知识蒸馏方法相比，G-HANet在训练效率和跨模态交互学习方面表现优越。通过在五个TCGA基准数据集上的广泛实验，结果表明G-HANet显著超越了现有的WSI方法，并在基因组和多模态方法中表现出竞争力。

🔬 方法详解

问题定义：本文旨在解决癌症预后中单模态全切片图像信息不足的问题，现有的histo-genomic多模态方法因基因组测序的可及性问题而受到限制。

核心思路：G-HANet通过在训练过程中蒸馏histo-genomic知识，提升了单模态全切片图像的推断能力，旨在有效结合形态特征与基因表达信息。

技术框架：G-HANet的整体架构包括跨模态关联分支（CAB）和超注意力生存分支（HSB）。CAB负责从全切片图像中重建基因组数据，提取功能基因型与形态表型之间的关联；HSB则利用蒸馏的histo-genomic关联进行超注意力建模。

关键创新：G-HANet的主要创新在于其端到端的模型设计，能够高效地学习跨模态交互，显著提升了训练效率和推断性能。

关键设计：模型中采用了特定的损失函数和网络结构，以优化跨模态学习效果，并通过生成的形态基础权重实现超注意力建模。具体参数设置和网络结构细节在实验部分进行了详细描述。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在五个TCGA基准数据集上的实验结果表明，G-HANet在性能上显著超越了现有的WSI方法，具体提升幅度达到XX%（具体数据需根据实验结果填写），并在基因组和多模态方法中表现出竞争力，展示了其在癌症预后中的有效性。

🎯 应用场景

G-HANet的研究成果具有广泛的应用潜力，尤其是在癌症预后和精准肿瘤学领域。通过提升单模态全切片图像的推断能力，该方法能够为临床医生提供更为准确的癌症预后评估，推动个性化治疗的发展，尤其在资源有限的地区具有重要的实际价值。

📄 摘要（原文）

Histo-genomic multi-modal methods have recently emerged as a powerful paradigm, demonstrating significant potential for improving cancer prognosis. However, genome sequencing, unlike histopathology imaging, is still not widely accessible in underdeveloped regions, limiting the application of these multi-modal approaches in clinical settings. To address this, we propose a novel Genome-informed Hyper-Attention Network, termed G-HANet, which is capable of effectively distilling the histo-genomic knowledge during training to elevate uni-modal whole slide image (WSI)-based inference for the first time. Compared with traditional knowledge distillation methods (i.e., teacher-student architecture) in other tasks, our end-to-end model is superior in terms of training efficiency and learning cross-modal interactions. Specifically, the network comprises the cross-modal associating branch (CAB) and hyper-attention survival branch (HSB). Through the genomic data reconstruction from WSIs, CAB effectively distills the associations between functional genotypes and morphological phenotypes and offers insights into the gene expression profiles in the feature space. Subsequently, HSB leverages the distilled histo-genomic associations as well as the generated morphology-based weights to achieve the hyper-attention modeling of the patients from both histopathology and genomic perspectives to improve cancer prognosis. Extensive experiments are conducted on five TCGA benchmarking datasets and the results demonstrate that G-HANet significantly outperforms the state-of-the-art WSI-based methods and achieves competitive performance with genome-based and multi-modal methods. G-HANet is expected to be explored as a useful tool by the research community to address the current bottleneck of insufficient histo-genomic data pairing in the context of cancer prognosis and precision oncology.