MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysis

📄 arXiv: 2404.15580v2 📥 PDF

作者: Jiaxin Zhuang, Linshan Wu, Qiong Wang, Peng Fei, Varut Vardhanabhuti, Lin Luo, Hao Chen

分类: cs.CV

发布日期: 2024-04-24 (更新: 2025-01-10)

备注: submitted to a journal, updated v2

💡 一句话要点

提出Mask in Mask框架以提升3D医学图像分析性能

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 自监督学习 3D医学图像 特征学习 Mask in Mask 医学影像分析 深度学习 跨层对齐 卷积神经网络

📋 核心要点

  1. 现有的MAE方法在处理高维3D医学图像时缺乏层次设计，限制了下游任务的性能。
  2. 本文提出的MiM框架通过多级粒度的掩码输入和跨层对齐机制，增强了特征学习的层次性。
  3. 在十三个公共数据集上的实验结果显示，MiM在分割和分类任务中显著优于其他自监督学习方法。

📝 摘要（中文）

本文提出了一种新颖的Mask in Mask (MiM)自监督预训练框架，旨在通过从不同尺度的层次视觉标记中学习判别性表示，来提升3D医学图像分析中的Masked AutoEncoder (MAE)性能。该方法引入了多级粒度的掩码输入，同时在细粒度和粗粒度层面进行重建，并通过跨层对齐机制强化解剖相似性。实验结果表明，MiM在器官、病变和肿瘤分割及疾病分类任务中优于其他自监督学习方法，且在大规模数据集上预训练进一步提升了下游任务的性能。

🔬 方法详解

问题定义：本文旨在解决现有MAE在处理高维3D医学图像时的性能瓶颈，特别是缺乏层次设计导致的表现不足。

核心思路：MiM框架通过引入多级粒度的掩码输入，允许模型在细粒度和粗粒度层面同时进行重建，从而学习更丰富的特征表示。

技术框架：该框架包括多个模块，首先生成不同粒度的掩码输入，然后通过重建过程和跨层对齐机制进行特征学习，最后使用混合骨干网络提升学习效率。

关键创新：MiM的主要创新在于引入了层次化的掩码输入和跨层对齐机制，这与传统MAE方法的单一层次输入设计形成了显著区别。

关键设计：在模型设计中，采用了多级掩码输入策略，损失函数结合了重建损失和对齐损失，网络结构上使用了混合骨干网络以增强层次表示学习。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在实验中，MiM在器官、病变和肿瘤分割及疾病分类任务中表现出色，相较于其他自监督学习方法，性能提升显著，尤其是在大规模数据集上预训练后，性能进一步增强，显示出大规模预训练的重要性。

🎯 应用场景

该研究具有广泛的应用潜力，特别是在医学影像分析领域，如肿瘤检测、器官分割和疾病分类等。通过提升3D医学图像的分析能力，MiM框架能够为临床决策提供更为精准的支持，未来可能推动医疗影像处理的智能化进程。

📄 摘要（原文）

The Vision Transformer (ViT) has demonstrated remarkable performance in Self-Supervised Learning (SSL) for 3D medical image analysis. Masked AutoEncoder (MAE) for feature pre-training can further unleash the potential of ViT on various medical vision tasks. However, due to large spatial sizes with much higher dimensions of 3D medical images, the lack of hierarchical design for MAE may hinder the performance of downstream tasks. In this paper, we propose a novel \textit{Mask in Mask (MiM)} pre-training framework for 3D medical images, which aims to advance MAE by learning discriminative representation from hierarchical visual tokens across varying scales. We introduce multiple levels of granularity for masked inputs from the volume, which are then reconstructed simultaneously ranging at both fine and coarse levels. Additionally, a cross-level alignment mechanism is applied to adjacent level volumes to enforce anatomical similarity hierarchically. Furthermore, we adopt a hybrid backbone to enhance the hierarchical representation learning efficiently during the pre-training. MiM was pre-trained on a large scale of available 3D volumetric images, \textit{i.e.,} Computed Tomography (CT) images containing various body parts. Extensive experiments on thirteen public datasets demonstrate the superiority of MiM over other SSL methods in organ/lesion/tumor segmentation and disease classification. We further scale up the MiM to large pre-training datasets with more than 10k volumes, showing that large-scale pre-training can further enhance the performance of downstream tasks. The improvement also concluded that the research community should pay more attention to the scale of the pre-training dataset towards the healthcare foundation model for 3D medical images.