Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy

📄 arXiv: 2311.18664v1 📥 PDF

作者: Pedro Esteban Chavarrias Solano, Andrew Bulpitt, Venkataraman Subramanian, Sharib Ali

分类: cs.CV, cs.AI, cs.MM

发布日期: 2023-11-30

备注: 19 pages

💡 一句话要点

提出多任务学习方法以改善结肠镜下深度估计

🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics)

关键词: 深度估计 多任务学习 结肠镜 医学影像 计算机视觉 表面法线 注意力机制

📋 核心要点

  1. 现有的深度估计方法在结肠镜数据集上的有效性尚未得到充分验证，尤其是在低纹理区域的表现较差。
  2. 本文提出了一种多任务学习框架，通过共享编码器和两个解码器，利用表面法线预测来增强几何特征提取。
  3. 实验结果显示，所提方法在相对误差和$δ_{1}$准确率上均有显著提升，提供了结肠镜深度估计的新基准。

📝 摘要（中文）

结肠镜筛查是评估结肠和直肠异常的金标准程序。准确测量异常黏膜区域及其三维重建对于量化调查区域和客观评估疾病负担至关重要。然而，由于器官复杂的拓扑结构和可变的物理条件，深度估计面临巨大挑战。本文提出了一种新颖的多任务学习方法，通过共享编码器和两个解码器（表面法线解码器和深度估计解码器），结合注意力机制和跨任务一致性损失，显著提高了深度估计的准确性。实验结果表明，相较于最先进的基线方法BTS，本文方法在相对误差上提高了14.17%，在$δ_{1}$准确率上提高了10.4%。

🔬 方法详解

问题定义：本文旨在解决结肠镜下深度估计的挑战，现有方法在复杂的器官拓扑和低纹理区域表现不佳，导致深度估计不准确。

核心思路：提出一种多任务学习（MTL）方法，通过共享编码器和两个解码器（表面法线解码器和深度估计解码器），利用表面法线预测来改善深度估计的准确性。

技术框架：整体架构包括一个共享编码器，负责提取特征，和两个解码器，分别用于表面法线和深度估计。深度估计解码器中引入了注意力机制，以增强全局上下文感知能力。

关键创新：通过跨任务一致性损失，促进表面法线和深度估计之间的几何关系，从而提高特征提取的有效性，这是与现有方法的本质区别。

关键设计：在损失函数中引入了跨任务一致性损失，确保两个任务之间的输出保持一致，同时在深度估计解码器中使用了注意力机制，以提升对全局信息的捕捉能力。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，所提方法在相对误差上提高了14.17%，在$δ_{1}$准确率上提高了10.4%，显著优于最先进的基线方法BTS，提供了结肠镜深度估计的新基准。

🎯 应用场景

该研究在医学影像分析领域具有重要应用潜力，尤其是在结肠镜检查中，能够提供更准确的深度估计，帮助医生更好地评估病变情况。未来，该方法可扩展到其他内窥镜检查和医学成像任务中，提升自动化诊断的准确性和效率。

📄 摘要（原文）

Colonoscopy screening is the gold standard procedure for assessing abnormalities in the colon and rectum, such as ulcers and cancerous polyps. Measuring the abnormal mucosal area and its 3D reconstruction can help quantify the surveyed area and objectively evaluate disease burden. However, due to the complex topology of these organs and variable physical conditions, for example, lighting, large homogeneous texture, and image modality estimating distance from the camera aka depth) is highly challenging. Moreover, most colonoscopic video acquisition is monocular, making the depth estimation a non-trivial problem. While methods in computer vision for depth estimation have been proposed and advanced on natural scene datasets, the efficacy of these techniques has not been widely quantified on colonoscopy datasets. As the colonic mucosa has several low-texture regions that are not well pronounced, learning representations from an auxiliary task can improve salient feature extraction, allowing estimation of accurate camera depths. In this work, we propose to develop a novel multi-task learning (MTL) approach with a shared encoder and two decoders, namely a surface normal decoder and a depth estimator decoder. Our depth estimator incorporates attention mechanisms to enhance global context awareness. We leverage the surface normal prediction to improve geometric feature extraction. Also, we apply a cross-task consistency loss among the two geometrically related tasks, surface normal and camera depth. We demonstrate an improvement of 14.17% on relative error and 10.4% improvement on $δ_{1}$ accuracy over the most accurate baseline state-of-the-art BTS approach. All experiments are conducted on a recently released C3VD dataset; thus, we provide a first benchmark of state-of-the-art methods.