MambaUIE&SR: Unraveling the Ocean's Secrets with Only 2.8 GFLOPs

📄 arXiv: 2404.13884v2 📥 PDF

作者: Zhihao Chen, Yiyuan Ge

分类: eess.IV, cs.CV

发布日期: 2024-04-22 (更新: 2024-05-24)

备注: arXiv admin note: text overlap with arXiv:2305.08824 by other authors

🔗 代码/项目: GITHUB

💡 一句话要点

提出MambaUIE以高效解决水下图像增强问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 水下图像增强 状态空间模型 视觉状态空间 动态交互块 深度学习 计算机视觉 图像处理

📋 核心要点

  1. 现有的水下图像增强方法在处理局部细节特征时效果不佳，导致图像质量提升有限。
  2. 本文提出MambaUIE架构，通过引入视觉状态空间模块和动态交互块，有效结合全局和局部信息。
  3. 实验结果显示，MambaUIE在保持高准确率的同时，GFLOPs减少至2.715G，显著提升了效率。

📝 摘要（中文）

水下图像增强（UIE）技术旨在解决由于光吸收和散射导致的水下图像退化问题。近年来，基于卷积神经网络（CNN）和变换器（Transformer）的方法得到了广泛研究。然而，结合CNN和Transformer的方法仍受到Transformer复杂性的影响，无法最大化性能。本文提出了一种基于状态空间模型（SSM）的Mamba架构，能够有效建模长距离信息，同时保持线性复杂度。我们定制了MambaUIE架构，引入视觉状态空间（VSS）模块以捕获宏观层面的全局上下文信息，同时挖掘微观层面的局部信息。实验结果表明，MambaUIE在UIEB数据集上相较于现有方法GFLOPs减少了67.4%。

🔬 方法详解

问题定义：水下图像增强面临光吸收和散射导致的图像质量下降问题，现有方法在提取局部细节特征时存在不足，无法充分利用重要信息。

核心思路：本研究提出MambaUIE架构，利用状态空间模型（SSM）来高效建模全局和局部信息，旨在提高水下图像的增强效果。

技术框架：MambaUIE的整体架构包括视觉状态空间（VSS）模块用于捕获全局上下文信息，动态交互块（DIB）和空间前馈网络（SGFN）用于局部信息的聚合。

关键创新：MambaUIE是首个基于SSM构建的水下图像增强模型，突破了在准确性与计算复杂度之间的限制，显著提高了效率。

关键设计：在网络结构中，VSS模块和DIB的设计使得全局与局部信息的交互更加高效，参数设置经过优化以保持较小的模型规模，同时确保高准确率。

📊 实验亮点

实验结果表明，MambaUIE在UIEB数据集上相较于现有最先进方法GFLOPs减少了67.4%，达到2.715G，同时保持了高准确率，展示了其在水下图像增强中的优越性能。

🎯 应用场景

该研究在水下图像处理领域具有广泛的应用潜力，尤其在海洋探测、环境监测和水下机器人视觉系统中。通过提高水下图像的质量，能够更好地支持科学研究和实际应用，推动相关技术的发展。

📄 摘要（原文）

Underwater Image Enhancement (UIE) techniques aim to address the problem of underwater image degradation due to light absorption and scattering. In recent years, both Convolution Neural Network (CNN)-based and Transformer-based methods have been widely explored. In addition, combining CNN and Transformer can effectively combine global and local information for enhancement. However, this approach is still affected by the secondary complexity of the Transformer and cannot maximize the performance. Recently, the state-space model (SSM) based architecture Mamba has been proposed, which excels in modeling long distances while maintaining linear complexity. This paper explores the potential of this SSM-based model for UIE from both efficiency and effectiveness perspectives. However, the performance of directly applying Mamba is poor because local fine-grained features, which are crucial for image enhancement, cannot be fully utilized. Specifically, we customize the MambaUIE architecture for efficient UIE. Specifically, we introduce visual state space (VSS) blocks to capture global contextual information at the macro level while mining local information at the micro level. Also, for these two kinds of information, we propose a Dynamic Interaction Block (DIB) and Spatial feed-forward Network (SGFN) for intra-block feature aggregation. MambaUIE is able to efficiently synthesize global and local information and maintains a very small number of parameters with high accuracy. Experiments on UIEB datasets show that our method reduces GFLOPs by 67.4% (2.715G) relative to the SOTA method. To the best of our knowledge, this is the first UIE model constructed based on SSM that breaks the limitation of FLOPs on accuracy in UIE. The official repository of MambaUIE at https://github.com/1024AILab/MambaUIE.