Spectral-Spatial Mamba for Hyperspectral Image Classification

📄 arXiv: 2404.18401v3 📥 PDF

作者: Lingbo Huang, Yushi Chen, Xin He

分类: cs.CV

发布日期: 2024-04-29 (更新: 2024-08-01)

备注: 23 pages

期刊: Remote Sens. 2024, 16, 2449

DOI: 10.3390/rs16132449

💡 一句话要点

提出光谱空间Mamba以解决高光谱图像分类中的计算复杂性问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱五：交互与反应 (Interaction & Reaction)

关键词: 高光谱图像 深度学习 Transformer Mamba模型 图像分类 信息融合 计算效率

📋 核心要点

  1. 现有的Transformer模型在高光谱图像分类中计算复杂度高，限制了其应用。
  2. 提出的光谱空间Mamba通过生成光谱和空间令牌，实现信息的有效融合，提升分类性能。
  3. 实验结果显示，SS-Mamba在多个数据集上表现优异，具有竞争力的性能提升。

📝 摘要（中文）

近年来，深度学习模型在高光谱图像（HSI）分类中表现出色，尤其是Transformer因其在建模空间-光谱特征的长程依赖性方面的优势而受到关注。然而，Transformer的自注意力机制导致其计算复杂度呈二次增长，限制了其在HSI处理中的应用。为此，本文首次尝试将基于状态空间模型的Mamba应用于HSI分类，提出了光谱空间Mamba（SS-Mamba）。SS-Mamba主要由光谱空间令牌生成模块和多个堆叠的光谱空间Mamba块组成，通过对HSI立方体进行处理，实现了光谱和空间信息的有效融合。实验结果表明，该模型在多个HSI数据集上取得了与最先进方法相当的竞争性结果，展示了Mamba方法在HSI分类中的新潜力。

🔬 方法详解

问题定义：本文旨在解决高光谱图像分类中Transformer模型计算复杂度过高的问题，导致其在实际应用中的局限性。

核心思路：提出光谱空间Mamba（SS-Mamba），通过生成光谱和空间令牌，利用Mamba的高效计算能力，克服Transformer的计算瓶颈。

技术框架：SS-Mamba包括光谱空间令牌生成模块和多个堆叠的光谱空间Mamba块。令牌生成模块将HSI立方体转换为空间和光谱令牌，随后这些令牌被送入堆叠的Mamba块进行处理。

关键创新：SS-Mamba的核心创新在于将Mamba模型应用于HSI分类，利用其高效的计算能力和信息融合机制，显著提升了分类性能。

关键设计：每个光谱空间Mamba块由两个基本的Mamba块和一个特征增强模块组成，分别处理空间和光谱令牌，并通过中心区域信息进行调制，增强特征表达能力。

📊 实验亮点

实验结果表明，SS-Mamba在多个高光谱数据集上取得了与当前最先进方法相当的性能，展示了其在计算效率和分类精度上的优势，具体提升幅度未知。

🎯 应用场景

该研究在高光谱图像分类领域具有广泛的应用潜力，能够用于环境监测、农业监测、城市规划等多个领域。通过提高分类精度和效率，SS-Mamba有望推动高光谱遥感技术的实际应用，促进相关行业的发展。

📄 摘要（原文）

Recently, deep learning models have achieved excellent performance in hyperspectral image (HSI) classification. Among the many deep models, Transformer has gradually attracted interest for its excellence in modeling the long-range dependencies of spatial-spectral features in HSI. However, Transformer has the problem of quadratic computational complexity due to the self-attention mechanism, which is heavier than other models and thus has limited adoption in HSI processing. Fortunately, the recently emerging state space model-based Mamba shows great computational efficiency while achieving the modeling power of Transformers. Therefore, in this paper, we make a preliminary attempt to apply the Mamba to HSI classification, leading to the proposed spectral-spatial Mamba (SS-Mamba). Specifically, the proposed SS-Mamba mainly consists of spectral-spatial token generation module and several stacked spectral-spatial Mamba blocks. Firstly, the token generation module converts any given HSI cube to spatial and spectral tokens as sequences. And then these tokens are sent to stacked spectral-spatial mamba blocks (SS-MB). Each SS-MB block consists of two basic mamba blocks and a spectral-spatial feature enhancement module. The spatial and spectral tokens are processed separately by the two basic mamba blocks, respectively. Besides, the feature enhancement module modulates spatial and spectral tokens using HSI sample's center region information. In this way, the spectral and spatial tokens cooperate with each other and achieve information fusion within each block. The experimental results conducted on widely used HSI datasets reveal that the proposed model achieves competitive results compared with the state-of-the-art methods. The Mamba-based method opens a new window for HSI classification.