SAC$^2$-Net: Semantic Anchoring and Complementary-Consensus Fusion for Multimodal Micro-Expression Recognition

📄 arXiv: 2606.25542v1 📥 PDF

作者: Xuepeng Zheng, Tong Chen

分类: cs.CV

发布日期: 2026-06-24

💡 一句话要点

提出SAC$^2$-Net以解决多模态微表情识别中的异质性问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱三：空间感知与语义 (Perception & Semantics) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 微表情识别 多模态融合 语义锚定 互补共识 动作单元 深度学习 计算机视觉

📋 核心要点

  1. 现有微表情识别方法在处理微小面部动作和数据稀缺时存在显著挑战，尤其是在模态之间的异质性和可靠性问题。
  2. 本文提出的SAC$^2$-Net通过语义锚定软对齐和互补共识融合来解决模态间的异质性，增强了微表情的识别能力。
  3. 在五个微表情识别基准上进行的广泛实验表明，SAC$^2$-Net在不同评估设置下均实现了最先进的性能，显示出显著的提升。

📝 摘要（中文）

微表情识别（MER）因面部微小动作、数据有限及动作单元（AU）与情感类别之间的模糊关系而面临挑战。光流和运动放大技术被广泛应用于描述微妙的面部动态，但这两种模态常常表现出不对称的失败模式。为了解决跨模态异质性和空间变化的模态可靠性问题，本文提出了SAC$^2$-Net，一个语义锚定和互补共识网络，通过语义锚定软对齐（SASA）和互补共识融合（CCF）来实现多模态MER的有效融合。实验表明，SAC$^2$-Net在多个MER基准上表现出最先进或高度竞争的性能。

🔬 方法详解

问题定义：本文旨在解决多模态微表情识别中的跨模态异质性和空间变化的模态可靠性问题。现有方法在处理光流和运动放大模态时，常常面临噪声和失真，导致信息丢失。

核心思路：SAC$^2$-Net的核心思路是通过语义锚定来对齐不同模态，并在此基础上进行可靠性感知的融合，以充分利用模态间的互补性。

技术框架：该方法包括两个主要模块：语义锚定软对齐（SASA）和互补共识融合（CCF）。SASA将激活的AU转化为文本提示，作为稳定的语义锚点来对齐不同模态的表示；CCF则通过互补交换修复不可靠的局部证据，并通过共识精炼实现共享的空间聚焦。

关键创新：SASA构建了层次化的AU感知软标签，保留了样本之间的语义接近性，这与传统的硬对比学习方法有本质区别。CCF通过互补性修复和共识精炼，增强了模态融合的可靠性。

关键设计：在网络结构上，SAC$^2$-Net采用了多层次的特征提取模块，并设计了特定的损失函数以优化模态对齐和融合效果。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在五个微表情识别基准上，SAC$^2$-Net实现了最先进的性能，特别是在粗粒度和细粒度评估设置中，表现出显著的提升，超越了多个基线模型，验证了其有效性。

🎯 应用场景

该研究在情感计算、心理健康监测和人机交互等领域具有广泛的应用潜力。通过提高微表情识别的准确性，SAC$^2$-Net能够帮助开发更智能的情感识别系统，促进人机交互的自然性和有效性。

📄 摘要（原文）

Micro-expression recognition (MER) is challenging due to subtle facial movements, limited data, and the ambiguous relationship between Action Units (AUs) and emotion categories. Optical flow and motion magnification have been widely used to describe subtle facial dynamics from different perspectives: the former captures local motion displacement, while the latter amplifies weak appearance changes. In this work, we observe that these two modalities often exhibit asymmetric failure patterns: one modality may become noisy, distorted, or uninformative, while the other still preserves discriminative AU-related evidence. This phenomenon reveals their complementarity, but also raises two key challenges for fusion: cross-modal heterogeneity and spatially varying modality reliability. Motivated by this observation, we propose SAC$^2$-Net, a Semantic Anchoring and Complementary-Consensus Network for multimodal MER, which first aligns visual modalities with semantic anchors and then performs reliability-aware fusion. To reduce cross-modal heterogeneity before fusion, we introduce Semantic Anchoring Soft Alignment (SASA), which converts activated AUs into textual prompts and uses them as stable semantic anchors to align motion-magnified and optical-flow representations. Unlike hard contrastive learning, SASA constructs hierarchical AU-aware soft labels to preserve semantic proximity among samples with overlapping or anatomically related AU patterns. Based on the aligned representations, Complementary-Consensus Fusion (CCF) first repairs unreliable local evidence through complementary exchange and then enforces a shared spatial focus through consensus refinement. Extensive experiments on five MER benchmarks show that SAC$^2$-Net achieves state-of-the-art or highly competitive performance across coarse-grained, fine-grained, large-scale, and cross-dataset evaluation settings.