Contrastive Learning of View-Invariant Representations for Facial Expressions Recognition

📄 arXiv: 2311.06852v1 📥 PDF

作者: Shuvendu Roy, Ali Etemad

分类: cs.CV

发布日期: 2023-11-12

备注: Accepted in ACM Transactions on Multimedia Computing, Communications, and Applications

💡 一句话要点

提出ViewFX框架以解决非正面视角下的面部表情识别问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 面部表情识别 对比学习 视角不变 自监督学习 监督学习 特征表示 情感计算

📋 核心要点

  1. 现有面部表情识别方法在处理非正面视角图像时表现不佳，导致识别准确率下降。
  2. 本文提出的ViewFX框架通过自监督和监督对比损失学习视角不变特征，从而提高识别性能。
  3. 实验结果显示，ViewFX在KDEF和DDCF数据集上设立了新的最先进水平，且对视角变化的鲁棒性增强。

📝 摘要（中文）

尽管面部表情识别（FER）领域取得了显著进展，但现有方法在处理非正面视角的图像时表现不佳。本文提出了ViewFX，一个基于对比学习的视角不变FER框架，能够准确分类不同视角下的面部表情。ViewFX通过自监督对比损失学习视角不变特征，并引入监督对比损失以增强不同表情之间的区分。实验表明，该方法在KDEF和DDCF两个公共多视角数据集上超越了现有技术，且对挑战性角度和训练标签数量的敏感性显著降低。

🔬 方法详解

问题定义：本文旨在解决面部表情识别中由于视角变化导致的识别准确率下降问题。现有方法通常在训练时仅使用正面视角数据，导致在非正面视角下表现不佳。

核心思路：提出ViewFX框架，通过自监督对比学习和监督对比学习相结合，学习视角不变的特征表示，从而提高不同视角下的表情分类准确性。

技术框架：ViewFX框架包括三个主要模块：自监督对比损失模块、监督对比损失模块和Barlow twins损失模块。自监督模块将同一表情的不同视角样本聚集在特征空间中，而监督模块则将不同表情的特征分开。

关键创新：本文的主要创新在于结合自监督和监督对比损失，显著提高了模型对不同视角的鲁棒性，并引入Barlow twins损失以减少特征冗余。与现有方法相比，ViewFX在特征学习上更为全面。

关键设计：在损失函数设计上，采用自监督对比损失和监督对比损失的组合，同时引入Barlow twins损失以优化特征表示的相关性。此外，模型的参数设置经过详细调优，以确保在不同数据集上的最佳性能。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在KDEF和DDCF数据集上的实验结果显示，ViewFX框架在面部表情识别任务中设立了新的最先进水平，识别准确率分别提高了X%和Y%。该方法在面对挑战性视角时表现出显著的鲁棒性，且对训练标签数量的敏感性明显降低。

🎯 应用场景

该研究的潜在应用领域包括人机交互、情感计算和安全监控等。通过提高面部表情识别的准确性，ViewFX可以在社交机器人、虚拟助手和情感分析等场景中发挥重要作用，未来可能推动相关技术的广泛应用。

📄 摘要（原文）

Although there has been much progress in the area of facial expression recognition (FER), most existing methods suffer when presented with images that have been captured from viewing angles that are non-frontal and substantially different from those used in the training process. In this paper, we propose ViewFX, a novel view-invariant FER framework based on contrastive learning, capable of accurately classifying facial expressions regardless of the input viewing angles during inference. ViewFX learns view-invariant features of expression using a proposed self-supervised contrastive loss which brings together different views of the same subject with a particular expression in the embedding space. We also introduce a supervised contrastive loss to push the learnt view-invariant features of each expression away from other expressions. Since facial expressions are often distinguished with very subtle differences in the learned feature space, we incorporate the Barlow twins loss to reduce the redundancy and correlations of the representations in the learned representations. The proposed method is a substantial extension of our previously proposed CL-MEx, which only had a self-supervised loss. We test the proposed framework on two public multi-view facial expression recognition datasets, KDEF and DDCF. The experiments demonstrate that our approach outperforms previous works in the area and sets a new state-of-the-art for both datasets while showing considerably less sensitivity to challenging angles and the number of output labels used for training. We also perform detailed sensitivity and ablation experiments to evaluate the impact of different components of our model as well as its sensitivity to different parameters.