How Suboptimal is Training rPPG Models with Videos and Targets from Different Body Sites?

📄 arXiv: 2403.10582v1 📥 PDF

作者: Björn Braun, Daniel McDuff, Christian Holz

分类: eess.IV, cs.LG

发布日期: 2024-03-15

💡 一句话要点

提出使用不同身体部位PPG信号训练rPPG模型以提高准确性

🎯 匹配领域: 支柱八：物理动画 (Physics-based Animation)

关键词: 光电容积脉搏 rPPG 深度学习 心血管监测 数据集比较 模型训练 信号处理

📋 核心要点

  1. 现有的rPPG模型大多依赖于手指接触PPG作为标签，缺乏面部PPG数据集，导致训练效果受限。
  2. 本文提出利用同步的手部和面部PPG信号数据集，探讨不同身体部位PPG信号对rPPG模型训练的影响。
  3. 实验结果表明，使用额头PPG信号训练的模型在波形预测上均方误差降低了40%，且更好地捕捉了真实信号的形态特征。

📝 摘要（中文）

远程相机通过光电容积脉搏（rPPG）测量血容量脉搏是一项具有吸引力的技术，能够以低成本和可扩展性评估心血管信息。当前大多数模型使用手指接触PPG作为标签，训练于面部视频。然而，不同身体部位的PPG信号在形态特征上存在显著差异。本文利用一个新发布的数据集，比较了使用手部和面部PPG信号训练rPPG模型的效果，发现使用额头PPG信号训练的模型在波形预测上表现出更低的均方误差，且能够更好地学习真实PPG信号的形态特征。尽管手指PPG训练的模型仍能有效学习心率等主频信息。

🔬 方法详解

问题定义：本文旨在探讨使用不同身体部位的PPG信号（如手指与额头）训练rPPG模型的效果，现有方法主要依赖手指PPG信号作为标签，可能导致训练效果不佳。

核心思路：通过使用一个包含手部和面部PPG信号的独特数据集，比较不同部位PPG信号对模型训练的影响，从而验证训练数据来源对模型性能的关键作用。

技术框架：研究采用了深度神经网络模型，首先收集同步的PPG信号和视频数据，然后进行模型训练，最后评估模型在不同PPG信号上的表现。主要模块包括数据采集、模型训练和性能评估。

关键创新：本文的创新在于首次系统性地比较了不同身体部位PPG信号对rPPG模型训练的影响，揭示了训练数据来源对模型性能的显著影响。

关键设计：在模型训练中，采用了均方误差作为损失函数，网络结构基于当前最先进的深度学习架构，确保模型能够有效学习PPG信号的时序特征。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，使用额头PPG信号训练的模型在波形预测上均方误差降低了40%，相比于使用手指PPG信号的模型，表现出更高的准确性和更好的信号形态捕捉能力。尽管手指PPG训练的模型仍能有效学习心率等主频信息，但在波形重建上存在明显不足。

🎯 应用场景

该研究为rPPG技术在心血管健康监测中的应用提供了新的思路，尤其是在远程医疗和可穿戴设备领域。通过优化训练数据来源，可以提高模型的准确性和可靠性，推动rPPG技术的普及和应用。

📄 摘要（原文）

Remote camera measurement of the blood volume pulse via photoplethysmography (rPPG) is a compelling technology for scalable, low-cost, and accessible assessment of cardiovascular information. Neural networks currently provide the state-of-the-art for this task and supervised training or fine-tuning is an important step in creating these models. However, most current models are trained on facial videos using contact PPG measurements from the fingertip as targets/ labels. One of the reasons for this is that few public datasets to date have incorporated contact PPG measurements from the face. Yet there is copious evidence that the PPG signals at different sites on the body have very different morphological features. Is training a facial video rPPG model using contact measurements from another site on the body suboptimal? Using a recently released unique dataset with synchronized contact PPG and video measurements from both the hand and face, we can provide precise and quantitative answers to this question. We obtain up to 40 % lower mean squared errors between the waveforms of the predicted and the ground truth PPG signals using state-of-the-art neural models when using PPG signals from the forehead compared to using PPG signals from the fingertip. We also show qualitatively that the neural models learn to predict the morphology of the ground truth PPG signal better when trained on the forehead PPG signals. However, while models trained from the forehead PPG produce a more faithful waveform, models trained from a finger PPG do still learn the dominant frequency (i.e., the heart rate) well.