Deep Multimodal Wearable Sensor Fusion for Detection of Body-Focused Repetitive Behaviors

📄 arXiv: 2608.09830v1 📥 PDF

作者: Samaneh Rezaeimanesh, Mohsen Behradfar, Mohammad Fili, Guiping Hu

分类: cs.LG

发布日期: 2026-08-10

💡 一句话要点

提出深度多模态传感器融合方法以检测身体专注重复行为

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多模态融合 深度学习 行为检测 可穿戴传感器 心理健康 强迫症 数据融合

📋 核心要点

  1. 现有方法难以早期、客观地检测身体专注重复行为，因其动作微妙且与普通手势重叠。
  2. 本文提出了一种多模态深度学习框架，结合多种传感器数据以提高检测准确性。
  3. 实验结果显示，该框架在二分类检测中F1分数为0.985，九分类方案下宏平均F1分数为0.700，显著提升了检测性能。

📝 摘要（中文）

身体专注重复行为，如拔头发和抠皮，是与强迫症和焦虑症相关的强迫性运动行为。由于这些动作微妙且与普通非病理性手势重叠，早期客观检测仍然困难。本文开发并评估了一种多模态深度学习框架，通过手腕佩戴的传感器数据检测和分类这些行为。数据由Child Mind Institute使用Helios设备收集，结合了惯性测量单元、热电堆传感器和飞行时间传感器，捕获运动学、热量和接近信息。该框架结合了卷积神经网络和门控递归单元，利用特定模态的自编码器和后融合分类器，充分利用时间和空间动态。结果显示，该方法在二分类检测中F1分数达到0.985，ROC曲线下面积为0.997，显著优于单模态基线。

🔬 方法详解

问题定义：本文旨在解决身体专注重复行为的早期检测问题，现有方法由于动作微妙且与普通手势重叠，导致检测困难。

核心思路：提出了一种多模态深度学习框架，通过融合不同类型的传感器数据，充分利用时间和空间动态信息，以提高检测的准确性和客观性。

技术框架：整体架构包括卷积神经网络（CNN）、门控递归单元（GRU）、模态特定自编码器和后融合分类器，能够处理来自不同传感器的多模态数据。

关键创新：最重要的创新在于将多种传感器数据进行融合，利用深度学习模型捕获复杂的时空特征，从而显著提高了检测精度，与传统单模态方法相比具有本质区别。

关键设计：在网络结构上，采用了卷积神经网络和门控递归单元的组合，损失函数设计为适应多类分类任务，确保模型能够有效学习各类行为的特征。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，该框架在二分类检测中F1分数达到0.985，ROC曲线下面积为0.997，九分类方案下宏平均F1分数为0.700，曲线下面积为0.963，显著优于单模态基线，证明了多模态传感器融合的有效性。

🎯 应用场景

该研究的潜在应用领域包括心理健康监测、临床干预和个性化治疗方案的制定。通过实时、可穿戴的行为监测，能够为精神健康诊断提供客观依据，推动生物医学研究和临床护理的发展。

📄 摘要（原文）

Body-focused repetitive behaviors, such as hair pulling and skin picking, are compulsive motor actions commonly associated with obsessive-compulsive and anxiety disorders. Their early, objective detection remains difficult because the movements are subtle and overlap with ordinary, non-pathological gestures. We developed and evaluated a multimodal deep learning framework to detect and classify these behaviors from wrist-worn sensor data. The data, collected by the Child Mind Institute using the Helios wrist-worn device, combine inertial measurement units, thermopile sensors, and time-of-flight sensors, capturing kinematic, thermal, and proximity information. The framework combined a convolutional neural network with a gated recurrent unit, alongside modality-specific autoencoders and a late-fusion classifier, to exploit temporal and spatial dynamics. It achieved an F1 score of 0.985 and an area under the receiver operating characteristic curve of 0.997 for binary detection, distinguishing these behaviors from other activities, and a macro-averaged F1 score of 0.700 with an area under the curve of 0.963 across a nine-class scheme that distinguished each individual behavior from a single grouped Non-Target class, improving over single-modality baselines. Post-hoc interpretability based on Shapley additive explanations showed that the time-of-flight and inertial modalities dominated discriminative power by capturing spatial proximity and dynamic movement, while hierarchical clustering indicated that misclassifications were driven primarily by the anatomical region of the gesture. These findings demonstrate that multimodal sensor fusion enables accurate, objective, and continuous behavioral monitoring. This work establishes a foundation for real-time, wearable-assisted mental health diagnostics and personalized interventions in biomedical research and clinical care.