Terrain-Aware Stride-Level Trajectory Forecasting for a Powered Hip Exoskeleton via Vision and Kinematics Fusion

📄 arXiv: 2404.11945v1 📥 PDF

作者: Ruoqi Zhao, Xingbang Yan, Yubo Fan

分类: cs.RO

发布日期: 2024-04-18

备注: 6 pages, submitted to IEEE RA-L, under review. This work has been submitted to the IEEE Robotics and Automation Letters (RA-L) for possible publication

🔗 代码/项目: GITHUB

💡 一句话要点

提出Sandwich Fusion Transformer以解决地形适应性轨迹预测问题

🎯 匹配领域: 支柱一：机器人控制 (Robot Control) 支柱六：视频提取与匹配 (Video Extraction)

关键词: 动力外骨骼 轨迹预测 多模态融合 视觉信息 运动学 深度学习 地形适应性

📋 核心要点

  1. 现有动力外骨骼在复杂地形下的助力效果不佳，难以提供适应性支持。
  2. 提出的SFTIK网络通过融合视觉信息和运动学数据，预测髋关节角度，提升了预测精度。
  3. 实验结果显示，SFTIK在计算效率和预测准确性上均显著优于传统方法，具有实际应用价值。

📝 摘要（中文）

动力髋关节外骨骼在跑步机行走中已显示出助力能力，但在真实场景中应对变化地形的挑战仍然存在。本文分享了一个包含10名健康受试者在五种常见地形上行走的真实数据集，并设计了一种名为Sandwich Fusion Transformer for Image and Kinematics (SFTIK)的网络。该网络通过融合前后步态的地形图像和IMU时间序列，预测下一步的髋关节角度。实验结果表明，SFTIK在计算效率和预测准确性上均优于基线方法，具有良好的应用前景。

🔬 方法详解

问题定义：本文旨在解决动力髋关节外骨骼在变化地形下的轨迹预测问题。现有方法在处理复杂地形时，往往无法提供准确的助力轨迹，导致外骨骼的适应性不足。

核心思路：论文提出了一种新颖的网络结构SFTIK，通过融合前后步态的地形图像和IMU数据，来预测下一步的髋关节角度。这种设计旨在利用视觉信息提升预测的准确性。

技术框架：SFTIK的整体架构包括输入模块、特征提取模块和预测模块。输入模块接收地形图像和IMU时间序列，特征提取模块通过深度学习提取关键特征，最后预测模块输出髋关节角度。

关键创新：SFTIK的主要创新在于引入了宽度级别的patchify技术，专门针对自我中心的地形图像进行处理，从而降低计算需求，并显著提升了预测性能。

关键设计：在网络设计中，采用了特定的损失函数以优化预测精度，并设置了适当的超参数以平衡计算效率与预测准确性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，SFTIK在计算效率上达到了3.31 G Flops，预测的均方根误差(RMSE)为3.445 ± 0.804°，皮尔逊相关系数(PCC)为0.971 ± 0.025，显示出显著优于基线方法的性能提升。

🎯 应用场景

该研究的潜在应用领域包括康复机器人、助行器和智能外骨骼等，能够为不同地形下的行走提供精准的助力轨迹，提升用户的行走体验。未来，该技术有望在智能交通、机器人导航等领域发挥重要作用。

📄 摘要（原文）

Powered hip exoskeletons have shown the ability for locomotion assistance during treadmill walking. However, providing suitable assistance in real-world walking scenarios which involve changing terrain remains challenging. Recent research suggests that forecasting the lower limb joint's angles could provide target trajectories for exoskeletons and prostheses, and the performance could be improved with visual information. In this letter, We share a real-world dataset of 10 healthy subjects walking through five common types of terrain with stride-level label. We design a network called Sandwich Fusion Transformer for Image and Kinematics (SFTIK), which predicts the thigh angle of the ensuing stride given the terrain images at the beginning of the preceding and the ensuing stride and the IMU time series during the preceding stride. We introduce width-level patchify, tailored for egocentric terrain images, to reduce the computational demands. We demonstrate the proposed sandwich input and fusion mechanism could significantly improve the forecasting performance. Overall, the SFTIK outperforms baseline methods, achieving a computational efficiency of 3.31 G Flops, and root mean square error (RMSE) of 3.445 \textpm \ 0.804\textdegree \ and Pearson's correlation coefficient (PCC) of 0.971 \textpm\ 0.025. The results demonstrate that SFTIK could forecast the thigh's angle accurately with low computational cost, which could serve as a terrain adaptive trajectory planning method for hip exoskeletons. Codes and data are available at https://github.com/RuoqiZhao116/SFTIK.