ASDF: Assembly State Detection Utilizing Late Fusion by Integrating 6D Pose Estimation

📄 arXiv: 2403.16400v3 📥 PDF

作者: Hannah Schieber, Shiyu Li, Niklas Corell, Philipp Beckerle, Julian Kreimeier, Daniel Roth

分类: cs.CV, cs.RO

发布日期: 2024-03-25 (更新: 2024-08-09)

💡 一句话要点

提出ASDF以解决组装状态检测中的6D姿态估计问题

🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics)

关键词: 6D姿态估计 组装状态检测 YOLOv8 实时物体检测 动态场景 晚期融合 增强现实

📋 核心要点

  1. 现有的6D姿态估计方法主要关注单个物体，难以应对动态组装场景中的遮挡和外观变化问题。
  2. ASDF方法通过扩展YOLOv8框架，结合姿态信息与状态检测，提出了Pose2State模块以提高组装状态检测的精度。
  3. 实验结果显示，ASDF在GBOT数据集上超越了纯深度学习网络，且在组装状态检测和6D姿态估计上均有显著提升。

📝 摘要（中文）

在医疗和工业领域，提供组装过程的指导至关重要，以确保效率和安全。组装中的错误可能导致手术时间延长或工业制造和维护时间增加。现有的6D姿态估计技术主要集中在单个物体和静态捕捉上，而组装场景具有动态性，包括组装过程中的遮挡和物体外观的变化。为了解决6D姿态估计与组装状态检测的挑战，本文提出了ASDF方法，基于YOLOv8框架，融合姿态知识与网络检测的姿态信息，从而实现精确的组装状态预测。实验结果表明，ASDF在组装状态检测和6D姿态估计方面均有显著提升。

🔬 方法详解

问题定义：本文旨在解决在动态组装场景中进行6D姿态估计与组装状态检测的挑战。现有方法多集中于静态物体，无法有效处理组装过程中的遮挡和外观变化。

核心思路：ASDF方法通过结合YOLOv8的实时物体检测能力，提出Pose2State模块，将姿态估计与状态检测相结合，从而提高组装状态的预测精度。

技术框架：该方法的整体架构包括物体检测、姿态估计和状态检测三个主要模块。Pose2State模块负责融合检测到的姿态信息与真实姿态，从而实现更精确的状态预测。

关键创新：ASDF的核心创新在于其晚期融合策略，通过将姿态知识与网络检测的信息进行结合，显著提高了组装状态检测的准确性，并增强了姿态估计的鲁棒性。

关键设计：在技术细节上，ASDF采用了YOLOv8的网络结构，结合特定的损失函数以优化姿态与状态的融合过程，同时对网络参数进行了精细调整，以适应动态组装场景的需求。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在GBOT数据集上的实验结果显示，ASDF方法在组装状态检测中相较于纯深度学习网络有显著提升，且在姿态估计的鲁棒性方面也表现出色，具体提升幅度未知。

🎯 应用场景

该研究的潜在应用领域包括医疗手术指导、工业组装和维护等场景。通过提供实时的组装状态反馈，ASDF能够显著提高组装效率和安全性，减少错误发生的概率，具有重要的实际价值和广泛的应用前景。

📄 摘要（原文）

In medical and industrial domains, providing guidance for assembly processes can be critical to ensure efficiency and safety. Errors in assembly can lead to significant consequences such as extended surgery times and prolonged manufacturing or maintenance times in industry. Assembly scenarios can benefit from in-situ augmented reality visualization, i.e., augmentations in close proximity to the target object, to provide guidance, reduce assembly times, and minimize errors. In order to enable in-situ visualization, 6D pose estimation can be leveraged to identify the correct location for an augmentation. Existing 6D pose estimation techniques primarily focus on individual objects and static captures. However, assembly scenarios have various dynamics, including occlusion during assembly and dynamics in the appearance of assembly objects. Existing work focus either on object detection combined with state detection, or focus purely on the pose estimation. To address the challenges of 6D pose estimation in combination with assembly state detection, our approach ASDF builds upon the strengths of YOLOv8, a real-time capable object detection framework. We extend this framework, refine the object pose, and fuse pose knowledge with network-detected pose information. Utilizing our late fusion in our Pose2State module results in refined 6D pose estimation and assembly state detection. By combining both pose and state information, our Pose2State module predicts the final assembly state with precision. The evaluation of our ASDF dataset shows that our Pose2State module leads to an improved assembly state detection and that the improvement of the assembly state further leads to a more robust 6D pose estimation. Moreover, on the GBOT dataset, we outperform the pure deep learning-based network and even outperform the hybrid and pure tracking-based approaches.