COVTrack++: Learning Open-Vocabulary Multi-Object Tracking from Continuous Videos via a Synergistic Paradigm

📄 arXiv: 2603.24016 📥 PDF

作者: Zekun Qian, Wei Feng, Ruize Han, Junhui Hou

分类: cs.CV, cs.LG

发布日期: 2026-07-05

💡 一句话要点

提出COVTrack++以解决开放词汇多目标跟踪问题

🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics) 支柱七：动作重定向 (Motion Retargeting)

关键词: 开放词汇跟踪 多目标跟踪 连续注释 协同框架 运动动态

📋 核心要点

  1. 现有的多目标跟踪方法通常只关注特定类别，限制了其在多样化场景中的应用。
  2. 本文提出COVTrack++框架，通过构建C-TAO数据集和三个模块实现检测与关联的协同机制。
  3. 在TAO数据集上的实验结果显示，COVTrack++在新颖的关联和定位任务上分别提升了4.8%和5.8%。

📝 摘要（中文）

多目标跟踪（MOT）传统上关注特定类别，限制了其在多样化物体场景中的应用。开放词汇多目标跟踪（OVMOT）通过支持跟踪训练期间未见的新物体来解决这一问题。然而，当前进展受到两个挑战的制约：缺乏连续注释的视频数据和定制的OVMOT框架。为了解决数据瓶颈，本文构建了C-TAO，这是第一个连续注释的OVMOT训练集，注释密度提高了26倍，捕捉了平滑的运动动态和中间物体状态。为了解决框架瓶颈，提出了COVTrack++，一个通过三个模块实现检测与关联之间双向互惠机制的协同框架。实验结果表明，COVTrack++在TAO数据集上实现了最先进的性能，验证集和测试集的TETA分别达到了35.4%和30.5%。

🔬 方法详解

问题定义：本文旨在解决开放词汇多目标跟踪（OVMOT）中的数据和框架瓶颈。现有方法缺乏连续注释的视频数据，且未能有效整合检测与关联过程。

核心思路：通过构建C-TAO数据集和提出COVTrack++框架，本文实现了检测与关联的协同工作，提升了跟踪的准确性和鲁棒性。

技术框架：COVTrack++框架包含三个主要模块：多线索自适应融合（MCF）、多粒度层次聚合（MGA）和时间置信传播（TCP），实现了检测与关联的双向互惠机制。

关键创新：最重要的创新在于C-TAO数据集的构建和COVTrack++框架的设计，尤其是MCF和MGA模块的引入，使得模型能够更好地处理复杂的物体状态和运动动态。

关键设计：在设计中，MCF模块动态平衡外观、运动和语义线索，MGA模块利用层次空间关系增强特征，TCP模块通过高置信度物体提升低置信度候选物体的稳定性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

COVTrack++在TAO数据集上实现了最先进的性能，验证集和测试集的TETA分别达到了35.4%和30.5%。此外，相较于之前的方法，新的关联精度提高了4.8%，定位精度提高了5.8%，展现了强大的零样本泛化能力。

🎯 应用场景

该研究的潜在应用领域包括智能监控、自动驾驶、无人机监控等场景，能够有效跟踪多种类别的物体，提升系统的智能化水平。未来，该技术可能在复杂环境下的实时跟踪任务中发挥重要作用。

📄 摘要（原文）

Multi-Object Tracking (MOT) has traditionally focused on a few specific categories, restricting its applicability to real-world scenarios involving diverse objects. Open-Vocabulary Multi-Object Tracking (OVMOT) addresses this by enabling tracking of arbitrary categories, including novel objects unseen during training. However, current progress is constrained by two challenges: the lack of continuously annotated video data for training, and the lack of a customized OVMOT framework to synergistically handle detection and association. We address the data bottleneck by constructing C-TAO, the first continuously annotated training set for OVMOT, which increases annotation density by 26x over the original TAO and captures smooth motion dynamics and intermediate object states. For the framework bottleneck, we propose COVTrack++, a synergistic framework that achieves a bidirectional reciprocal mechanism between detection and association through three modules: (1) Multi-Cue Adaptive Fusion (MCF) dynamically balances appearance, motion, and semantic cues for association feature learning; (2) Multi-Granularity Hierarchical Aggregation (MGA) exploits hierarchical spatial relationships in dense detections, where visible child nodes (e.g., object parts) assist occluded parent objects (e.g., whole body) for association feature enhancement; (3) Temporal Confidence Propagation (TCP) recovers flickering detections through high-confidence tracked objects boosting low-confidence candidates across frames, stabilizing trajectories. Extensive experiments on TAO demonstrate state-of-the-art performance, with novel TETA reaching 35.4% and 30.5% on validation and test sets, improving novel AssocA by 4.8% and novel LocA by 5.8% over previous methods, and show strong zero-shot generalization on BDD100K.