Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising

📄 arXiv: 2607.04653v1 📥 PDF

作者: Guangting Zheng, Haojing Chen, Hao Li, Jingtao Zhang, Zhen Yang, Xiaosong Jia, Xue Yang, Shaofeng Zhang, Yanyong Zhang

分类: cs.CV

发布日期: 2026-07-06

🔗 代码/项目: PROJECT_PAGE

💡 一句话要点

提出VPT框架以解决视频物理一致性问题

🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics)

关键词: 视频扩散模型 物理一致性 模态解耦 角色感知 去噪策略 多模态学习 视频生成

📋 核心要点

  1. 现有视频扩散模型在视觉质量上表现优异，但在长距离物理一致性方面存在不足，尤其是无法有效捕捉不同实体的运动模式。
  2. 本文提出VPT框架，通过角色感知信号对实体进行分组，并采用模态解耦去噪策略，使得辅助模态作为软约束，减少推理中的错误积累。
  3. 实验结果显示，VPT在VideoPhy基准测试中显著提升了物理一致性，同时保持了视觉质量，取得了显著的性能提升。

📝 摘要（中文）

现代视频扩散模型在视觉保真度方面表现出色，但在维持长距离物理一致性方面仍面临巨大挑战。传统的像素重建目标主要关注外观细节，往往无法捕捉场景的动态特征。为此，本文提出了一种名为VPT的微调框架，通过角色感知信号将实体分组为不同的物理角色，并引入模态解耦去噪策略，以减轻推理过程中的递归预测误差。实验结果表明，VPT在VideoPhy基准测试中相较于Wan2.1-T2V-1.3B在SA和PC指标上分别提升了39.4%和17.9%。

🔬 方法详解

问题定义：本文旨在解决视频扩散模型在长距离物理一致性方面的不足，现有方法未能有效区分不同实体的运动模式，且辅助模态在推理过程中可能导致错误积累。

核心思路：提出VPT框架，通过角色感知信号将实体分为不同的物理角色，并采用模态解耦去噪策略，使得视觉和辅助模态之间的依赖关系减弱，从而提高物理一致性。

技术框架：VPT框架包括角色感知信号模块、模态解耦去噪模块和损失权重衰减策略。角色感知信号模块负责将实体分类，模态解耦去噪模块则为视觉和辅助通道分配独立的噪声水平。

关键创新：VPT的主要创新在于角色感知信号的引入，使得不同物理角色的建模更加清晰，同时模态解耦去噪策略有效减轻了推理中的递归预测错误。

关键设计：在损失函数设计上，采用了损失权重衰减策略，使得辅助模态作为软约束而非强依赖，降低了模型的复杂性和错误传播风险。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，VPT在VideoPhy基准测试中相较于Wan2.1-T2V-1.3B在SA和PC指标上分别提升了39.4%和17.9%，并在VideoPhy-2基准测试中也表现出一致的改进，验证了其有效性。

🎯 应用场景

该研究的潜在应用领域包括视频生成、增强现实和虚拟现实等场景，能够为这些领域提供更高的物理一致性和视觉质量，提升用户体验。未来，该框架可扩展至其他多模态学习任务，推动相关技术的发展。

📄 摘要（原文）

While modern video diffusion models excel in visual fidelity, maintaining long-range physical consistency remains a formidable challenge. Conventional pixel-reconstruction objectives mainly focus on appearance details and often fail to capture the underlying dynamics of a scene. To mitigate this, recent efforts have integrated auxiliary modalities (e.g., optical flow) to introduce physics priors via joint training with video appearance. However, these methods have three main limitations: (1) they do not distinguish the different motion patterns of different entity types; (2) joint modeling of visual and auxiliary modalities can cause capacity conflicts and weaken the pretrained visual prior; and (3) auxiliary modalities may accumulate errors during inference. To address these issues, we propose \textbf{VPT}, a fine-tuning framework for improving physical consistency in video diffusion models. VPT introduces a role-aware signal that groups entities into agents, controlled objects, passive objects, and background, so that different physical roles can be modeled more clearly. We further propose a modality-decoupled denoising strategy, where the visual and auxiliary channels are assigned independent noise levels. Together with a loss-weight decay strategy, this design makes auxiliary modalities serve as soft constraints rather than strong dependencies, mitigating recursive prediction errors during inference. We also introduce cross-step auto-guidance to further strengthen physical dynamics. Experiments show that VPT improves physical consistency while preserving visual quality, achieving relative gains of 39.4\% in SA and 17.9\% in PC on VideoPhy benchmark over Wan2.1-T2V-1.3B, and consistent improvements on VideoPhy-2 benchmark. The project page is available at https://tom-zgt.github.io/VPT.