HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models

📄 arXiv: 2608.13205v1 📥 PDF

作者: Jiazi Bu, Pengyang Ling, Yujie Zhou, Yibin Wang, Yuhang Zang, Xuanlang Dai, Shengyuan Ding, Tianyi Wei, Xiaohang Zhan, Jiaqi Wang, Tong Wu, Dahua Lin, Xingang Pan

分类: cs.CV

发布日期: 2026-08-13

备注: Project Website: https://bujiazi.github.io/hpsd.github.io/ Code: https://github.com/Bujiazi/HPSD

💡 一句话要点

提出HPSD以解决TI2V模型自蒸馏问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 文本-图像-视频 自蒸馏 多模态生成 深度学习 视频生成

📋 核心要点

  1. 现有的自蒸馏方法在监督策略上存在不足，无法有效利用教师生成的高质量样本进行模型训练。
  2. 本文提出的HPSD框架通过让同一模型在不同模式下进行自蒸馏，解决了传统方法的监督不匹配问题。
  3. 实验结果表明，HPSD显著提升了T2V性能，同时在TI2V生成上也取得了显著进展，增强了模型的基础生成能力。

📝 摘要（中文）

文本-图像-视频（TI2V）模型是一种新兴的统一架构，能够同时支持文本到视频（T2V）和图像到视频（I2V）生成。尽管在高质量的第一帧或详细的文本提示下，TI2V模型的视觉质量显著优于T2V模式，但如何将这种优势内化为模型的基础生成能力仍然是一个挑战。现有的自蒸馏方法存在监督策略不匹配的问题，导致模型无法有效学习。为此，本文提出了一种新的自蒸馏框架——混合策略自蒸馏（HPSD），通过让同一TI2V模型在不同条件下充当教师和学生，显著提升了T2V性能，并增强了模型的基础生成能力。

🔬 方法详解

问题定义：本文旨在解决TI2V模型在自蒸馏过程中，教师与学生之间的监督策略不匹配问题。现有方法往往依赖固定的离线分布，导致模型无法有效学习到动态生成策略。

核心思路：HPSD框架通过让同一TI2V模型在教师模式（高质量第一帧和增强提示）与学生模式（基础T2V模式）下进行自蒸馏，旨在吸收教师的优势，同时保持对学生策略的精确修正。

技术框架：HPSD的整体架构包括教师和学生两个角色。教师在TI2V模式下生成高质量样本，学生则在基础T2V模式下进行自我修正。学生从教师的轨迹中获取锚点，并对其进行局部优化，最终在自生成的样本上进行速度级别的监督。

关键创新：HPSD的创新在于将同一模型作为教师和学生，通过动态调整监督策略，克服了传统方法的静态监督限制，实现了更有效的自蒸馏。

关键设计：在HPSD中，学生继承教师的轨迹点作为锚点，并通过局部优化来调整这些点，损失函数设计为速度级别的监督，以确保学生生成的样本能够更好地反映教师的生成能力。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，HPSD在T2V任务上性能提升显著，相较于基线方法，T2V生成质量提高了XX%（具体数据未知），同时在TI2V生成上也取得了显著进展，验证了其有效性。

🎯 应用场景

该研究的潜在应用领域包括视频生成、内容创作和多模态学习等。通过提升TI2V模型的生成能力，HPSD可以在影视制作、游戏开发和虚拟现实等领域发挥重要作用，推动多模态技术的进一步发展。

📄 摘要（原文）

Text-Image-to-Video (TI2V) models are an emerging unified architecture, where a single model simultaneously supports text-to-video (T2V) and image-to-video (I2V) generation. Given a high-quality first frame or a detailed textual prompt, TI2V models unlock substantially better visual quality than their T2V mode, raising a natural question: can the capability elicited by such privileged conditions be internalized into the model's own base generation ability? A common approach toward this goal is model self-distillation. However, the most straightforward solution, supervised fine-tuning, follows an off-policy strategy: its supervision is confined to teacher-generated endpoints from a fixed offline distribution rather than student-visited states, lacking precise correction tailored to the evolving policy. Recent on-policy distillation methods instead suffer from condition-state mismatch, where supervision is steered toward the given first frame instead of the student's actual content, misleading the correction. To achieve self-distillation that absorbs the teacher's privileged prior while retaining precise policy correction, in this work, we propose Hybrid-Policy Self-Distillation (HPSD), a novel self-distillation framework where a single TI2V model acts as both teacher and student under different conditions: the teacher operates in TI2V mode with a high-quality first frame and an enhanced prompt, while the student runs in the base T2V mode with only the vanilla prompt. Specifically, the student inherits off-policy teacher trajectory points as anchors, locally refines them toward its own policy, and finally receives velocity-level supervision on these self-generated roll-outs. Extensive experiments demonstrate that HPSD significantly improves T2V performance while also delivering notable TI2V gains, effectively strengthening the model's base generation ability.