Rethinking Demonstration Unlearning in Imitation Learning for Robotics
作者: Jiazhuo Li, Yu Zhang, Yiming Fei, Kangkang Dong, Xiaojun Zhu, Houde Liu, Jinze Tao
分类: cs.RO, cs.LG
发布日期: 2026-08-21
备注: 21 pages, 7 figures, 14 tables
💡 一句话要点
提出审计机制以解决模仿学习中的示范遗忘问题
🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)
关键词: 模仿学习 示范遗忘 机器人技术 策略编辑 行为评估 证据检测 重训练机制
📋 核心要点
- 现有模仿学习方法在删除人类示范后，重训练成本高，难以实现高效的示范遗忘。
- 本文提出了一种重训练校准审计机制，通过行为和证据两个维度评估策略编辑的有效性。
- 在五个预注册条件下的实验中，编辑后的策略在行为和证据上表现出不同的变化，验证了方法的有效性。
📝 摘要（中文）
模仿学习在机器人领域依赖于人类示范，但有时需要删除这些示范。传统的重训练方法成本高昂，尤其在策略和数据集规模增大时。本文提出了一种新的审计机制，评估编辑后的策略是否与重训练策略一致，并检测其是否仍然包含删除示范的证据。通过行为和证据两个维度进行评估，实验结果表明，该方法在多个真实机器人和仿真环境中有效，能够在不显著增加成本的情况下实现示范的有效遗忘。
🔬 方法详解
问题定义：本文旨在解决模仿学习中删除人类示范后的重训练成本高的问题。现有方法无法有效评估编辑后的策略是否真正实现了示范的遗忘，且缺乏对策略行为和证据的全面分析。
核心思路：提出了一种重训练校准审计机制，分别从行为和证据两个维度评估策略的编辑效果。行为维度关注编辑后的策略在相同状态下的动作偏差，而证据维度则通过成员攻击检测策略是否仍然包含删除的示范信息。
技术框架：整体流程包括两个主要模块：行为评估和证据评估。行为评估通过与独立重训练的比较，建立一个基准，评估编辑策略的动作偏差；证据评估则通过对每个示范进行成员攻击，报告其排名和绝对成员损失水平。
关键创新：最重要的创新在于引入了重训练校准审计机制，通过两个独立的评估维度，全面分析策略编辑的效果。这种方法与传统的单一评估方法有本质区别，能够更准确地反映策略的真实表现。
关键设计：在行为评估中，使用独立重训练建立基准，确保评估的准确性；在证据评估中，采用每个示范的成员攻击，确保能够检测到策略是否仍然包含删除的示范信息。
🖼️ 关键图片
📊 实验亮点
在ACT臂的实验中，通过重定向编辑，成功将盲评分机器人成功率恢复至20次试验中的18次，显示出该方法在实际应用中的有效性和优越性。
🎯 应用场景
该研究在机器人领域具有广泛的应用潜力，尤其是在需要动态调整和优化机器人行为的场景中。通过有效的示范遗忘机制，机器人可以更灵活地适应新的任务需求，提升其自主学习能力，未来可能在智能制造、服务机器人等领域发挥重要作用。
📄 摘要（原文）
Imitation learning for robotics depends on human demonstrations, some of which people may later ask to remove. Retraining without them is the natural reference, but its cost grows with policy and dataset scale, motivating cheaper operators that edit a trained policy. Metrics inherited from machine unlearning, such as forgetting loss or a single membership attack, do not establish what an edit removed from a policy acting in closed loop. We therefore introduce a retrain-calibrated audit that reads demonstration unlearning along two axes: behavior, whether the edited policy acts like one retrained without the removed demonstrations, and evidence, whether an auditor can still detect it was trained on them. The behavior axis measures action divergence to that retrain at matched states, calibrated by a floor built from independent retrains, so a policy at the floor is as close to a retrain as retrains are to each other. The evidence axis applies a per-demonstration membership attack against a retrain null, reporting both its rank and its absolute member-loss level, since rank alone accepts operators that inflate member losses past the null. A conformal test then combines both axes into one hypothesis of joint retrain consistency, against a fleet of independent retrains large enough to reject at conventional significance. Across five preregistered conditions on three real-robot policy classes and two simulation suites, the axes dissociate in both directions on one checkpoint, as an edit may repair task behavior while leaving evidence unchanged, or reduce evidence while moving behavior away from retraining. On the ACT arm, a redirect edit restores blind-scored robot success to 18 of 20 trials.