Adapt for Planning, Not just Prediction: Optimistic Test-Time World Model Adaptation

📄 arXiv: 2610.35636v1 📥 PDF

作者: Akshay Mete, Dileep Kalathil, P. R. Kumar

分类: cs.LG, cs.AI, cs.RO

发布日期: 2026-10-26

备注: Accepted at the CoRL 2026 Workshop on Pretrain to Adapt

💡 一句话要点

提出乐观测试时世界模型适应以解决规划精度下降问题

🎯 匹配领域: 支柱一：机器人控制 (Robot Control) 支柱二：RL算法与架构 (RL & Architecture)

关键词: 测试时适应 世界模型 规划优化 机器人控制 动态环境

📋 核心要点

  1. 现有的潜在世界模型在测试时保持不变，导致在分布转移情况下规划性能下降。
  2. 本文提出的乐观测试时适应（OpTTA）通过引入规划者的目标达成成本，优化模型适应过程。
  3. 实验结果显示，OpTTA在多种任务中平均提高了13.5个百分点的规划成功率，相较于传统方法有显著提升。

📝 摘要（中文）

潜在世界模型通常在离线预训练后保持不变，这导致在测试时规划性能在分布转移下可能下降。测试时适应（TTA）通过在部署期间更新模型来缓解这种下降。与固定数据的预训练不同，规划的TTA是闭环的：规划者使用模型的展望选择动作，而结果转移驱动下一次更新。本文提出乐观测试时适应（OpTTA），其目标包括观察到的转移的预测损失和规划者在候选展望上的目标达成成本，二者均在模型参数上最小化。实验表明，OpTTA在多种任务中相较于仅基于预测的目标（AdaJEPA）平均提高了13.5个百分点的最终规划成功率。

🔬 方法详解

问题定义：本文旨在解决潜在世界模型在测试时由于分布转移导致的规划性能下降问题。现有方法在适应过程中仅关注预测准确性，未能有效指导规划。

核心思路：提出乐观测试时适应（OpTTA），通过结合预测损失和规划者目标达成成本，优化模型适应过程，使其在规划时更具乐观性。

技术框架：OpTTA的整体架构包括两个主要模块：一是基于观察到的转移计算的预测损失，二是基于规划者候选展望的目标达成成本。模型在这两个目标下进行参数优化。

关键创新：OpTTA的创新在于引入了规划者的目标达成成本，使得模型在适应过程中不仅关注预测准确性，还能优化规划性能。这一设计与传统的仅基于预测的适应方法存在本质区别。

关键设计：在损失函数中，OpTTA同时考虑了预测损失和目标达成成本，确保模型在适应过程中能够保持对目标的乐观态度。实验中使用了不同的编码器和训练目标，验证了方法的有效性。

📊 实验亮点

实验结果显示，OpTTA在视觉和环境转移的操作、到达和导航任务中，平均提高了13.5个百分点的最终规划成功率，相较于仅基于预测的目标（AdaJEPA）提升了6.9个百分点。当编码器也进行适应时，成功率分别为13.5和7.5个百分点，显示出方法的有效性和优势。

🎯 应用场景

该研究的潜在应用领域包括机器人控制、自动驾驶和智能制造等场景，能够有效提升系统在动态环境中的适应能力和决策效率。未来，OpTTA可能为多种智能系统提供更为灵活和高效的规划能力，推动自主系统的发展。

📄 摘要（原文）

Latent world models are typically pre-trained offline and kept frozen at test time, so planning with them can degrade under distribution shift. Test-time adaptation (TTA) can mitigate this degradation by updating the model during deployment. Unlike pre-training on fixed data, TTA for planning is closed-loop: the planner uses model rollouts to select actions, and the resulting transitions drive the next update. This raises the question: which objective should guide TTA for planning? Under a prediction-only objective, the model is fit only to the outcomes of its own actions, so it can remain wrong about the actions that would reach the goal, and low prediction error need not translate into better planning. Inspired by reward-biased maximum likelihood from adaptive control, we introduce Optimistic Test-Time Adaptation (OpTTA). Its objective has two terms, the prediction loss on observed transitions and the planner's goal-reaching cost on its candidate rollouts, both minimized over the model parameters. The planning term biases adaptation toward models that are optimistic about reaching the goal, without any additional environment interaction. Across visual and environment shifts on manipulation, reaching and navigation tasks, OpTTA raises final planning success over the frozen model by 13.5 percentage points on average, compared with 6.9 for the prediction-only objective (AdaJEPA), when only the predictor is adapted. When the encoder is adapted as well, the gains are 13.5 and 7.5 points. These experiments span three JEPA world models with different encoders and training objectives and, for temporal straightening, both CEM and gradient-based planners. Our results suggest that test-time adaptation of world models for planning benefits from an optimistic, task-aware term in addition to the prediction loss.