Vision-Language-Action or World-Action? Just Action Models

📄 arXiv: 2610.37219v1 📥 PDF

作者: Yonghao Yu, Lang Huang, Runyi Li, Toshihiko Yamasaki

分类: cs.CV, cs.RO

发布日期: 2026-10-27

🔗 代码/项目: PROJECT_PAGE

💡 一句话要点

提出Just Action Model以解决控制策略知识继承问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 视觉-语言-动作 世界-动作模型 控制策略 多模态学习 机器人控制

📋 核心要点

  1. 现有的VLA和WAM模型在控制策略知识继承上存在争议，导致方法选择不明确。
  2. 提出的Just Action Model（JAM）通过解耦关键选择，提供了一个通用的框架，简化了训练过程。
  3. 实验结果显示，JAM在多个基准数据集上表现优异，成功率显著提升，并减少了在线训练的计算复杂度。

📝 摘要（中文）

本文探讨了视觉-语言-动作（VLA）与世界-动作模型（WAM）社区之间的争论，是否控制策略应从视觉-语言模型或视觉生成器中继承预训练知识。通过解耦背后的初始化、未来观察的目标以及推理时的机制，提出了一种通用框架Just Action Model（JAM）。该模型使用预训练的视觉骨干和动作专家，通过动作损失和前瞻损失进行训练，且在训练和推理中不涉及视觉生成。实验结果表明，JAM在LIBERO和LIBERO-Plus数据集上超越了FastWAM和ImageWAM，且在RoboTwin上也取得了相似或更高的成功率。

🔬 方法详解

问题定义：本文旨在解决控制策略在视觉-语言模型与视觉生成器之间知识继承的争论，现有方法未能明确区分不同选择的影响。

核心思路：提出Just Action Model（JAM），通过解耦背后的初始化、目标和推理机制，构建一个通用框架，避免视觉生成的依赖。

技术框架：JAM包括一个预训练的视觉骨干和一个动作专家，使用动作损失和前瞻损失进行训练。前瞻损失用于训练当前观察特征预测未来的冻结编码，训练后丢弃预测头。

关键创新：JAM的创新在于其解耦设计，使得控制策略不再依赖于视觉生成，且可以自由选择骨干网络，提升了灵活性和适应性。

关键设计：JAM的训练过程中，使用了特定的损失函数来优化动作和前瞻预测，确保在不增加推理复杂度的情况下，提升控制性能。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，JAM在LIBERO和LIBERO-Plus数据集上成功率分别提升至98.2%和64.5%，相较于FastWAM显著提高。同时，JAM在在线训练中每个样本的计算复杂度降低了73%，展示了其高效性。

🎯 应用场景

该研究的潜在应用领域包括机器人控制、自动驾驶和智能家居等场景，能够有效提升系统的决策能力和响应速度。未来，JAM框架可能推动多模态学习和智能系统的进一步发展，促进更复杂任务的自动化处理。

📄 摘要（原文）

The vision-language-action (VLA) and world-action model (WAM) communities debate whether control policies should inherit pretrained knowledge from vision-language models or from visual generators. The debate, however, bundles three separable choices: backbone initialization, the objective applied to future observations, and whether the machinery for that future loss is kept at inference. A gap between a VLA and a WAM therefore cannot simply be attributed to any one of them. We show that decoupling these choices yields a common framework for both families, which we term Just Action Model (JAM): a pretrained visual backbone with an action expert, trained with an action loss and a foresight loss. The foresight loss trains current-observation features to predict frozen encodings of the recorded future through a prediction head discarded after training, so no visual generation is involved in either training or inference, and the backbone can be chosen freely. We first compare objectives within fixed policy architectures, then evaluate the foresight recipe across backbone families. The resulting JAM policies outperform their matched FastWAM and ImageWAM counterparts on LIBERO and LIBERO-Plus, and achieve similar or slightly higher success rates on RoboTwin. Compared with FastWAM, JAM improves the success rates from 97.0%/55.3%/91.2% to 98.2%/64.5%/91.4%, respectively. With cached foresight targets, JAM reduces estimated online training FLOPs per sample per update by 73% relative to FastWAM. Our results show that future supervision improves control without visual generation or additional inference machinery, and strong foresight performance is attainable with either generator-initialized or vision-language backbones. Project Page: https://y0uroy.github.io/JAM