Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Models
作者: Jin Cui, Yanbin Hu, Xinyue Long, Linkai Li, Boran Zhao, Pengju Ren
分类: cs.RO
发布日期: 2026-08-03
备注: 13 pages, 7 figures
💡 一句话要点
提出AtVLA框架以解决视觉-语言-动作模型的空间感知问题
🎯 匹配领域: 支柱一：机器人控制 (Robot Control) 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 视觉-语言-动作 空间感知 机器人操作 注意力机制 多模态学习 深度学习 视觉编码器
📋 核心要点
- 现有的视觉-语言-动作模型在空间精确的机器人操作中表现不佳，尤其是在处理空间感知时存在注意力伪影。
- 论文提出的AtVLA框架通过引入可学习的注册令牌来改善视觉编码器的表现，旨在恢复空间信息和注意力分布。
- 在LIBERO、SimplerEnv和真实世界基准测试中，AtVLA的成功率从94.2%提升至98.4%，在真实场景中的成功率从46.5%提升至69.0%。
📝 摘要（中文）
视觉-语言-动作（VLA）模型的视觉表示在空间精确的机器人操作中仍然不可靠。我们发现，VLA中的视觉编码器存在与通用视觉变换器相似的注意力伪影，并且这些伪影与后训练获得的空间感知能力密切相关。为了解决这一问题，我们提出了AtVLA框架，通过在视觉编码器中插入可学习的注册令牌，专门承载空间信息，同时恢复干净的注意力分布，以实现精确的目标定位和细致的接触。实验结果表明，AtVLA在多个基准测试中显著提高了成功率。
🔬 方法详解
问题定义：本论文旨在解决视觉-语言-动作模型在空间感知中的不可靠性，现有方法在处理空间信息时容易出现注意力伪影，导致定位精度不足。
核心思路：论文提出的AtVLA框架通过在视觉编码器中插入可学习的注册令牌，专门用于承载空间信息，从而改善注意力分布的清晰度和准确性。
技术框架：AtVLA框架包括视觉编码器、注册令牌、注意力修正模块和不确定性门控局部细化模块。视觉编码器负责提取特征，注册令牌用于增强空间信息，注意力修正模块确保注意力分布的准确性。
关键创新：AtVLA的主要创新在于引入可学习的注册令牌，这些令牌专门用于承载空间信息，与传统方法相比，显著提高了空间感知的准确性和可靠性。
关键设计：在设计中，注册令牌通过端到端训练获得，损失函数专注于空间信息的恢复，网络结构采用了改进的视觉编码器，以适应高分辨率的输入和输出。
🖼️ 关键图片
📊 实验亮点
AtVLA在多个基准测试中表现出色，LIBERO的成功率从94.2%提升至98.4%，而在真实世界场景中的成功率从46.5%提升至69.0%。这种显著的提升表明了该框架在实际应用中的有效性和潜力。
🎯 应用场景
该研究的潜在应用领域包括机器人操作、自动驾驶、智能家居等场景，能够显著提升机器人在复杂环境中的空间感知和操作能力。未来，AtVLA框架可能推动更高效的多模态学习和人机交互技术的发展。
📄 摘要（原文）
Visual representations of VLA models remain unreliable for spatially precise robotic manipulation. We uncover that vision encoders in VLAs also exhibit attention artifacts previously documented in generic Vision Transformers, and further show that, in embodied policies, these artifacts are closely associated with spatial perception capabilities acquired during post-training. As the encoder learns task-relevant information such as object location, depth ordering, and local geometry, limited global-token capacity causes part of this information to spill into low-information patch tokens. We introduce AtVLA, a framework that inserts learnable register tokens into the visual encoder. Trained end-to-end using only embodied data and the original action objective, these registers emerge as dedicated carriers of embodied spatial information, while the remaining patch tokens recover clean and spatially faithful attention distributions crucial for precise target localization and fine-grained contact. Clean attention restores reliable localization, but cannot recover geometric details lost in low-resolution observations. AtVLA therefore couples attention rectification with uncertainty-gated local refinement. The action expert samples multiple action chunks and estimates uncertainty from their disagreement; only for uncertain predictions, action-conditioned attention rollout identifies the task-relevant region, which is cropped, re-encoded at high resolution, and appended to the cached prefix for refined action generation. Across LIBERO, SimplerEnv, and a challenging single-view real-world benchmark, AtVLA improves the average LIBERO success rate from 94.2% to 98.4% and real-world success from 46.5% to 69.0%. The cropping is triggered on approximately 30% of replanning steps, resulting in only 1.4-1.6x the total computation of the base model under the representative deployment setting.