Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning
作者: Matthew Foutter, Matteo Cercola, Lena Wild, Yunshan Wang, Michelle Li, Daniele Gammelli, Marco Pavone
分类: cs.RO, cs.AI
发布日期: 2026-07-06
🔗 代码/项目: PROJECT_PAGE
💡 一句话要点
提出基于Pinocchio的信实推理方法以提升机器人决策能力
🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 信实推理 机器人决策 视觉-语言-行动 强化学习 自主驾驶 可解释性 因果推理
📋 核心要点
- 现有的视觉-语言-行动模型在推理过程中可能存在不一致性，导致决策过程的透明性不足。
- 本文提出了一种基于学习的评估器Pinocchio，旨在通过评分推理的信实性来提升机器人决策的可靠性。
- 实验结果表明，后训练的规划器在信实性上提升了4%和18%，同时保持了竞争性的任务性能。
📝 摘要（中文）
具身链式思维作为一种增强机器人决策和可解释性的机制，然而其是否真实反映政策的决策过程尚不清楚。本文区分了功能性推理与信实推理，指出现有的对齐策略在信实性方面的不足。通过对自主驾驶的SoTA推理模型进行人类评估，发现推理质量与轨迹改进之间存在不一致性。提出了一种基于学习的评估器Pinocchio，用于评分观察的基础和逐步一致性，并在后训练中作为密集奖励信号，显著提升了信实性和任务性能。
🔬 方法详解
问题定义：本文旨在解决现有视觉-语言-行动模型在推理过程中信实性不足的问题，现有方法的推理步骤可能掩盖了因果关系，限制了政策的泛化能力。
核心思路：通过引入一个学习的评估器Pinocchio，量化推理的信实性，确保推理过程不仅提高任务性能，还真实反映政策的内部决策过程。
技术框架：整体架构包括一个基于Pinocchio的评估模块，该模块评分观察的基础和逐步一致性，并在后训练阶段作为奖励信号，结合强化学习优化决策策略。
关键创新：最重要的创新在于引入了信实性评估机制，利用学习的评估器来量化推理的质量，与现有方法相比，强调了推理过程的透明性和一致性。
关键设计：在设计中，Pinocchio评估器的损失函数专注于观察的基础和推理步骤的一致性，确保模型在后训练阶段能够有效利用信实性评分进行优化。
🖼️ 关键图片
📊 实验亮点
实验结果显示，后训练的规划器在信实性上提升了4%和18%，相较于SoTA对齐和轨迹误差的基线，同时在合成的离散测试集上，政策对罕见反事实场景的响应能力提高了1.6倍，表明信实推理对增强模型的鲁棒性和可解释性至关重要。
🎯 应用场景
该研究的潜在应用领域包括自主驾驶、机器人决策支持系统等，能够提升机器人在复杂环境中的决策能力和可解释性，未来可能对智能交通和人机协作等领域产生深远影响。
📄 摘要（原文）
Embodied Chain-of-Thought has emerged as a promising mechanism to enhance robot decision-making and interpretability in black-box Vision-Language Action (VLA) models. However, whether this verbalized Chain-of-Thought truthfully reflects the policy's underlying decision process remains poorly understood. We distinguish between functional reasoning, in which reasoning improves task performance, and faithful reasoning, in which reasoning truly reflects the policy's internal decision process. We argue that SoTA alignment strategies offer a necessary but insufficient notion of faithfulness, admitting reasoning whose intermediate steps can mask the causal links in action prediction through confounding factors (e.g., reasoning that is ungrounded in the environment and internally disconnected or inconsistent), restricting policy generalization. We study this gap through a human evaluation of a SoTA reasoning model for autonomous driving, revealing an inconsistent coupling between reasoning quality and downstream trajectory improvement. We then operationalize a behavioral surrogate for embodied faithfulness through a learned critic, Pinocchio, scoring observation grounding and stepwise coherence, and use this critic as a dense reward signal in post-training an embodied policy with reinforcement learning. Across withheld driving benchmarks, our post-trained planner improves faithfulness by 4% and 18% over SoTA alignment and trajectory error post-training baselines, respectively, while maintaining competitive downstream task performance. Finally, on a synthetic out-of-distribution test set, post-training for faithfulness improves policy responsiveness to rare counterfactual scenarios by 1.6x that of a SoTA policy, suggesting that faithful reasoning traces contribute to more robust, generalizable, and interpretable embodied intelligence. Project page: https://mjf-su.github.io/pinocchio/