ReGround: Restoring Visual Grounding in Multi-Step Reasoning through Self-Diagnosis and Visual Re-Examination

📄 arXiv: 2608.04385v1 📥 PDF

作者: Lei Peng, Shuai Lv, Wei Hu

分类: cs.CV

发布日期: 2026-08-05

备注: Accepted to ACM Multimedia 2026 (MM '26). 8 pages main text, 4 figures, plus appendix

🔗 代码/项目: GITHUB | PROJECT_PAGE

💡 一句话要点

提出ReGround以解决多步推理中的视觉定位问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 视觉语言模型 多步推理 自我诊断 视觉重新审查 模型增强

📋 核心要点

  1. 现有的视觉语言模型在多步推理中容易失去视觉定位，导致推理依赖语言而非图像证据。
  2. 论文提出ReGround框架，通过自我诊断和视觉重新审查来恢复视觉定位，避免了架构修改和外部工具的需求。
  3. 在八个基准测试中，ReGround在视觉密集的多步推理任务上表现出显著提升，且推理开销仅为适度。

📝 摘要（中文）

视觉语言模型（VLMs）在多步推理过程中常常失去视觉定位，随着推理链的延长，后续推理步骤越来越依赖语言先验而非图像证据。本文通过对2510个样本的重新审查，发现注意力熵在第一轮中通常下降，图像重新注入后又上升。有效的视觉重新审查需要图像重新注入和有针对性的自我诊断。缺乏针对性诊断时，重新审查可能会降低性能，而准确的自我诊断则能显著提高性能。我们提出了ReGround，一个两阶段框架，教会VLMs自我诊断定位失败并选择性地重新审查视觉证据。实验表明，该方法在八个基准测试上表现出一致的提升，尤其是在视觉密集的多步推理任务中。

🔬 方法详解

问题定义：本文旨在解决视觉语言模型在多步推理过程中失去视觉定位的问题。现有方法在推理链延长时，后续推理步骤越来越依赖语言先验，导致性能下降。

核心思路：ReGround框架的核心思想是通过自我诊断定位失败并选择性地重新审查视觉证据，从而恢复视觉定位。该设计旨在提高模型在推理过程中的视觉依赖性。

技术框架：ReGround采用两阶段框架，第一阶段进行自我诊断，识别视觉定位失败；第二阶段进行视觉证据的重新审查。该框架无需对模型架构进行修改或使用外部工具。

关键创新：ReGround的主要创新在于引入了有针对性的自我诊断机制，显著提高了视觉重新审查的有效性。与现有方法相比，该框架能够在不增加复杂性的情况下，提升模型的推理能力。

关键设计：在模型设计中，采用了能力引导的方式，通过同一模型家族中的更强变体提供诊断支撑，确保在数据构建阶段的准确性。推理时，策略模型能够自主进行诊断，保持大部分辅助增益。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在八个基准测试中，ReGround在视觉密集的多步推理任务上表现出一致的性能提升，尤其在关键基准上提升幅度达到数个点，相较于工具增强的基线，推理开销仅为适度。

🎯 应用场景

ReGround的研究成果在多模态学习、智能问答、图像描述生成等领域具有广泛的应用潜力。通过提高视觉语言模型的推理能力，该方法能够增强人机交互的智能化水平，推动相关技术的实际应用与发展。

📄 摘要（原文）

Vision-Language Models (VLMs) often lose visual grounding during multi-step reasoning: as reasoning chains grow longer, later inference steps rely increasingly on language priors rather than image evidence. We identify a consistent benchmark-level signature associated with this degradation: across 2,510 re-examined samples from four benchmarks, attention entropy over image tokens typically decreases during Round 1 and rises again after image re-injection. However, we find that effective visual re-examination requires two complementary ingredients: image re-injection and targeted self-diagnosis. Without targeted diagnosis, re-examination can even hurt performance, whereas accurate self-diagnosis yields substantial gains -- a swing of several points on key benchmarks, indicating that diagnostic quality is a key factor in whether re-examination helps or hurts in our setting. We present ReGround, a two-stage framework that teaches VLMs to self-diagnose grounding failures and selectively re-examine visual evidence, without architectural modifications or external tools. Through capability bootstrapping, a stronger variant from the same model family provides diagnostic scaffolding only during data construction, while the policy model learns to diagnose autonomously at inference time and retains most of the assisted gains. Experiments on eight benchmarks across two VLM backbones demonstrate consistent gains, especially on visually intensive multi-step reasoning tasks, while incurring only modest inference overhead relative to tool-augmented baselines. Project page: https://sespoir.github.io/reground-page/ . Code: https://github.com/sespoir/ReGround .