ReScene: Structured Indoor Scene Reconstruction from Multi-View Captures

📄 arXiv: 2606.28060v1 📥 PDF

作者: Haoran Xu, Lechao Zhang, Daoguo Dong, Yan Gao, Xin Tan

分类: cs.CV

发布日期: 2026-06-26

💡 一句话要点

提出ReScene框架以解决多视角室内场景重建问题

🎯 匹配领域: 支柱三：空间感知与语义 (Perception & Semantics) 支柱四：生成式动作 (Generative Motion)

关键词: 多视角重建 物理一致性 场景图 语义一致性 具身人工智能

📋 核心要点

  1. 现有方法在多视角场景重建中存在依赖专用硬件、单视角偏差和物理不一致等挑战。
  2. ReScene框架通过多视角几何信息贯穿整个流程，重点在于语义一致性和3D覆盖完整性。
  3. 实验结果显示，ReScene在Chamfer距离和LPIPS上分别减少17%和26%，且速度提升达10倍。

📝 摘要（中文）

从多视角捕获中构建适用于仿真场景的3D模型是具身人工智能的关键瓶颈。现有方法依赖于专用硬件，或在物体重建中存在单视角偏差，导致布局几何合理但物理不一致。本文提出ReScene框架，通过多视角几何信息贯穿整个流程，解决跨视角关系融合和物理一致的场景组装问题。ReScene在ScanNet场景上设定了几何、渲染和感知质量的新状态，Chamfer距离减少17%，LPIPS减少26%，且运行速度比以往方法快10倍。基于重建场景，生成了一个具身视觉问答数据集，表现接近强闭源模型。

🔬 方法详解

问题定义：本文旨在解决从多视角捕获中重建物理一致的室内场景的问题。现有方法往往依赖于专用硬件，或在物体重建中存在单视角偏差，导致生成的场景几何合理但物理上不一致。

核心思路：ReScene框架的核心思路是将多视角几何信息作为统一先验贯穿整个重建流程，重点关注跨视角关系的融合和物理一致的场景组装。

技术框架：ReScene主要由两个模块组成：HierView和Relation-Aware Assembly。HierView根据语义一致性和3D覆盖完整性优先选择重建视角，而Relation-Aware Assembly则将来自视觉-语言模型的多帧关系预测与几何和房间外壳先验融合，形成一个加权场景图。

关键创新：ReScene的关键创新在于引入了基于语义一致性和3D覆盖的视角选择机制，以及通过加权场景图实现的物理一致的场景组装。这与现有方法的最大掩码启发式方法形成了本质区别。

关键设计：在HierView模块中，采用了新的视角选择策略，避免了将图像占用与物体覆盖混淆的情况；在Relation-Aware Assembly模块中，设计了基于信心加权的场景图，确保了多帧关系的有效融合。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

ReScene在ScanNet场景上设定了新状态，Chamfer距离减少17%，LPIPS减少26%，且运行速度比以往多视角方法快10倍。这些实验结果表明，ReScene在几何、渲染和感知质量上均有显著提升，展示了其优越性。

🎯 应用场景

该研究的潜在应用领域包括虚拟现实、增强现实和机器人导航等。通过生成物理一致的3D场景，ReScene可以为具身人工智能提供更好的环境理解能力，从而提升其在复杂场景中的决策和交互能力。未来，该方法有望推动智能体在真实世界中的应用。

📄 摘要（原文）

Constructing simulation-ready 3D scenes from multi-view captures is a key bottleneck for Embodied Artificial Intelligence, as downstream tasks require object-level structure, explicit inter-object relations, and physical plausibility. Existing approaches either rely on specialized capture hardware, suffer from single-view bias in object reconstruction, or yield layouts that are geometrically reasonable but physically inconsistent. We identify that the problem is not single-object reconstruction but cross-view relation fusion and physically plausible scene assembly. To address this challenge, we present ReScene, a framework that threads multi-view geometry throughout the pipeline as a unifying prior. Our method consists of two main components: HierView prioritizes reconstruction views based on semantic consistency and 3D coverage completeness, replacing the largest-mask heuristic that conflates image occupancy with object coverage; and Relation-Aware Assembly fuses multi-frame relation predictions from a vision-language model with geometric and room-shell priors into a confidence-weighted scene graph, enabling physically consistent scene assembly. ReScene sets a new state of the art across geometry, rendering, and perceptual quality on a set of ScanNet scenes, achieving a 17% reduction in Chamfer Distance and 26% in LPIPS over the strongest prior baseline, while running up to 10x faster than prior multi-view methods. Based on the reconstructed scenes, we also generate an embodied visual question answering dataset, on which fine-tuned Qwen-VL approaches the performance of strong closed-source models on several spatial reasoning tasks.