AirGroundBench: Probing Spatial Intelligence in Multimodal Large Models under Heterogeneous Multi-View Embodied Collaboration
作者: Haotian Li, Yida Wang, Leyuan Wang, Jinshan Lai, Keyang Wang, Zonghao Guo, Qiang Ma, Liuyu Xiang, Jianwei Hu, Zhaofeng He
分类: cs.CV
发布日期: 2026-06-26
💡 一句话要点
提出AirGroundBench以评估多模态大模型在异构多视角协作中的空间智能
🎯 匹配领域: 支柱七：动作重定向 (Motion Retargeting) 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 多模态大模型 空间智能 异构视角 无人机-地面车辆协作 视觉问答 视觉语言导航 几何一致性
📋 核心要点
- 现有方法主要关注单一视角的感知，缺乏对多视角协作的系统评估，导致空间理解能力不足。
- 本文提出AirGroundBench基准，专注于异构无人机与地面车辆的协作，评估其在多视角下的空间智能。
- 评估结果表明，模型在空间感知上表现良好，但在跨视角对齐和推理方面存在瓶颈，影响决策能力。
📝 摘要（中文）
近年来，多模态大语言模型（MLLMs）在具身智能方面展现出强大潜力，但其在异构视角下保持几何一致的空间理解能力仍未得到充分评估。现有基准主要集中于单一代理、单一视角的感知，缺乏对协作空地设置的系统性评估。本文提出AirGroundBench，一个用于评估异构无人机-地面车辆协作中多视角空间智能的诊断基准。该基准由11个高保真模拟环境构建，包含1,021对同步的空地观测，提供约62,000个双视角的视觉问答实例和115个闭环视觉语言导航任务。评估结果显示，尽管双视角输入在一定程度上优于单视角，但与人类表现之间仍存在显著差距，凸显了几何一致性作为当前具身MLLMs的关键限制。
🔬 方法详解
问题定义：本文旨在解决多模态大语言模型在异构视角下的空间理解能力不足的问题。现有方法主要集中于单一视角的感知，未能有效评估多视角协作中的几何一致性。
核心思路：论文提出AirGroundBench基准，通过构建高保真模拟环境和同步观测对，系统评估多视角下的空间智能，特别是在无人机与地面车辆的协作场景中。
技术框架：AirGroundBench基准包含11个模拟环境，提供1,021对空地观测，涵盖62,000个视觉问答实例和115个导航任务。任务类型分为四个能力维度：空间感知、跨视角对齐、空间变换与推理、具身决策。
关键创新：最重要的创新在于提供结构化的空间注释，包括跨视角物体身份和度量的2D、3D边界框，从而支持几何基础的评估与分析。
关键设计：在实验中，采用了多种输入设置（无人机、地面车辆和双视角），并通过对比评估13个代表性MLLMs，分析其在不同任务中的表现与瓶颈。
🖼️ 关键图片
📊 实验亮点
实验结果显示，模型在空间感知任务中表现良好，但在跨视角对齐和推理方面存在显著瓶颈，影响了视觉语言导航中的决策能力。尽管双视角输入相较于单视角有一定提升，但与人类表现之间仍有明显差距，强调了几何一致性的重要性。
🎯 应用场景
该研究的潜在应用领域包括无人机监控、自动驾驶、智能城市等，能够为多模态大模型在复杂环境中的应用提供有效的评估标准和改进方向，推动具身智能的发展。
📄 摘要（原文）
In recent years, multimodal large language models (MLLMs) have shown strong potential for embodied intelligence, yet their ability to maintain geometrically consistent spatial understanding across heterogeneous views remains under-evaluated. Existing benchmarks largely focus on single-agent, single-view perception, leaving a gap in the systematic assessment of collaborative air-ground settings, where multi-scale observations are complementary but introduce scale mismatch, asymmetric occlusion, and reference-frame inconsistencies. We present AirGroundBench, a diagnostic benchmark for evaluating multi-view spatial intelligence in heterogeneous UAV-UGV collaboration. AirGroundBench is built from 11 high-fidelity simulated environments with 1,021 synchronized air-ground observation pairs, yielding approximately 62,000 dual-view, four-option single-choice visual question answering instances and 115 closed-loop vision-language navigation episodes. It covers 10 task types organized into four progressively demanding capability dimensions: spatial perception, cross-view alignment, spatial transformation and reasoning, and embodied decision-making. To support geometry-grounded evaluation and analysis, we provide structured spatial annotations, including cross-view object identities and metric 2D and 3D bounding boxes. Evaluations of 13 representative MLLMs under UAV-only, UGV-only, and dual-view input settings reveal consistent bottlenecks: models perform relatively well on spatial perception but struggle with cross-view alignment and transformation-intensive reasoning, and these deficits propagate to sequential decision-making in vision-language navigation. Although dual-view inputs provide measurable gains over single-view variants, a persistent gap from human performance remains, highlighting geometric consistency as a key limitation of current embodied MLLMs.