Budget-Constrained Embodied Perception: Four Resource Walls and a Pre-Registered Evaluation of Access-Structured Perception on Open Models at less than 31B
作者: Defu Lin, Wenhui Chen, Ziyao Lin, Jianlin Chen, Peiji Long, Chi Man Vong
分类: cs.AI
发布日期: 2026-08-24
💡 一句话要点
提出ASP方法以解决预算约束下的多模态感知问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 多模态感知 预算约束 具身智能体 结构化状态 查询条件访问 长视频理解 检索准确率
📋 核心要点
- 现有方法在固定预算下难以有效处理多模态感知任务，导致性能低下。
- 论文提出ASP方法，通过结构化状态和查询条件预算分配，优化多模态模型的决策过程。
- 实验结果显示，ASP在检索准确率上显著优于传统方法，验证了其有效性和优势。
📝 摘要（中文）
本文探讨了在固定决策令牌预算下，具身多模态智能体如何处理不断增长的观察流。我们通过四个资源壁垒形式化这一约束，包括感知香农壁垒、地平线壁垒、回合壁垒和条件组合壁垒。提出的ASP方法是一个无训练的封装器，结合了结构化状态、逐步访问的预算分配和逐字的情节索引。通过在SEW-Bench上评估七个开放权重模型，ASP在4096令牌的决策预算下实现了75%至94%的情节检索准确率，显著优于传统的查询独立采样方法。
🔬 方法详解
问题定义：本文旨在解决在固定决策令牌预算下，具身多模态智能体如何有效处理不断增长的观察流的问题。现有方法在这一约束下表现不佳，导致检索准确率低。
核心思路：论文提出的ASP方法通过结合结构化状态、逐字情节索引和查询条件的预算分配，旨在优化多模态模型的决策过程，从而提高检索准确率。
技术框架：ASP方法的整体架构包括三个主要模块：结构化状态模块、逐字情节索引模块和查询条件预算分配模块。该框架支持在固定预算下的迭代访问。
关键创新：ASP的核心创新在于引入了查询条件访问机制，而不仅仅依赖于模型参数数量或上下文增长。这一设计使得在固定预算下的决策更加高效。
关键设计：在设计中，ASP采用了4096令牌的决策预算，并通过实验验证了不同模块的有效性。去除压缩状态后，模型性能显著提升，表明结构化状态在特定条件下的必要性。实验中还发现，预算重新分配优于简单的采样预算增加。
🖼️ 关键图片
📊 实验亮点
实验结果表明，在4096令牌的决策预算下，ASP方法的情节检索准确率达到了75%至94%，而传统的查询独立采样方法仅为3%至19%。此外，预算重新分配在每个基础模型上均优于简单的预算增加，显示出ASP方法的显著优势。
🎯 应用场景
该研究的潜在应用领域包括智能机器人、自动驾驶、虚拟助手等具身智能体的开发。通过优化多模态感知，ASP方法能够提升这些系统在复杂环境中的决策能力，具有重要的实际价值和未来影响。
📄 摘要（原文）
Embodied multimodal agents must answer from growing observation streams under a fixed per-decision token budget. We formalize this constraint through four resource walls: a perceptual Shannon wall for bounded state, a horizon wall for query-independent frame selection, a round wall for non-adaptive retrieval, and a conditional composition wall for fixed-depth inference. We introduce ASP, a training-free wrapper for frozen multimodal models that combines a capped structured state, a verbatim episodic index, and query-conditioned budget allocation with iterative access. Following a pre-registered protocol, we evaluate seven open-weight models from 3B to 31B on SEW-Bench, a license-free synthetic long-horizon walkthrough benchmark constructed to instantiate these walls. The registered natural-video benchmarks were not run because their frames require dataset agreements; our evidence therefore concerns access mechanisms, not natural-scene perception. Under a 4,096-token decision budget, ASP reaches 75 to 94% episodic retrieval accuracy, compared with 3 to 19% for equal-budget query-independent sampling, and budget reallocation outperforms quadrupling the sampling budget on every backbone. However, the full three-component architecture does not validate channel duality: removing the compressive state raises the flagship mean from 35.4 to 58.0, ASP does not outperform the verbatim-only baseline on any backbone, and two of four pre-registered falsification criteria fire. These results show that query-conditioned access, rather than parameter count or context growth alone, is decisive under a fixed budget, while prompted online compression does not earn its cost in this setting.