Active Neural Topological Mapping for Multi-Agent Exploration

📄 arXiv: 2311.00252v1 📥 PDF

作者: Xinyi Yang, Yuxiang Yang, Chao Yu, Jiayu Chen, Jingchen Yu, Haibing Ren, Huazhong Yang, Yu Wang

分类: cs.RO, cs.LG, cs.MA

发布日期: 2023-11-01

备注: Accepted by Robotics and Automation Letters

💡 一句话要点

提出多智能体神经拓扑映射以解决合作探索问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 多智能体系统 拓扑映射 强化学习 图神经网络 环境探索 合作规划

📋 核心要点

  1. 现有的拓扑探索方法多采用经典规划方法，效率低且设计手工化，难以适应不同场景。
  2. 本文提出的MANTM结合了拓扑映射器和强化学习的分层规划器，旨在提高多智能体的探索效率和泛化能力。
  3. 在Habitat模拟器中的实验表明，MANTM在未见场景中相较于规划基线减少了至少26.40%的步骤，表现优越。

📝 摘要（中文）

本文研究了多智能体合作探索问题，要求多个智能体在有限时间内通过感知信号探索未知环境。现有的度量地图方法虽然能捕捉空间细节，但通信流量大且在不同场景中表现不佳。拓扑地图作为一种替代方案，由节点和边构成，信息抽象且不易受场景结构影响。本文提出的多智能体神经拓扑映射（MANTM）通过结合拓扑映射器和基于强化学习的分层拓扑规划器（HTP），显著提高了探索效率和泛化能力。实验结果表明，MANTM在未见场景中相较于基线方法减少了至少26.40%的步骤。

🔬 方法详解

问题定义：本文旨在解决多智能体在未知环境中的合作探索问题。现有方法在规划过程中效率低下，且由于手工设计，难以实现良好的泛化能力。

核心思路：提出多智能体神经拓扑映射（MANTM），通过结合拓扑映射器和基于强化学习的分层拓扑规划器（HTP），以实现更高效的探索和更好的泛化能力。

技术框架：MANTM主要由两个模块组成：拓扑映射器和分层拓扑规划器。拓扑映射器使用视觉编码器和基于距离的启发式方法构建图形，而HTP利用图神经网络捕捉智能体与图节点之间的关系，进行全局目标选择。

关键创新：MANTM的创新在于将图神经网络应用于多智能体的目标选择过程，能够有效捕捉智能体之间的协作关系，提升规划效率。

关键设计：拓扑映射器采用视觉编码器提取环境特征，使用距离启发式方法确定主要节点和对应的幽灵节点。HTP则通过图神经网络进行全局目标选择，确保智能体在探索过程中能够高效协作。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，MANTM在Habitat模拟器中相较于规划基线减少了至少26.40%的步骤，且在与强化学习竞争者的比较中也减少了至少7.63%的步骤，证明了其在未见场景中的优越性。

🎯 应用场景

该研究的潜在应用领域包括机器人探索、无人驾驶、环境监测等多智能体系统。通过提高探索效率和泛化能力，MANTM能够在复杂和动态的环境中实现更高效的任务执行，具有重要的实际价值和未来影响。

📄 摘要（原文）

This paper investigates the multi-agent cooperative exploration problem, which requires multiple agents to explore an unseen environment via sensory signals in a limited time. A popular approach to exploration tasks is to combine active mapping with planning. Metric maps capture the details of the spatial representation, but are with high communication traffic and may vary significantly between scenarios, resulting in inferior generalization. Topological maps are a promising alternative as they consist only of nodes and edges with abstract but essential information and are less influenced by the scene structures. However, most existing topology-based exploration tasks utilize classical methods for planning, which are time-consuming and sub-optimal due to their handcrafted design. Deep reinforcement learning (DRL) has shown great potential for learning (near) optimal policies through fast end-to-end inference. In this paper, we propose Multi-Agent Neural Topological Mapping (MANTM) to improve exploration efficiency and generalization for multi-agent exploration tasks. MANTM mainly comprises a Topological Mapper and a novel RL-based Hierarchical Topological Planner (HTP). The Topological Mapper employs a visual encoder and distance-based heuristics to construct a graph containing main nodes and their corresponding ghost nodes. The HTP leverages graph neural networks to capture correlations between agents and graph nodes in a coarse-to-fine manner for effective global goal selection. Extensive experiments conducted in a physically-realistic simulator, Habitat, demonstrate that MANTM reduces the steps by at least 26.40% over planning-based baselines and by at least 7.63% over RL-based competitors in unseen scenarios.