AUG: A New Dataset and An Efficient Model for Aerial Image Urban Scene Graph Generation

📄 arXiv: 2404.07788v1 📥 PDF

作者: Yansheng Li, Kun Li, Yongjun Zhang, Linlin Wang, Dingwen Zhang

分类: cs.CV, cs.AI

发布日期: 2024-04-11

💡 一句话要点

提出AUG数据集与LPG模型以解决城市航拍图像场景图生成问题

🎯 匹配领域: 支柱七：动作重定向 (Motion Retargeting)

关键词: 场景图生成 航拍图像 图卷积网络 局部保持 城市分析 智能交通 数据集构建

📋 核心要点

  1. 现有的场景图生成方法主要集中在平视视角，缺乏航拍视角的数据集，导致对城市场景的理解受限。
  2. 本文提出了航拍城市场景图生成数据集（AUG）及局部保持图卷积网络（LPG），以更好地捕捉城市场景中的空间关系。
  3. 实验结果显示，LPG在AUG数据集上的表现显著优于现有方法，验证了局部保持策略的有效性。

📝 摘要（中文）

场景图生成（SGG）旨在从给定图像中理解视觉对象及其语义关系。现有的SGG数据集多为平视视角，缺乏航拍视角的数据集。本文构建并发布了航拍城市场景图生成（AUG）数据集，包含25,594个对象、16,970个关系和27,175个属性的手动标注。为避免复杂城市场景中局部上下文的干扰，提出了一种新的局部保持图卷积网络（LPG），该网络在挖掘全局上下文的同时，保留对象的初始特征和动态更新的邻域信息。此外，针对潜在关系对数量庞大但有效关系对稀少的问题，提出了自适应边界框缩放因子（ABS-PRD）来智能修剪无意义的关系对。实验结果表明，LPG显著优于现有最先进的方法，验证了局部保持策略的有效性。

🔬 方法详解

问题定义：本文旨在解决航拍视角下城市场景图生成的问题。现有方法在平视视角下存在对象遮挡等问题，影响了场景图生成的准确性。

核心思路：提出局部保持图卷积网络（LPG），通过整合对象的初始特征与动态更新的邻域信息，既保留局部上下文，又挖掘全局上下文。

技术框架：LPG的整体架构包括输入层、图卷积层和输出层。输入层接收航拍图像及其标注信息，图卷积层进行特征提取，输出层生成场景图。

关键创新：LPG的主要创新在于其局部保持机制，与传统图卷积网络不同，能够有效处理复杂城市场景中的局部信息。

关键设计：在网络设计中，采用了自适应边界框缩放因子（ABS-PRD）来修剪无意义的关系对，确保模型聚焦于有效的对象关系。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在AUG数据集上的实验结果显示，LPG模型的性能显著优于现有最先进的方法，具体提升幅度达到XX%（具体数据需根据实验结果补充），验证了局部保持策略的有效性。

🎯 应用场景

该研究的潜在应用领域包括城市规划、智能交通系统和无人机监控等。通过更准确的场景图生成，能够提升城市环境的理解与分析能力，促进智能城市的发展。

📄 摘要（原文）

Scene graph generation (SGG) aims to understand the visual objects and their semantic relationships from one given image. Until now, lots of SGG datasets with the eyelevel view are released but the SGG dataset with the overhead view is scarcely studied. By contrast to the object occlusion problem in the eyelevel view, which impedes the SGG, the overhead view provides a new perspective that helps to promote the SGG by providing a clear perception of the spatial relationships of objects in the ground scene. To fill in the gap of the overhead view dataset, this paper constructs and releases an aerial image urban scene graph generation (AUG) dataset. Images from the AUG dataset are captured with the low-attitude overhead view. In the AUG dataset, 25,594 objects, 16,970 relationships, and 27,175 attributes are manually annotated. To avoid the local context being overwhelmed in the complex aerial urban scene, this paper proposes one new locality-preserving graph convolutional network (LPG). Different from the traditional graph convolutional network, which has the natural advantage of capturing the global context for SGG, the convolutional layer in the LPG integrates the non-destructive initial features of the objects with dynamically updated neighborhood information to preserve the local context under the premise of mining the global context. To address the problem that there exists an extra-large number of potential object relationship pairs but only a small part of them is meaningful in AUG, we propose the adaptive bounding box scaling factor for potential relationship detection (ABS-PRD) to intelligently prune the meaningless relationship pairs. Extensive experiments on the AUG dataset show that our LPG can significantly outperform the state-of-the-art methods and the effectiveness of the proposed locality-preserving strategy.