Multimodal Graph Negative Learning

📄 arXiv: 2606.12863 📥 PDF

作者: Zhengyu Wu, Xu Wang, Hongchao Qin, Xunkai Li, Guang Zeng, Rong-Hua Li, Guoren Wang

分类: cs.LG

发布日期: 2026-06-12

💡 一句话要点

提出GraphMNL以解决多模态图节点表示不平衡问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多模态图 负学习 节点表示 语义不平衡 图神经网络 分类性能 跨分支指导

📋 核心要点

  1. 现有方法在多模态图中存在节点级分支语义不平衡的问题，导致分类性能下降。
  2. 论文提出GraphMNL框架，通过负学习指导 inferior 分支，避免强制模仿主导分支的偏见。
  3. 在Grocery数据集上，GraphMNL取得了72.47%的准确率和76.60的F1分数，显著提升了分类效果。

📝 摘要（中文）

多模态属性图（MAGs）结合了图的拓扑结构与异构模态属性，如文本和图像，从而能够更丰富地建模复杂的关系系统。然而，这种表达能力使得MAGs的学习依赖于多个语义源，包括结构拓扑、文本和视觉属性。现有方法通常通过跨分支一致性或对齐来缓解这种异质性，但在主导分支偏见的情况下，强制模仿可能会传播偏见并抑制有用的原始语义。为此，本文提出了GraphMNL，一个图感知的多模态负学习框架，通过负学习作为跨分支指导，解决了这一问题。GraphMNL在Grocery数据集上实现了72.47%的准确率和76.60的F1分数，表现优于现有方法。

🔬 方法详解

问题定义：本文旨在解决多模态属性图中节点表示的语义不平衡问题。现有方法通过强制模仿主导分支的预测，可能导致偏见传播，抑制有用的原始语义。

核心思路：GraphMNL框架采用负学习作为跨分支指导，教导inferior分支哪些类别不太可能属于某节点，而不是强制其模仿主导分支的预测。

技术框架：GraphMNL的整体架构包括分支库的构建、主导与inferior分支的识别、图感知的可靠性仲裁、以及对非目标类别的目标保留负学习。

关键创新：最重要的创新在于将目标监督与分支指导解耦，使得监督损失能够学习正确的类别，而负学习则在分支一致性不可靠时抑制不太可能的替代类别。

关键设计：关键设计包括分支库的构建、可靠性仲裁机制的实现，以及负学习的损失函数设计，确保模型能够有效识别和利用各分支的语义信息。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

GraphMNL在Grocery数据集上取得了72.47%的准确率和76.60的F1分数，显著优于现有基线方法，展示了其在处理多模态图节点表示不平衡问题上的有效性和优势。

🎯 应用场景

该研究的潜在应用领域包括社交网络分析、推荐系统和生物信息学等复杂关系系统的建模。通过提高多模态图的学习能力，GraphMNL能够在实际应用中提供更准确的分类和预测，推动相关领域的发展。

📄 摘要（原文）

Multimodal attributed graphs (MAGs) integrate graph topology with heterogeneous modality attributes, such as text and images, thereby enabling richer modeling of complex relational systems. However, such expressiveness also makes learning on MAGs depend on multiple semantic sources, including structural topology, textual and visual attributes, each of which can be regarded as a branch for node representation. Node-level branch semantic imbalance arises when these branches differ across nodes in semantic informativeness and reliability: a branch that provides discriminative semantics for one node may mislead another due to bias in modality quality or structural context. Existing methods often mitigate such heterogeneity through cross-branch agreement or alignment, implicitly treating the dominant prediction as reliable supervision. When the dominant branch is biased, forced imitation may propagate its bias to other branches and suppress original semantics that are useful for classification. We propose GraphMNL, a graph-aware multimodal negative learning framework that addresses this issue by using Negative Learning as cross-branch guidance. Instead of forcing inferior branches to imitate a teacher prediction, the model teaches them which classes a node is unlikely to belong to. GraphMNL builds a branch library, identifies dominant and inferior branches via graph-aware reliability arbitration, gates unstable transfer, and applies target-preserving negative learning over non-target classes. This design decouples target supervision from branch guidance so that supervised losses learn the correct class, while Negative Learning suppresses unlikely alternatives when branch agreement is unreliable. Through the comprehensive experimental evaluation, GraphMNL achieves the best performance on Grocery datasets with 72.47% accuracy and 76.60 F1 score on Reddit M datasets.