Multimodal Graph Negative Learning

📄 arXiv: 2606.12863v1 📥 PDF

作者: Zhengyu Wu, Xu Wang, Hongchao Qin, Xunkai Li, Guang Zeng, Rong-Hua Li, Guoren Wang

分类: cs.LG

发布日期: 2026-06-11

💡 一句话要点

提出GraphMNL以解决多模态图节点表示不平衡问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多模态图 负学习 节点表示 语义不平衡 图感知

📋 核心要点

  1. 现有方法在多模态图学习中存在节点表示不平衡的问题，导致语义信息的可靠性差异。
  2. 本文提出GraphMNL框架，通过负学习指导劣质分支，避免强制模仿主导分支的偏差。
  3. GraphMNL在Grocery数据集上取得了72.47%的准确率和76.60的F1分数，显著提升了分类性能。

📝 摘要（中文）

多模态属性图（MAGs）结合了图的拓扑结构与异构模态属性，如文本和图像，从而实现对复杂关系系统的更丰富建模。然而，节点级别的语义不平衡问题使得不同节点的模态分支在语义信息和可靠性上存在差异。现有方法通常通过跨分支一致性来缓解这种异质性，但当主导分支存在偏差时，强制模仿可能会传播其偏差。为此，本文提出了GraphMNL，一个基于图的多模态负学习框架，通过负学习作为跨分支指导，帮助劣质分支识别不太可能属于的类别。实验结果表明，GraphMNL在Grocery数据集上取得了72.47%的准确率和76.60的F1分数，表现优于现有方法。

🔬 方法详解

问题定义：本文旨在解决多模态属性图中节点表示的不平衡问题，现有方法通过强制模仿主导分支的预测，可能导致劣质分支的偏差传播，从而影响分类效果。

核心思路：GraphMNL框架的核心思想是利用负学习指导劣质分支，帮助它们识别不太可能属于的类别，而不是强制模仿主导分支的预测。这样的设计可以避免偏差传播，保留有用的语义信息。

技术框架：GraphMNL的整体架构包括分支库构建、主导与劣质分支的识别、图感知的可靠性仲裁、以及对非目标类别的目标保留负学习。各模块协同工作，以实现更有效的节点表示学习。

关键创新：最重要的技术创新在于引入了负学习作为跨分支指导，这与现有方法的强制模仿策略本质上不同，避免了偏差的传播。

关键设计：在参数设置上，GraphMNL通过图感知的可靠性仲裁来识别分支的优劣，并在损失函数中引入目标保留负学习，以确保学习过程中的语义信息不被压制。整体网络结构设计上，确保了各分支之间的有效协作与信息共享。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在综合实验评估中，GraphMNL在Grocery数据集上取得了72.47%的准确率和76.60的F1分数，显著优于现有基线方法，展示了其在多模态图学习中的有效性与优势。

🎯 应用场景

该研究的潜在应用领域包括社交网络分析、推荐系统和生物信息学等。通过更准确的节点表示，GraphMNL能够提升多模态数据的处理能力，进而推动相关领域的研究与应用发展，具有重要的实际价值和未来影响。

📄 摘要（原文）

Multimodal attributed graphs (MAGs) integrate graph topology with heterogeneous modality attributes, such as text and images, thereby enabling richer modeling of complex relational systems. However, such expressiveness also makes learning on MAGs depend on multiple semantic sources, including structural topology, textual and visual attributes, each of which can be regarded as a branch for node representation. Node-level branch semantic imbalance arises when these branches differ across nodes in semantic informativeness and reliability: a branch that provides discriminative semantics for one node may mislead another due to bias in modality quality or structural context. Existing methods often mitigate such heterogeneity through cross-branch agreement or alignment, implicitly treating the dominant prediction as reliable supervision. When the dominant branch is biased, forced imitation may propagate its bias to other branches and suppress original semantics that are useful for classification. We propose GraphMNL, a graph-aware multimodal negative learning framework that addresses this issue by using Negative Learning as cross-branch guidance. Instead of forcing inferior branches to imitate a teacher prediction, the model teaches them which classes a node is unlikely to belong to. GraphMNL builds a branch library, identifies dominant and inferior branches via graph-aware reliability arbitration, gates unstable transfer, and applies target-preserving negative learning over non-target classes. This design decouples target supervision from branch guidance so that supervised losses learn the correct class, while Negative Learning suppresses unlikely alternatives when branch agreement is unreliable. Through the comprehensive experimental evaluation, GraphMNL achieves the best performance on Grocery datasets with 72.47% accuracy and 76.60 F1 score on Reddit M datasets.