MMSR: Symbolic Regression is a Multi-Modal Information Fusion Task

📄 arXiv: 2402.18603v5 📥 PDF

作者: Yanjie Li, Jingyi Liu, Weijun Li, Lina Yu, Min Wu, Wenqiang Li, Meilan Hao, Su Wei, Yusong Deng

分类: cs.LG, cs.AI, cs.CL

发布日期: 2024-02-28 (更新: 2024-09-19)

备注: The Information Fusion has accepted this paper

DOI: 10.1016/j.inffus.2024.102681

🔗 代码/项目: GITHUB

💡 一句话要点

提出MMSR以解决符号回归中的多模态信息融合问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 符号回归 多模态融合 对比学习 大规模预训练 特征提取 数据建模 人工智能

📋 核心要点

  1. 现有符号回归方法如遗传编程在超参数敏感性和效率上存在不足，难以有效解决复杂的自然规律建模问题。
  2. 本文提出MMSR，将符号回归视为多模态信息融合任务，通过对比学习促进模态特征的对齐和融合，简化训练过程。
  3. 实验结果显示，MMSR在多个主流数据集上表现优异，相较于现有基线方法，性能显著提升，验证了其有效性。

📝 摘要（中文）

数学公式是人类探索自然法则的智慧结晶，符号回归(SR)旨在用简洁的数学公式描述复杂的自然规律。传统的符号回归方法如遗传编程和强化学习在效率和超参数敏感性上存在不足。为此，研究者将数据到表达式的映射视为翻译问题，引入大规模预训练模型。本文提出MMSR，将SR问题视为纯多模态问题，并在训练过程中引入对比学习以促进模态特征融合。实验结果表明，MMSR在多个主流数据集上超越了多种大规模预训练基线，取得了最先进的结果。

🔬 方法详解

问题定义：本文旨在解决符号回归(SR)问题，现有方法如遗传编程和强化学习在超参数设置和效率上存在明显不足，导致难以有效建模复杂的自然规律。

核心思路：论文提出将数据到表达式的映射视为多模态翻译问题，采用大规模预训练模型，并引入对比学习以促进模态特征的对齐和融合，从而提高符号回归的效率和准确性。

技术框架：整体架构包括数据输入模块、特征提取模块、对比学习模块和特征融合模块。通过对比学习模块，模型能够在训练过程中实现模态特征的有效对齐，进而提升最终的表达式生成效果。

关键创新：MMSR的核心创新在于将符号回归视为纯多模态问题，并通过对比学习实现模态特征的融合。这一方法与传统的单一模态处理方式有本质区别，能够更好地处理数据与表达式之间的复杂关系。

关键设计：在损失函数设计上，MMSR同时训练对比学习损失和其他损失，采用一步训练策略，避免了传统方法中先训练对比学习损失再训练其他损失的低效过程。这种设计使得特征提取模块和特征融合模块的协同效果显著提升。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，MMSR在多个主流数据集上表现优异，超越了多种大规模预训练基线，尤其在SRBench数据集上取得了最先进的结果，验证了其有效性和优越性。

🎯 应用场景

MMSR的研究成果在科学建模、工程设计和数据分析等领域具有广泛的应用潜力。通过高效的符号回归，研究者可以更准确地提取和描述自然规律，推动科学研究和技术创新。此外，该方法的多模态处理能力也为未来的智能系统提供了新的思路，可能在自动化建模和智能决策中发挥重要作用。

📄 摘要（原文）

Mathematical formulas are the crystallization of human wisdom in exploring the laws of nature for thousands of years. Describing the complex laws of nature with a concise mathematical formula is a constant pursuit of scientists and a great challenge for artificial intelligence. This field is called symbolic regression (SR). Symbolic regression was originally formulated as a combinatorial optimization problem, and Genetic Programming (GP) and Reinforcement Learning algorithms were used to solve it. However, GP is sensitive to hyperparameters, and these two types of algorithms are inefficient. To solve this problem, researchers treat the mapping from data to expressions as a translation problem. And the corresponding large-scale pre-trained model is introduced. However, the data and expression skeletons do not have very clear word correspondences as the two languages do. Instead, they are more like two modalities (e.g., image and text). Therefore, in this paper, we proposed MMSR. The SR problem is solved as a pure multi-modal problem, and contrastive learning is also introduced in the training process for modal alignment to facilitate later modal feature fusion. It is worth noting that to better promote the modal feature fusion, we adopt the strategy of training contrastive learning loss and other losses at the same time, which only needs one-step training, instead of training contrastive learning loss first and then training other losses. Because our experiments prove training together can make the feature extraction module and feature fusion module wearing-in better. Experimental results show that compared with multiple large-scale pre-training baselines, MMSR achieves the most advanced results on multiple mainstream datasets including SRBench. Our code is open source at https://github.com/1716757342/MMSR