GEMSS: A Variational Bayesian Method for Discovering Multiple Sparse Solutions in Classification and Regression Problems

📄 arXiv: 2602.08913 📥 PDF

作者: Kateřina Henclová, Václav Šmídl

分类: cs.LG, stat.ML

发布日期: 2026-06-12

💡 一句话要点

提出GEMSS方法以发现多重稀疏解的分类与回归问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 特征选择 稀疏解 变分推断 高维数据 机器学习 数据科学 高斯混合模型

📋 核心要点

  1. 现有特征选择方法通常只能找到单一解，无法揭示多个可能的稀疏特征组合，限制了对数据的全面理解。
  2. GEMSS方法通过变分推断同时发现多个稀疏特征组合，采用尖峰-板条先验和高斯混合模型来处理多模态后验。
  3. 在128个实验中，GEMSS在特征选择上显著优于五种主流方法，能够有效识别多个高质量的稀疏解。

📝 摘要（中文）

高维、欠定且高度相关的系统在数据科学实践中十分常见，尤其是在物理测量分析中。特征选择是一个基本挑战，因为多个不同的稀疏子集可能同样有效地解释响应。传统方法通常只孤立出单一解，掩盖了可能的解释全景。本文提出了GEMSS（高斯集成多重稀疏解），一种变分算法，旨在同时发现多种多样的稀疏特征组合。该方法采用结构化的尖峰-板条先验来实现稀疏性，利用高斯混合模型来近似难以处理的多模态后验，并通过基于Jaccard的惩罚进一步控制解的多样性。通过128个综合实验进行测试，结果表明GEMSS在特征选择方面表现优于五种主流方法。

🔬 方法详解

问题定义：本文旨在解决高维、欠定系统中多重稀疏解的发现问题。现有方法往往只提供单一解，无法全面捕捉数据的多样性和复杂性。

核心思路：GEMSS通过变分推断技术，结合尖峰-板条先验和高斯混合模型，能够同时识别多个稀疏特征组合，从而提供更全面的解释。

技术框架：GEMSS的整体架构包括特征选择模块、变分推断模块和优化模块。特征选择模块负责构建稀疏特征组合，变分推断模块用于处理后验分布，优化模块则通过随机梯度下降优化目标函数。

关键创新：GEMSS的主要创新在于同时发现多个稀疏解的能力，利用结构化的先验和多模态后验近似，克服了传统方法的局限性。

关键设计：该方法采用了基于Jaccard的惩罚项以控制解的多样性，优化过程中使用了随机梯度下降，确保了高效性和准确性。

📊 实验亮点

在128个实验中，GEMSS方法在特征选择任务中表现优异，显著优于五种主流特征选择方法，能够有效识别多个高质量的稀疏解，展示了其在实际应用中的强大能力。

🎯 应用场景

GEMSS方法在多个领域具有广泛的应用潜力，尤其是在生物信息学、化学计量学和其他需要高维数据分析的领域。通过识别多个稀疏特征组合，研究人员能够获得更深入的领域特定见解，推动科学研究和实际应用的发展。

📄 摘要（原文）

High-dimensional, underdetermined and highly correlated systems are common in data science practice, especially when analyzing physical measurements. In such settings, feature selection poses a fundamental challenge because multiple distinct sparse subsets may explain the response equally well. Their identification is crucial not only for predictive modeling but also for generating domain-specific insights into the underlying mechanisms. Yet, conventional methods typically isolate a single solution, obscuring the full spectrum of plausible explanations. This work introduces GEMSS (Gaussian Ensemble for Multiple Sparse Solutions), a variational algorithm designed to simultaneously discover multiple, diverse sparse feature combinations. The method employs a structured spike-and-slab prior for sparsity, a mixture of Gaussians to approximate the intractable multimodal posterior, and a Jaccard-based penalty to further control solution diversity. A single objective function is optimized via stochastic gradient descent. The method is tested on 128 comprehensive experiments by a novel benchmarking framework designed to generate artificial problems with multiple sparse solutions of equal predictive properties. This allows us to measure the retrieval of ground truth features rather than only evaluating predictive performance -- characteristics more fitting to our practical needs. A comparative analysis shows that GEMSS consistently outperforms five prominent feature selection methods adapted through the ALFESE framework. Finally, we demonstrate practical usability through 3 challenging real-world datasets from metabolomics and physical chemistry: GEMSS successfully isolates multiple distinct yet quality solutions. GEMSS is available as a PyPI package 'gemss'. The corresponding repositorythis http URLincludes the full codebase and a free, no-code application GEMSS Explorer.