Integrated Feature Selection for classification and Representation Learning with Group-Lasso Siamese Networks

📄 arXiv: 2610.39499v1 📥 PDF

作者: Nisargo Ranjan Pal, Pradeep Singh

分类: cs.AI

发布日期: 2026-10-29

💡 一句话要点

提出基于Group-Lasso的Siamese网络进行特征选择与表示学习

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 特征选择 Siamese网络 Group-Lasso 表示学习 高维数据 机器学习 分类性能

📋 核心要点

  1. 现有方法主要关注分类准确率，未能有效识别紧凑的特征子集，导致特征冗余和计算效率低下。
  2. 本文提出将Group-Lasso原理与Siamese网络结合，旨在同时实现特征选择和数据表示学习，优化特征子集的紧凑性与分类性能。
  3. 在12个基准数据集上进行的实验表明，所提方法在特征选择和分类性能上均优于现有方法，展示了良好的效果。

📝 摘要（中文）

最小绝对收缩和选择算子（Lasso）在线性模型中广泛应用于特征选择，并已扩展至多层感知器（MLP）网络。本文首次将Group-Lasso原理引入Siamese网络架构，以实现特征选择。与现有方法主要优化分类准确率不同，所提方法旨在识别紧凑的特征子集，同时保留输入数据的内在表示，并期望实现高分类性能。通过在12个基准数据集上进行实验，结果表明所提Siamese Lasso框架能够有效识别紧凑且具有区分性的特征子集，保持竞争力或优越的分类性能。

🔬 方法详解

问题定义：本文旨在解决现有特征选择方法在分类准确率优化中忽视特征紧凑性的问题，导致特征冗余和计算效率低下。

核心思路：通过将Group-Lasso原理引入Siamese网络架构，提出一种新的特征选择方法，旨在同时优化特征的紧凑性和分类性能。

技术框架：整体架构包括两个主要模块：特征选择模块和分类模块。特征选择模块利用Group-Lasso进行特征筛选，分类模块则基于Siamese网络进行数据表示学习。

关键创新：将Group-Lasso与Siamese网络结合，是该研究的核心创新点，突破了传统方法仅关注分类准确率的局限，强调特征选择的紧凑性。

关键设计：采用欧几里得距离和改进的Jaccard相似度度量进行特征选择，特别设计的损失函数未显式考虑正则化，确保了特征选择的有效性。

📊 实验亮点

实验结果显示，所提Siamese Lasso框架在12个基准数据集上表现优异，能够识别出紧凑且具有区分性的特征子集，分类性能与现有方法相比保持竞争力或更优，特征维度范围从6到7070不等。

🎯 应用场景

该研究的潜在应用领域包括高维稀疏数据分析、图像分类、文本分类等。通过有效的特征选择与表示学习，能够提升模型的性能和计算效率，具有广泛的实际价值和未来影响。

📄 摘要（原文）

The Least Absolute Shrinkage and Selection Operator (Lasso) has been widely employed for feature selection in linear models and has also been extended to multi-layer perceptron (MLP) networks for both feature and group feature selection. Group-Lasso has also been used in feature selection using feed-forward neural networks. To the best of our knowledge, this is the first study to incorporate the Group-Lasso principle into a Siamese network architecture for feature selection. Unlike existing approaches that primarily optimize classification accuracy, the proposed method aims to identify a compact subset of features that simultaneously preserves the intrinsic representation of the input data (the goal of Siamese network) and is expected to achieve high classification performance. Two variants of the proposed framework are developed: one based on the Euclidean distance and the other on a modified Jaccard similarity measure for real-valued vectors. The latter is specifically designed to improve feature selection for high?dimensional sparse data. The effectiveness of the proposed approach is demonstrated on 12 benchmark datasets with feature dimensions ranging from 6 to 7070 using eight different classifiers. Experimental results demonstrate that the proposed Siamese Lasso framework consistently identifies compact and discriminative feature subsets maintaining competitive or superior classification performance, although the loss function used explicitly did not consider any regularizer to learn classification. In addition, the quality of the embeddings produced by these networks is assessed using manifold learning techniques such as UMAP and t-SNE.