Batch-wise Adaptive Pruning: Periodic Neuron Activation-Aware Weight Pruning for Language Reasoning Model

📄 arXiv: 2608.14003v1 📥 PDF

作者: Yongmin Kim, Shota Takashiro, Yusuke Iwasawa, Takeshi Kojima, Yutaka Matsuo

分类: cs.CL

发布日期: 2026-08-14

备注: Accepted at COLM 2026. 28 pages, 12 figures, 18 tables. Code: https://github.com/matsuolab/batch-wise-prune

💡 一句话要点

提出批量自适应剪枝以解决推理模型推理效率问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 自适应剪枝 推理模型 计算效率 激活记忆 周期性选择 自然语言处理 深度学习

📋 核心要点

  1. 现有的自适应剪枝方法在批量推理中表现不佳，导致准确率下降和计算效率低下。
  2. 本文提出了一种新颖的训练无关自适应剪枝方法，采用周期性top-k选择和激活记忆机制，适应批量推理的特点。
  3. 实验结果显示，该方法在多个推理基准上超越了现有的最先进方法，显著提升了准确性和推理速度。

📝 摘要（中文）

大型推理模型（LRMs）在复杂任务中表现出色，但在推理过程中消耗大量计算资源。现有的训练无关自适应剪枝方法在批量推理时效果显著下降，导致准确率下降。本文提出了一种专门针对LRMs批量推理的训练无关自适应剪枝方法，采用周期性top-k选择和激活记忆机制，显著提高了推理效率和准确性。实验结果表明，该方法在DeepSeek-R1-Distill-Qwen-7B模型上，平均准确率提升39.7个百分点，并在50%实际稀疏度下实现了1.40倍的速度提升。

🔬 方法详解

问题定义：本文旨在解决大型推理模型在批量推理时的计算效率和准确率问题。现有方法在批量推理中由于共享剪枝掩码，导致激活分布失配，准确率显著下降。

核心思路：提出了一种训练无关的自适应剪枝方法，采用周期性top-k选择替代阈值选择，避免了聚合引起的激活分布偏移，同时引入激活记忆机制，保留重要神经元。

技术框架：该方法包括两个主要模块：周期性top-k选择模块和激活记忆模块。前者在每个更新周期内选择重要神经元，后者则积累重要性分数以保留重复激活的神经元。

关键创新：最重要的创新在于引入了周期性top-k选择和激活记忆机制，这与现有方法的阈值选择和单次激活评估形成了本质区别，显著提高了批量推理的准确性和效率。

关键设计：在参数设置上，选择了合适的更新周期和top-k比例，以确保在保持速度的同时最大化准确性。激活记忆机制通过累积重要性分数来优化神经元的保留策略。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，在DeepSeek-R1-Distill-Qwen-7B模型上，本文方法在批量大小为4、目标稀疏度为50%的情况下，平均准确率提升39.7个百分点，并实现了1.40倍的推理速度提升，显著优于现有最先进的自适应剪枝方法。

🎯 应用场景

该研究的潜在应用领域包括自然语言处理、智能问答系统和复杂推理任务等。通过提高推理效率和准确性，该方法能够在实际生产环境中降低计算成本，提升用户体验，具有重要的实际价值和广泛的应用前景。

📄 摘要（原文）

Large Reasoning Models (LRMs) achieve strong performance on complex tasks through extended chain-of-thought generation, but incur substantial computational costs during inference. In production settings, batched inference is essential for high throughput, yet the existing training-free adaptive pruning methods we evaluate severely degrade in this regime. Because a batch must share a single pruning mask, these methods aggregate activations across samples and then apply threshold-based selection; the threshold, calibrated offline on unaggregated activations, no longer matches the aggregated distribution, so the realized sparsity ratio drifts and accuracy on reasoning tasks collapses under batched inference. In this work, we propose a training-free adaptive pruning method designed specifically for batched inference in LRMs, built on two components. First, we replace threshold-based selection with periodic top-k selection over the aggregated importance scores, which is unaffected by the shift that aggregation induces in the activation distribution, and which runs selection once per update period rather than at every token, preserving the speedup. Second, based on the observation that important neurons re-fire periodically during long reasoning generation, we introduce an activation memory that accumulates importance across update phases so that recurring neurons are retained. Experiments on diverse reasoning benchmarks demonstrate that our method outperforms the previous state-of-the-art adaptive pruning method by 39.7 percentage points in average accuracy at batch size 4 with 50% target sparsity on DeepSeek-R1-Distill-Qwen-7B, and reaches 1.40x speedup over dense inference at 50% actual sparsity.