End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inference

📄 arXiv: 2606.27743v1 📥 PDF

作者: Yuhang Chen, Jinhao Duan, Ruichen Zhang, Mingfu Liang, Xiaohan Wei, Yunchen Pu, Fei Tian, Chonglin Sun, Parish Aggarwal, Frank Shyu, Luke Simon, Sandeep Pandey, Tianlong Chen, Xi Liu

分类: cs.IR, cs.AI, cs.LG

发布日期: 2026-06-26

💡 一句话要点

提出L2A框架以解决动态资源适应的LLM推理问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大语言模型 动态资源适应 推理优化 门控网络 计算效率

📋 核心要点

  1. 现有方法在动态云环境中表现不佳，静态模型无法适应资源波动，导致计算资源浪费或崩溃。
  2. 本文提出L2A框架，将推理视为受限分配问题，结合输入和资源预算，设计了自适应的门控网络。
  3. L2A模型在GSM8K上实现了34%的层稀疏性，准确性与密集基线相差仅0.6%，显著优于静态或启发式基线。

📝 摘要（中文）

大语言模型（LLMs）推理通常在静态资源假设下进行，导致在动态云环境中表现不佳。为了解决这一问题，本文提出了学习分配（L2A）框架，将推理视为一个受限分配问题，考虑输入和运行时资源预算。通过集成轻量级的预算条件和输入感知的门控网络，L2A模型能够根据实时资源动态自适应配置计算负载，最大化推理深度，同时在资源紧张时保持节俭。实验表明，L2A在GSM8K数据集上实现了高达34%的层稀疏性，且在准确性上与密集基线相差仅0.6%。

🔬 方法详解

问题定义：本文旨在解决大语言模型在动态云环境中推理时的资源适应性问题。现有方法通常基于静态资源假设，无法应对资源波动，导致计算资源的浪费或模型崩溃。

核心思路：论文提出的L2A框架将推理过程视为一个受限分配问题，考虑输入特征和运行时资源预算，设计了轻量级的门控网络以实现动态适应。

技术框架：L2A框架包括三个主要模块：输入特征分析、资源预算评估和门控网络控制。模型根据实时资源动态调整计算负载，优化推理性能。

关键创新：L2A的创新在于其预算感知的自适应策略，超越了仅基于输入难度的传统方法，能够在资源充足时最大化推理深度，在资源紧张时保持计算效率。

关键设计：模型通过统一目标函数进行训练，优化任务性能、逻辑一致性和资源成本，关键参数包括门控网络的结构和损失函数的设计，以确保在不同资源条件下的有效性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

L2A模型在GSM8K数据集上实现了高达34%的层稀疏性，且在准确性上与密集基线相差仅0.6%。相比之下，所有静态或启发式基线模型均需单独调优，且在相似推理时间下准确性下降5-10%。

🎯 应用场景

该研究在云计算、边缘计算等动态资源环境中具有广泛的应用潜力，能够提高大语言模型的推理效率和资源利用率，适用于实时对话系统、智能客服等场景。未来，L2A框架可能推动更智能的资源管理和分配策略的发展。

📄 摘要（原文）

Large Language Models (LLMs) inference is typically deployed under a static resource assumption, where models execute a fixed computational graph regardless of the runtime environment. However, real-world cloud infrastructure is inherently dynamic, characterized by fluctuating availability (e.g., spot instance preemption) and tiered Quality-of-Service requirements. In such volatile settings, static models are inflexible: they either crash under resource constraints or waste compute on redundant operations. To bridge this gap, we propose Learning to Allocate (L2A), an end-to-end framework for resource-adaptive inference. Unlike prior methods that condition only on input difficulty, we formulate inference as a constrained allocation problem conditioned on both the input and the runtime resource budget itself. We introduce lightweight, budget-conditioned and input-aware gating networks integrated into the LLM. These gates are trained via a unified objective that jointly optimizes task performance, logical consistency, and resource costs along three axes matching how real-world dynamics manifest: layer skipping for memory and depth pressure, head pruning for throughput contention, and reasoning-token reduction for latency tightening. This lets the model learn a budget-aware policy beyond input difficulty alone: it adaptively configures its computational footprint with respect to real-time resource dynamics, maximizing reasoning depth when resources permit while enforcing strict frugality when budgets tighten. A single L2A model traces the entire compute-accuracy Pareto frontier on Llama-3-8B and Qwen-3-4B: at up to 34% realized layer sparsity, it stays within 0.6% of the dense baseline on GSM8K, with the same gap holding zero-shot on out-of-distribution tasks, while every static or heuristic baseline requires a separately tuned model and still drops by 5-10% at comparable inference time.