AffineQuant: Affine Transformation Quantization for Large Language Models

📄 arXiv: 2403.12544v1 📥 PDF

作者: Yuexiao Ma, Huixia Li, Xiawu Zheng, Feng Ling, Xuefeng Xiao, Rui Wang, Shilei Wen, Fei Chao, Rongrong Ji

分类: cs.LG

发布日期: 2024-03-19

备注: ICLR 2024

💡 一句话要点

提出AffineQuant以解决大规模语言模型量化问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 后训练量化 仿射变换 大规模语言模型 模型压缩 性能提升 深度学习优化

📋 核心要点

  1. 现有的后训练量化方法在优化范围上受到限制，无法有效减少量化误差。
  2. 本文提出AffineQuant，通过直接优化仿射变换，扩展了优化范围，提升了量化效果。
  3. 实验表明，AffineQuant在多种LLM上表现出显著的性能提升，创造了新的PTQ基准。

📝 摘要（中文）

大规模语言模型（LLMs）对资源的巨大需求引发了对压缩和加速神经网络技术的关注。后训练量化（PTQ）因其压缩效率和成本效益而受到重视。然而，现有PTQ方法仅限于优化量化前后权重之间的缩放变换。本文提出了一种新的PTQ方法AffineQuant，直接使用等效的仿射变换进行优化，显著减少量化误差。通过引入逐步掩码优化方法，确保变换的可逆性，从而保持PTQ的效率和泛化能力。实验结果显示，在不同LLMs和数据集上，AffineQuant显著提升了性能，LLaMA2-7B模型在W4A4量化下的C4困惑度达到15.76，且在零-shot任务中，LLaMA-30B模型的4/4位量化准确率达到58.61，创造了PTQ的新基准。

🔬 方法详解

问题定义：本文旨在解决现有后训练量化（PTQ）方法在优化范围上的局限性，导致量化误差较大，影响模型性能。

核心思路：AffineQuant通过直接优化仿射变换，扩展了优化的范围，确保量化前后输出的一致性，从而减少量化误差。

技术框架：该方法包括仿射变换的优化过程和逐步掩码优化策略。首先优化对角元素，然后逐步扩展到其他元素，确保变换的可逆性。

关键创新：AffineQuant的主要创新在于引入了仿射变换的直接优化，区别于传统方法仅优化缩放变换，理论上提高了可逆性和量化效果。

关键设计：在优化过程中，采用逐步掩码优化方法，确保变换的可逆性，符合Levy-Desplanques定理，优化过程中关注对角元素的调整。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在实验中，AffineQuant在LLaMA2-7B模型的W4A4量化下实现了15.76的C4困惑度，相较于OmniQuant降低了2.26。同时，在LLaMA-30B模型的零-shot任务中，4/4位量化的准确率达到58.61，较OmniQuant提升了1.98，创造了PTQ的新标杆。

🎯 应用场景

AffineQuant的研究成果在大规模语言模型的压缩和加速方面具有广泛的应用潜力，能够有效提升模型在资源受限环境下的运行效率。未来，该方法可推广至其他深度学习模型的量化和优化，推动更高效的人工智能应用发展。

📄 摘要（原文）

The significant resource requirements associated with Large-scale Language Models (LLMs) have generated considerable interest in the development of techniques aimed at compressing and accelerating neural networks. Among these techniques, Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its noteworthy compression efficiency and cost-effectiveness in the context of training. Existing PTQ methods for LLMs limit the optimization scope to scaling transformations between pre- and post-quantization weights. In this paper, we advocate for the direct optimization using equivalent Affine transformations in PTQ (AffineQuant). This approach extends the optimization scope and thus significantly minimizing quantization errors. Additionally, by employing the corresponding inverse matrix, we can ensure equivalence between the pre- and post-quantization outputs of PTQ, thereby maintaining its efficiency and generalization capabilities. To ensure the invertibility of the transformation during optimization, we further introduce a gradual mask optimization method. This method initially focuses on optimizing the diagonal elements and gradually extends to the other elements. Such an approach aligns with the Levy-Desplanques theorem, theoretically ensuring invertibility of the transformation. As a result, significant performance improvements are evident across different LLMs on diverse datasets. To illustrate, we attain a C4 perplexity of 15.76 (2.26 lower vs 18.02 in OmniQuant) on the LLaMA2-7B model of W4A4 quantization without overhead. On zero-shot tasks, AffineQuant achieves an average of 58.61 accuracy (1.98 lower vs 56.63 in OmniQuant) when using 4/4-bit quantization for LLaMA-30B, which setting a new state-of-the-art benchmark for PTQ in LLMs.