Quantization Damage Is Multiplicative, Not Additive
作者: Zekun Wu, Swati Dhiman, Adriano Koshiyama
分类: cs.LG, cs.CL
发布日期: 2026-08-06
备注: 16 pages, 9 figures, 7 tables. Under review at the Third Workshop on Uncertainty-Aware NLP (UncertaiNLP), EMNLP 2026 (non-archival)
💡 一句话要点
提出量化损伤乘法模型以解决语言模型决策不确定性问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 量化损伤 决策边际 大型语言模型 安全性评估 模型优化 统计比较 机器学习
📋 核心要点
- 现有方法假设量化添加固定噪声，未能准确预测模型决策的变化，导致安全性评估失效。
- 本文提出通过测量决策边际来评估量化影响，发现量化通过乘法因子缩小边际，而非简单添加噪声。
- 实验结果显示，量化在不同位宽下导致的边际收缩显著影响决策，尤其在3位和2位时，模型的决策能力大幅下降。
📝 摘要（中文）
量化是大型语言模型实际部署的重要手段，但在低于四位时会造成显著损伤。现有研究假设量化会添加固定大小的噪声，导致决策的安全性受到影响。本文通过测量决策的边际，发现量化实际上是通过一个与位宽相关的因子来缩小边际，而非简单地添加噪声。我们在16个模型中进行实验，结果表明，量化导致的边际收缩会降低模型的决策保护能力，尤其在低位宽时，模型的偏差会影响决策方向。
🔬 方法详解
问题定义：本文旨在解决量化对大型语言模型决策的影响，现有方法未能准确捕捉量化带来的变化，导致安全性评估失效。
核心思路：通过测量模型决策的边际，即选择选项的得分与最佳替代选项得分之差，来评估量化的影响，发现量化是通过乘法因子缩小边际。
技术框架：研究涉及16个模型，8个模型家族，使用三种量化方法，位宽从8位降至2位，系统地跟踪决策边际的变化。
关键创新：提出了边际收缩的概念，表明量化并非简单添加噪声，而是通过乘法因子影响决策的安全性，挑战了现有的量化噪声模型。
关键设计：在实验中，使用了统计比较方法，预测翻转率的中位数为1.8个百分点，且每个模型的常数不具可转移性，强调了模型特定的量化影响。
🖼️ 关键图片
📊 实验亮点
实验结果表明，在3位和2位量化下，模型的决策能力显著下降，边际收缩导致的翻转率预测中位数为1.8个百分点，且在安全决策和工具调用方面，传统的加性噪声模型无法有效解释损伤。
🎯 应用场景
该研究对大型语言模型的安全性和可靠性具有重要影响，尤其在实际应用中，如自动化工具和决策支持系统。通过理解量化对模型决策的影响，可以更好地设计和优化模型，确保其在低位宽下的安全性和有效性。
📄 摘要（原文）
Quantization is how large language models are actually deployed, and below four bits it is known to hurt. What nobody can say is which of the model's decisions will change at a given bit-width. The damage is silent: a compressed agent stops calling its tools, then loses half its safety refusals, yet benchmark scores barely move. Prior work assumes quantization adds noise of a roughly fixed size, which would make confident decisions safe. We measure the decision itself instead. The margin of a two-way decision is the model's score for the option it picks minus the score of its best alternative; we track it before and after quantization across 16 models from 8 model families, three quantization methods, and bit-widths from 8 down to 2. Quantization does not add fixed-size noise to the margin. It multiplies the margin by a factor that collapses with bit-width (median 0.86 at 4 bits, 0.33 at 3, 0.00 at 2); we call this margin shrinkage. This contraction reduces the protection a large margin affords; the model's own small biases pick the direction of failure: at 3 bits the decision to call a tool collapses toward inaction while the choice of which tool is untouched. In fitted statistical comparison, additive-noise accounts never win on the damaged tool and safety decisions. The fitted relation predicts flip rates within a median of 1.8 percentage points on held-out decisions, though no flip was used in the fit; per decision, the predicted flip probabilities are calibrated uncertainty estimates (expected calibration error 0.004 over 131,758 predictions). The same form holds in every model we measure, but the constants are each model's own and do not transfer. A small paired margin set, measured per model and bit-width, estimates which decisions break without full generative evaluation; under our cost-matched tests, nothing repairs damage more cheaply than one more bit.