TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
作者: Zhixiong Zhao, Zukang Xu, Zhixuan Chen, Xing Hu, Zhe Jiang, Dawei Yang
分类: cs.LG, cs.AI
发布日期: 2026-06-11 (更新: 2026-06-12)
备注: Accepted by ICML 2026
🔗 代码/项目: GITHUB
💡 一句话要点
提出TWLA框架以实现大语言模型的三元权重和低位激活量化
🎯 匹配领域: 支柱八：物理动画 (Physics-based Animation) 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 量化技术 大语言模型 三元化 激活压缩 后训练量化 推理加速 模型压缩
📋 核心要点
- 现有的量化方法在处理重尾激活分布时存在不足，导致激活保持高精度，限制了推理加速。
- TWLA框架通过后训练量化实现三元权重和低位激活量化，采用E2M-ATQ、KOTMS和ILA-AMP三个组件。
- 实验结果显示，TWLA在W1.58A4下保持高准确性，并显著提升推理速度，展示了其有效性。
📝 摘要（中文）
大语言模型（LLMs）在语言处理方面表现出色，但其内存和计算成本限制了部署。三元化作为一种有前景的压缩技术，能够显著减少模型大小和推理复杂度。然而，现有方法在处理重尾激活分布时表现不佳，导致激活保持高精度，从而限制了端到端推理加速。为了解决这一问题，本文提出了TWLA框架，通过后训练量化实现1.58位权重压缩和4位激活量化，同时保持高准确性。TWLA包含三个组件：E2M-ATQ、KOTMS和ILA-AMP，广泛实验表明TWLA在W1.58A4下保持高准确性，并显著加速推理。
🔬 方法详解
问题定义：本文旨在解决大语言模型在量化过程中由于重尾激活分布导致的高精度激活问题，限制了推理加速的现象。
核心思路：TWLA框架通过后训练量化实现权重三元化和激活低位化，采用多阶段优化和共享旋转技术，以降低模型复杂度并提高推理效率。
技术框架：TWLA由三个主要模块组成：E2M-ATQ用于权重三元化优化，KOTMS用于激活分布的重塑，ILA-AMP用于层间激活混合精度的优化。
关键创新：E2M-ATQ通过欧几里得初始化到流形重定位的两阶段优化，显著降低了层输出误差；KOTMS通过克罗内克结构的正交旋转抑制激活异常值；ILA-AMP则考虑了相邻层的二阶交互成本，优化了激活量化的位分配。
关键设计：在E2M-ATQ中，采用了特定的损失函数以最小化输出误差；KOTMS中的共享旋转设计用于统计抑制异常值；ILA-AMP则通过引入层间交互成本，防止弱层引发的级联效应。
🖼️ 关键图片
📊 实验亮点
实验结果表明，TWLA在W1.58A4量化配置下，能够保持高准确性，同时实现显著的推理加速，展示了1.58位权重和4位激活的有效性，相较于传统方法有明显的性能提升。
🎯 应用场景
TWLA框架在大语言模型的量化和加速方面具有广泛的应用潜力，能够有效降低模型的内存和计算需求，适用于边缘计算、移动设备和实时应用等场景。未来，该技术可能推动更大规模模型的部署与应用，提升自然语言处理任务的效率。
📄 摘要（原文）
Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment. Ternarization has emerged as a promising compression technique, offering significant reductions in model size and inference complexity. However, existing methods struggle with heavy-tailed activation distributions and therefore keep activations in high precision, fundamentally limiting end-to-end inference acceleration. To overcome this limitation, we propose TWLA, a post-training quantization (PTQ) framework that achieves 1.58-bit weight compression and 4-bit activation quantization while maintaining high accuracy. TWLA comprises three components: (1) Euclidean-to-Manifold Asymmetric Ternary Quantizer (E2M-ATQ) minimizes layer-output error under weight ternarization via a two-stage optimization from Euclidean initialization to manifold relocation; (2) Kronecker Orthogonal Tri-Modal Shaping (KOTMS) applies a Kronecker-structured orthogonal rotation to reshape weights into ternary-friendly tri-modal distributions, while the shared rotation statistically suppresses activation outliers; and (3) Inter-Layer Aware Activation Mixed Precision (ILA-AMP) explicitly introduces adjacent-layer second-order interaction costs in bit allocation and jointly optimizes for the layer-wise disparity of activation quantization gains induced by the shared orthogonal transform, preventing cascades triggered by a few weak layers. Extensive experiments demonstrate that TWLA maintains high accuracy under W1.58A4, while delivering significant inference acceleration. The code is available at https://github.com/Kishon-zzx/TWLA.