Context Is Not Free: Carbon-Aware LLM Routing for Multi-Turn LLM Web Services across Geo-Distributed Data Centers

📄 arXiv: 2610.35230v1 📥 PDF

作者: Xiaoyang Zhang, Yuejun Yan, Qingang Zhang, Yonggang Wen

分类: cs.DC, cs.AI, cs.CY

发布日期: 2026-10-26

💡 一句话要点

提出CarbonRouter以解决多轮LLM服务的碳排放问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 碳排放优化 多轮对话 大型语言模型 缓存感知 在线服务 可持续计算 云计算

📋 核心要点

  1. 现有的碳感知LLM路由器将对话的查询视为独立，忽略了历史重计算的碳成本，导致碳排放未得到有效控制。
  2. 本文提出CarbonRouter，通过联合选择模型、GPU类型和区域，采用缓存感知的碳成本模型，优化多轮对话的碳排放。
  3. 实验表明，CarbonRouter在多个真实数据集上相较于FairCarbon减少了16.1%的CO₂排放，并在长会话中表现更佳。

📝 摘要（中文）

随着大型语言模型（LLM）驱动的网络服务对环境的影响日益受到关注，这些服务通常在地理分布的数据中心运行，其电网碳强度因地区和时间而异。为了减少排放，本文提出CarbonRouter，一个在线框架，联合选择模型、GPU类型和区域，以最小化碳排放。CarbonRouter引入了一种缓存感知的碳成本模型，考虑历史重计算的碳成本，并通过轻量级门控机制保持会话在其缓存部署上。实验结果表明，CarbonRouter在两个真实对话数据集上相较于现有方法FairCarbon平均减少16.1%的CO₂排放，且在30轮以上的会话中，节省幅度更是达到23.1%。

🔬 方法详解

问题定义：本文旨在解决多轮对话中由于历史重计算导致的碳排放问题。现有方法未考虑会话状态，导致碳成本高昂。

核心思路：CarbonRouter通过引入缓存感知的碳成本模型，优化每个查询的模型、GPU类型和区域选择，减少不必要的历史重计算。

技术框架：CarbonRouter的整体架构包括三个主要模块：模型选择模块、GPU类型选择模块和区域选择模块，结合缓存信息进行决策。

关键创新：CarbonRouter的创新在于其缓存感知的碳成本模型，能够动态评估历史重计算的碳成本，与现有方法相比，显著降低了碳排放。

关键设计：在设计中，CarbonRouter使用了轻量级门控机制，确保会话在缓存部署上保持，避免频繁的历史重计算，同时通过缓存令牌计数来评估碳成本。

📊 实验亮点

实验结果显示，CarbonRouter在两个真实对话数据集上，相较于FairCarbon平均减少了16.1%的CO₂排放，且在对话轮次超过30时，节省幅度提升至23.1%。此外，CarbonRouter还改善了预测答案质量和估计延迟。

🎯 应用场景

该研究的潜在应用领域包括云计算服务、智能客服和在线教育等多轮对话系统。通过优化碳排放，CarbonRouter不仅有助于降低环境影响，还能提升服务的可持续性，符合未来绿色计算的发展趋势。

📄 摘要（原文）

The sustainability of the Web has drawn growing attention, as large language model (LLM)-based web services leave a substantial carbon footprint. These services typically run in geo-distributed data centers whose grid carbon intensity varies across regions and over time, so routing each query to a cleaner region or a smaller model can reduce emissions. A conversation, however, is a stateful web session: each turn reuses the key--value (KV) cache of its history, which resides only where earlier turns were served. Serving a turn on another deployment requires recomputing this history, whose carbon cost grows with the conversation and can outweigh the saving of the move. Existing carbon-aware LLM routers treat a conversation's queries as independent and ignore this cost. We present CarbonRouter, an online framework that jointly selects the model, GPU type, and region for each query to minimize carbon emissions. CarbonRouter introduces a cache-aware carbon cost model, which charges history recomputation to every candidate deployment using cached-token counts from the serving backends, and uses a lightweight gate that keeps a session on its cached deployment when it would likely move back at the next turn and rebuild its history again. We evaluate CarbonRouter in trace-driven experiments on two real-world conversation datasets with five LLMs, three GPU types, and four regions, where it achieves the lowest carbon of all policies. CarbonRouter reduces CO$_2$ by 16.1\% on average relative to FairCarbon, a state-of-the-art carbon-aware router with the same action space, while also improving predicted answer quality and estimated latency. Moreover, the saving rises further to 23.1\% for sessions longer than 30 turns.