EviCast: Forecasting Retrieval Gains Through Shared Response Calibration

📄 arXiv: 2611.02387v1 📥 PDF

作者: Yibo Wang, Jisheng Dang, Bimei Wang, Jiongpei Hou, Jingwen Zhao, Wenjun Ma, Fei Shen, Juncheng Li, Meng Wang, Qi Tian, Bin Hu, Tat-Seng Chua

分类: cs.CL, cs.LG

发布日期: 2026-11-02

备注: 13 pages, 6 figures, 8 tables

🔗 代码/项目: GITHUB

💡 一句话要点

提出EviCast框架以优化检索增强生成模型的性能

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 检索增强生成 模型优化 信息检索 生成模型 性能评估

📋 核心要点

  1. 现有的检索增强生成方法无法有效判断检索器配置的价值，导致性能下降和资源浪费。
  2. EviCast框架通过干预证据状态分解，预测检索的价值，优化生成器的响应能力。
  3. 在实验中，EviCast在HotpotQA和MuSiQue上表现出色，减少了80%的生成调用，并在低预算情况下实现了最低的预测误差。

📝 摘要（中文）

检索增强生成（RAG）模型通过外部证据扩展大型语言模型，但相关证据并不总能被生成器有效利用。现有方法无法在大规模执行前判断生成器、任务分布和检索器配置的有效性。本文提出EviCast框架，通过干预证据状态分解来预测检索价值。EviCast仅需一次测量生成器对受控证据状态的响应，并将此共享特征与每个候选检索器的离线证据状态分布结合，得出端到端的净效用及其95%置信下界。实验结果显示，EviCast在多个配置中表现优异，显著降低了诊断生成调用的次数。

🔬 方法详解

问题定义：本文解决的问题是如何在检索增强生成模型中有效评估检索器配置的价值。现有方法在执行前无法判断生成器与检索器的适配性，导致性能下降和资源浪费。

核心思路：EviCast的核心思路是通过干预证据状态分解，预测检索的价值。该方法通过一次性测量生成器对不同证据状态的响应，结合候选检索器的离线证据状态分布，从而高效评估其效用。

技术框架：EviCast框架主要包括三个模块：1) 生成器响应测量模块，负责获取生成器对受控证据状态的响应；2) 离线证据状态分布模块，存储各候选检索器的证据状态信息；3) 效用评估模块，结合上述两者计算端到端的净效用及其置信下界。

关键创新：EviCast的创新点在于其通过共享响应特征与离线证据状态分布结合，避免了每个配置的重复执行，从而显著提高了效率。与现有方法相比，EviCast在评估检索器配置时更具统计效率。

关键设计：EviCast在设计上采用了状态细化分析，能够准确判断何时共享校准比单个检索器校准更有效。此外，EviCast-T实例通过捕捉终端和桥接证据之间的互补性，进一步提升了性能。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

EviCast在Frozen评估中成功判断了所有14个HotpotQA检索配置，预测误差仅为2.75 F1分，同时减少了80%的诊断生成调用。在低预算情况下，其预测误差在七个预测器中最低，显示出其优越的性能。

🎯 应用场景

EviCast框架在检索增强生成模型中具有广泛的应用潜力，尤其在需要高效信息检索和生成的场景，如问答系统、对话生成和信息检索等领域。其优化的检索策略能够显著提升生成模型的性能，降低资源消耗，具有重要的实际价值和未来影响。

📄 摘要（原文）

Retrieval-augmented generation (RAG) extends large language models with external evidence, yet relevant evidence does not ensure that a generator can use it. Indiscriminate retrieval can degrade task performance while adding retrieval and long-context inference costs. Existing methods neither determine before large-scale execution whether a generator, task distribution, and retriever configuration is worth deploying nor diagnose multiple candidate retrievers under a shared budget. We introduce EviCast, a framework that forecasts retrieval value through interventional evidence-state decomposition. EviCast measures a generator's response to controlled evidence states once and combines this shared profile with the offline evidence-state distribution of each candidate retriever, yielding the end-to-end net utility and its 95% lower confidence bound without executing every RAG configuration. A state-refinement analysis characterizes when its binary and topology-aware instantiations are exact and when shared calibration is statistically more efficient than per-retriever calibration. In frozen evaluations, EviCast correctly decides all 14 HotpotQA retrieval configurations with a forecast error of 2.75 F1 points while reducing diagnostic generation calls by 80%, and it attains the lowest forecast error among seven predictors in the low-budget regime. Its topology-aware instantiation, EviCast-T, captures the complementarity between terminal and bridge evidence, correctly deciding all 16 configurations on MuSiQue and HotpotQA and 23 of 24 after prospective transfer to a third dataset. These results show that retrieval deployment depends on both evidence availability and the generator's ability to convert evidence structure into stable net utility. Code is available at https://github.com/YiboWang2002/EviCast.