Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B
作者: Iliya Mirzaei
分类: cs.CL, cs.AI, cs.LG
发布日期: 2026-07-30
💡 一句话要点
提出重复采样方法以超越自我反思和自我修正的局限性
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 语言模型 自我反思 重复采样 自然语言处理 模型评估 实验设计 统计分析
📋 核心要点
- 现有方法如自我反思和自我修正生成大量文本，导致难以判断其有效性。
- 论文提出通过重复采样相同问题并选择最常见答案的方法，作为对比基线。
- 实验结果表明，重复采样在相同成本下的表现优于其他方法，尤其是在较大模型中。
📝 摘要（中文）
本文探讨了语言模型在自我反思和自我修正过程中的局限性，指出这些方法往往生成更多文本，从而提高准确性，但并不一定反映方法本身的有效性。通过对比七种方法在不同参数模型上的表现，发现重复采样在相同成本下的表现优于其他方法。实验结果显示，自我检查方法在大多数情况下效果不佳，尤其是在更大的模型中，强调了简单重复采样的有效性。研究还提供了代码和生成结果，供后续研究使用。
🔬 方法详解
问题定义：本文旨在解决语言模型在自我反思和自我修正过程中生成过多文本的问题，这使得评估其有效性变得复杂。现有方法未能在相同成本下提供可靠的性能提升。
核心思路：论文提出的核心思路是通过重复采样相同问题并选择最常见的答案，来简化模型的决策过程，从而提高准确性。此方法的设计意在减少模型自我检查带来的不确定性。
技术框架：整体架构包括七种不同的方法，使用1.5B、3B和7B参数的开放模型，针对两个数学基准进行实验。每种方法生成的所有token都被计入，包括自我批评和反思的token。
关键创新：最重要的技术创新在于通过设计实验，系统地比较了不同方法在相同成本下的表现，发现重复采样在大多数情况下优于自我检查方法。这一发现挑战了传统的自我反思和自我修正的有效性假设。
关键设计：实验中采用了引导抽样和引导间隔的统计方法，使用了引导法和多重性校正来确保结果的可靠性。所有36个比较都是成对进行的，确保了实验的严谨性。实验还提供了代码和生成结果，便于后续验证。
🖼️ 关键图片
📊 实验亮点
实验结果显示，在相同的成本下，重复采样方法在所有36个比较中均未被其他方法超越，且有10种方法表现显著不佳。尤其是在7B参数模型中，自我检查方法的表现持续低于基线，强调了简单重复采样的优势。
🎯 应用场景
该研究的潜在应用领域包括自然语言处理、对话系统和自动问答等。通过优化语言模型的生成策略，可以提高模型在实际应用中的准确性和效率，减少不必要的计算开销，具有重要的实际价值和未来影响。
📄 摘要（原文）
Methods that make a language model plan, criticise and rewrite its own answer, reflect on mistakes, pick the best of several attempts, or debate with copies of itself nearly all make it generate far more text than a single chain of thought. Because generating more text raises accuracy by itself, a gain over one chain of thought does not show the method's idea is what helped. Wang et al. (2024) reported that a simple baseline, sampling the same question repeatedly and keeping the most common answer, often wins once budgets are comparable, but gave point estimates with no confidence intervals or significance tests. We rerun that comparison as a designed experiment: seven methods, open models of 1.5B, 3B and 7B parameters, two mathematics benchmarks, 150 questions each. We count every generated token, including those spent on critiques, reflections, debate turns and checking, and compare each method against repeated sampling at its own measured cost. All 36 comparisons are paired by question, with bootstrap intervals and multiplicity correction. No method is reliably better than repeated sampling at equal cost anywhere. Ten are reliably worse, all of them methods where the model inspects its own output, and all 18 self-inspection comparisons are negative. The two kinds of self-inspection part company as models grow. Choosing stops hurting: taking Best-of-N's eight samples and just counting the most common answer beats letting the model pick by 8.0 and 11.3 points at 1.5B, but only 2.0 and 1.3 at 7B, no longer distinguishable from zero. Rewriting does not recover: Self-Refine and a forced Reflexion stay 3.6 to 10.1 points below baseline at 7B. Reflexion as published never triggered its own retry on the smallest model. It judged itself correct every time and silently became a single chain of thought. We release code, prompts, all generations, and our verification scripts.