Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces

📄 arXiv: 2608.03401v1 📥 PDF

作者: Francesca Carlon, Vincent Ginis, Andres Algaba

分类: cs.LG, cs.AI

发布日期: 2026-08-04

备注: 52 pages, 13 figures, 30 tables

💡 一句话要点

评估推理接口以缩短推理时间和提高回答准确性

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 推理接口 大型语言模型 准确性提升 推理时间缩短 数字提示 训练设置 问答系统

📋 核心要点

  1. 现有大型语言模型在回答问题时推理时间较长，导致成本和延迟增加，影响用户体验。
  2. 本文提出通过数字提示和训练设置来缩短推理时间，同时评估不同模型在匹配推理水平下的表现。
  3. 实验结果表明，使用简洁提示可以显著提高准确性，且低努力推理的答案在某些情况下更为准确。

📝 摘要（中文）

大型语言模型通常在回答前进行较长时间的推理，导致成本和延迟增加。虽然提示和训练设置可以缩短推理时间，但较短的推理轨迹可能仅表明模型提前停止。本文评估了198个GPQA Diamond和500个MMLU-Pro问题在匹配推理水平下的配对运行。我们测试了一个数字/简洁提示，宣布Qwen3-14B的令牌限制，以及gpt-oss-20b和-120b的训练设置。结果显示，Qwen提示将推理轨迹缩短了12-17%，而在匹配令牌限制下的准确性变化较小且混合。简洁/早期回答指令在512个令牌时提高了MMLU-Pro的准确性3.8个百分点。对于gpt-oss，低和中等努力推理的候选-logit答案比匹配高努力答案准确性高出14.5-26.3个百分点。

🔬 方法详解

问题定义：本文旨在解决大型语言模型在回答问题时推理时间过长的问题，现有方法在推理效率和准确性上存在不足。

核心思路：通过引入数字提示和训练设置，缩短推理时间并提高回答的准确性，探索不同推理水平下模型的表现差异。

技术框架：研究设计了配对实验，分别在198个GPQA Diamond和500个MMLU-Pro问题上进行测试，比较不同模型在匹配推理水平下的表现。

关键创新：提出了数字/简洁提示的概念，能够有效缩短推理轨迹，并在某些情况下提高准确性，与传统方法相比具有显著优势。

关键设计：设置了令牌限制，测试了Qwen3-14B和gpt-oss-20b、-120b模型的不同努力水平，分析了完成和未完成运行的准确性差异。实验中关注了正确完成率、提前停止时的答案和概率分布等指标。

🖼️ 关键图片

img_0
img_1
img_2

📊 实验亮点

实验结果显示，Qwen提示将推理轨迹缩短12-17%，而在512个令牌时，简洁指令提高了MMLU-Pro的准确性3.8个百分点。gpt-oss模型的低和中等努力推理的答案准确性比高努力答案高出14.5-26.3个百分点，表明低努力推理在某些情况下更具优势。

🎯 应用场景

该研究的潜在应用领域包括智能问答系统、对话机器人和教育技术等。通过优化推理时间和提高回答准确性，可以显著提升用户体验和系统效率，具有广泛的实际价值和影响。

📄 摘要（原文）

Large language models often reason at length before answering, increasing cost and latency. Prompts and trained settings can shorten this reasoning, but a shorter trace may only show that the model stopped sooner. Here, we evaluate paired runs of the same question at matched reasoning horizons across 198 GPQA Diamond and 500 MMLU-Pro questions. We test a numeric/concision prompt that announces a token limit for Qwen3-14B and the trained effort settings of gpt-oss-20b and -120b. The Qwen prompt shortens reasoning traces by 12-17%, while accuracy changes at matched token limits are small and mixed. A concise/early-answer instruction raises MMLU-Pro accuracy by 3.8 percentage points at 512 tokens, including +2.7 points when both runs are unfinished. Its gain at 2,048 tokens is uncertain. For gpt-oss, candidate-logit answers from completed low- and medium-effort reasoning are 14.5-26.3 points more accurate than matched-horizon high-effort answers. Most of the 512-token advantage comes from lower effort finishing earlier, while differences among unfinished runs are smaller and mixed. Wrong early answers often concentrate probability on the chosen option, so earlier stopping does not uniformly improve probability quality. In these tests, a tight deadline can favor lower effort or a concise instruction, whereas allowing high effort to finish can recover higher final accuracy. Evaluations should report correct completion before a deadline, the answer obtained when a run is stopped, differences among unfinished runs, and probability assigned to the correct answer separately.