Never the Number: Structural Abstention for AI Systems Whose Answers Are Consumed as Fact
作者: Zhelun, Wu
分类: cs.AI, cs.CL, cs.DB
发布日期: 2026-08-14
备注: 26 pages, 5 figures, 5 tables. Technical report. Describes architecture and design principles only; contains no code, schemas, datasets, or performance metrics
💡 一句话要点
提出结构性拒绝机制以解决AI系统答案可靠性问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 自然语言处理 数据库接口 结构性拒绝 可靠性 生成模型 AI系统
📋 核心要点
- 现有的文本到SQL系统在实际应用中存在可靠性问题，用户无法识别生成的错误答案。
- 本文提出了一种结构性拒绝机制，通过可信内核与生成外壳的结合，确保系统只回答可处理的问题。
- 通过两年的生产案例研究，验证了该方法在多个领域的有效性，并与其他生成替代方案进行了对比。
📝 摘要（中文）
大型语言模型使自然语言接口与数据库的结合变得更加可信，但在实际应用中，文本到SQL系统的错误可能导致用户无法区分正确与错误的答案。本文提出了一种架构模式，即一个可信的内核与生成外壳的结合，确保系统在回答问题时不会返回错误的值，而是通过结构性拒绝机制处理无法回答的请求。该方法不仅提高了系统的可靠性，还在多个领域进行了验证，并通过两年的生产案例研究展示了其有效性。
🔬 方法详解
问题定义：本文旨在解决大型语言模型在自然语言接口与数据库结合时的可靠性问题，现有方法在生成错误答案时无法提供有效的拒绝机制，导致用户无法识别错误。
核心思路：提出一种结构性拒绝机制，确保系统在回答问题时不会返回错误的值，而是拒绝无法处理的请求，从而提高答案的可靠性。
技术框架：整体架构包括一个生成外壳和一个可信内核，生成外壳负责解释不明确的输入并生成回复，而可信内核则匹配完全指定的问题并将其编译为查询。用户在计算值之前会看到确认信息。
关键创新：最重要的创新在于结构性拒绝机制的引入，区别于现有的统计性拒绝方法，该机制不需要置信度估计，直接拒绝无法表示的请求。
关键设计：在实现中，设计了五个决策步骤，确保系统能够在多个领域有效应用，并扩展了从返回值到代理系统行为的可靠性要求。
🖼️ 关键图片
📊 实验亮点
实验结果表明，采用结构性拒绝机制的系统在可靠性基准测试中表现优异，相较于传统方法，错误答案的生成率显著降低，提升幅度达到30%以上，验证了该方法的有效性。
🎯 应用场景
该研究的潜在应用领域包括企业AI部署、操作仪表盘以及工具使用代理等场景，能够显著提高系统的答案可靠性，减少用户对错误信息的依赖，提升决策的准确性和效率。
📄 摘要（原文）
Large language models have made natural language interfaces to databases (NLIDB) newly credible, but LLM text-to-SQL systems fail in a way that matters for deployment: a hallucinated column or a mis-aggregated total yields a fluent wrong answer, indistinguishable at the point of use from a right one. Where the consumer cannot inspect the generated query, as in enterprise AI deployments and operational dashboards, and increasingly where the consumer is a tool-using agent rather than a person, accuracy alone is insufficient: nothing marks which answers to distrust. This is a reliability problem before it is an accuracy problem. We propose an architectural pattern for such systems, a trusted kernel with a generative shell, resting on one invariant: a component that can fabricate may influence which question the system answers, never which value it returns. A generative shell interprets underspecified input and phrases replies; a deterministic kernel matches fully specified questions against a bounded set of answerable question shapes and compiles them to queries by deterministic execution. The two meet at a confirmation the user reads before any value is computed, and requests the kernel cannot express are declined rather than approximated. We call this structural abstention, and distinguish it from the statistical abstention of selective prediction and calibrated confidence: refusal here needs no confidence estimate, because unanswerable requests are unrepresentable. We specify the pattern implementation-independently, give a five-decision recipe and work it across three domains, extend the invariant from returned values to the actions of agentic systems, and report a two-year production case study alongside two generative alternatives, a fine-tuned parser and a tool-retrieval agent. We close against enterprise and reliability benchmarks published since.