Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do
作者: Zandi Eberstadt
分类: cs.CL
发布日期: 2026-07-28
💡 一句话要点
研究表明指令调优模型在语法重用上优于人类
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 语法收敛 大型语言模型 指令调优 自然语言处理 人机交互 对话系统 上下文重用
📋 核心要点
- 现有研究尚未明确大型语言模型在语法收敛方面是否与人类相似，尤其是在多种语法结构下的表现。
- 本文通过替换人类对话中的发言，分析指令调优模型在上下文无关语法规则重用方面的表现，提出了一种新的评估方法。
- 实验结果显示，指令调优模型在语法规则重用和语义相似性上均优于人类响应，且在不同模型架构中表现一致。
📝 摘要（中文）
语法收敛是人类对话中的一种普遍现象，指说话者在语言上适应对话者的语法特征。本文探讨大型语言模型是否在语法收敛方面表现出类似于人类的特征。通过对16个开放权重的Llama和Gemma模型进行实验，研究发现这些模型在生成的对话中显示出与前一个人类发言的上下文无关语法规则的重用率显著高于与随机人类发言的重用率。指令调优模型在自然输出与实际人类发言的重合度上也表现出更高的相似性，尽管与预训练模型相比，指令调优模型的输出与无关发言的重合度更高。
🔬 方法详解
问题定义：本文旨在探讨大型语言模型是否在语法收敛方面表现出与人类相似的特征，现有方法未能充分评估这一现象的普遍性和深度。
核心思路：通过使用替换范式数据，研究模型生成的对话与人类对话之间的语法规则重用情况，分析指令调优对模型表现的影响。
技术框架：研究涉及16个开放权重的Llama和Gemma模型，使用1,901个匹配位置进行实验，比较模型生成的输出与人类发言的重合度。
关键创新：提出了一种新的评估方法，通过替换人类对话中的发言，系统性地分析了指令调优模型在语法重用上的表现，揭示了其优于人类的特性。
关键设计：实验中关注上下文无关语法规则的重用，分析了不同频率规则的重用情况，并比较了指令调优模型与预训练模型在输出相似性方面的差异。
🖼️ 关键图片
📊 实验亮点
实验结果显示，所有模型在上下文无关语法规则的重用上均显著高于随机人类发言，且指令调优模型在自然输出与实际人类发言的重合度上表现更佳。具体而言，指令调优模型的输出在语义相似性上优于预训练模型，且在低频规则的重用上表现出更大的差异。
🎯 应用场景
该研究为大型语言模型在自然语言处理中的应用提供了新的视角，尤其是在对话系统和人机交互领域。通过提高模型的语法适应能力，可以增强其在真实对话场景中的表现，提升用户体验。未来，这一研究成果可能推动更智能的对话系统的发展，促进人机沟通的自然性和流畅性。
📄 摘要（原文）
Syntactic convergence (the tendency of speakers to adapt in language towards the grammatical profiles of their interlocutors) is a well-documented feature of human dialogue widely considered to operate below conscious awareness. Whether large language models exhibit analogous syntactic convergence toward human users relative to human baselines and across a broad range of syntactic constructions remains an open question. Using substitution-paradigm data in which model generations replace one speaker's turns in pre-existing human dialogues, this study measures turn-adjacent reuse of context-free grammar (CFG) rules across sixteen open-weight Llama and Gemma models (1B-70B, pretrained and instruction-tuned) at 1,901 matched positions per model. Every model showed greater CFG-rule overlap with the preceding human turn than with a sampled unrelated human prime, and in every model this actual-versus-random difference was larger for lower-frequency rules. Each instruction-tuned model also showed greater natural-output overlap with the actual prime than the human response it replaced, and all eight matched architecture pairs exhibited greater actual-prime overlap after instruction tuning. However, relative to pretrained variants, instruction-tuned outputs overlapped more with unrelated primes, showed a smaller actual-versus-random increment, and had lower conditional rule-reuse odds once target rule-set size was held constant. In exploratory analyses, each model exhibited greater mean lexical and semantic similarity to the preceding turn than the matched human responses did. Instruction-tuned models additionally produced responses with greater mean semantic similarity than their pretrained counterparts in all eight architecture pairs, whereas the lexical similarity results were more heterogeneous.