A Novel Multi-Stage Prompting Approach for Language Agnostic MCQ Generation using GPT
作者: Subhankar Maity, Aniket Deroy, Sudeshna Sarkar
分类: cs.CL
发布日期: 2024-01-13
备注: Accepted at ECIR 2024(short paper)
💡 一句话要点
提出多阶段提示方法以提升语言无关的MCQ生成质量
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
📋 核心要点
- 现有的单阶段提示方法在生成多项选择题时，往往难以产生高质量的干扰项，影响题目的有效性。
- 论文提出的多阶段提示方法通过链式思维提示，逐步引导GPT模型生成更具连贯性和质量的MCQ。
- 实验结果表明，MSP方法在自动评估和人类评估中均表现优异，显著提升了生成问题的语法性和难度。
- method_zh
📝 摘要（中文）
本文介绍了一种多阶段提示方法（MSP），用于生成多项选择题（MCQs），充分利用了GPT模型（如text-davinci-003和GPT-4）在各种自然语言处理任务中的优越性。该方法引入了链式思维提示的创新概念，通过一系列相互关联的提示来指导MCQ生成过程。自动评估结果表明，MSP方法在生成高质量干扰项方面显著优于传统的单阶段提示（SSP）基线。此外，一次性MSP技术在多种语言（包括英语、德语、孟加拉语和印地语）中提升了干扰项生成的效果。人类评估显示，使用该方法生成的问题在语法性、可回答性和难度上表现优越，突显了其在多种语言中的有效性。
🖼️ 关键图片
📄 摘要（原文）
We introduce a multi-stage prompting approach (MSP) for the generation of multiple choice questions (MCQs), harnessing the capabilities of GPT models such as text-davinci-003 and GPT-4, renowned for their excellence across various NLP tasks. Our approach incorporates the innovative concept of chain-of-thought prompting, a progressive technique in which the GPT model is provided with a series of interconnected cues to guide the MCQ generation process. Automated evaluations consistently demonstrate the superiority of our proposed MSP method over the traditional single-stage prompting (SSP) baseline, resulting in the production of high-quality distractors. Furthermore, the one-shot MSP technique enhances automatic evaluation results, contributing to improved distractor generation in multiple languages, including English, German, Bengali, and Hindi. In human evaluations, questions generated using our approach exhibit superior levels of grammaticality, answerability, and difficulty, highlighting its efficacy in various languages.