iScore: Visual Analytics for Interpreting How Language Models Automatically Score Summaries

📄 arXiv: 2403.04760v1 📥 PDF

作者: Adam Coscia, Langdon Holmes, Wesley Morris, Joon Suh Choi, Scott Crossley, Alex Endert

分类: cs.HC, cs.AI, cs.CY, cs.LG

发布日期: 2024-03-07

备注: Accepted to IUI 2024. 16 pages, 5 figures, 1 table. For a demo video, see https://youtu.be/EYJX-_fQPf0 . For a live demo, visit https://adamcoscia.com/papers/iscore/demo/ . The source code is available at https://github.com/AdamCoscia/iScore

DOI: 10.1145/3640543.3645142

💡 一句话要点

提出iScore以解决语言模型评分透明性问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 语言模型 可视化分析 教育技术 自动评分 模型可解释性

📋 核心要点

  1. 现有的LLMs在教育环境中的应用面临透明性不足和信任度低的问题，尤其在评分摘要时表现不佳。
  2. 本文提出的iScore工具通过交互式可视化分析，帮助学习工程师更好地理解和评估LLMs的评分机制。
  3. 在为期一个月的实验中，使用iScore的学习工程师将模型评分准确性提高了三个百分点，验证了其有效性。

📝 摘要（中文）

随着大型语言模型（LLMs）在教育工具中的广泛应用，理解和评估这些模型变得至关重要。然而，LLMs的庞大规模和参数数量使得其透明性不足，影响了信任度。为此，本文通过与学习工程师的协作设计，开发了iScore，一个交互式可视化分析工具，帮助用户上传、评分和比较多个摘要。iScore允许用户迭代修改摘要语言，跟踪LLM评分变化，并在多个抽象层次上可视化模型权重。通过与三位学习工程师的为期一个月的部署，iScore显著提高了模型评分的准确性，并通过访谈揭示了其在理解和信任构建方面的价值。

🔬 方法详解

问题定义：本文旨在解决大型语言模型在评分摘要时缺乏透明性和可解释性的问题。现有方法在处理大文本输入、追踪评分来源和可解释性方法的扩展性方面存在显著挑战。

核心思路：iScore的核心思路是通过交互式可视化工具，帮助学习工程师实时修改摘要并观察评分变化，从而提升对模型的理解和信任。

技术框架：iScore的整体架构包括多个模块：用户上传摘要、模型评分、实时反馈和可视化展示。用户可以在不同层次上查看模型权重，进行多次迭代。

关键创新：iScore的主要创新在于其交互式设计，允许用户在评分过程中进行实时修改和反馈，这与传统的静态评分方法形成鲜明对比。

关键设计：iScore设计了紧密集成的视图，支持用户在多个抽象层次上可视化模型权重，并通过迭代修改摘要语言来观察评分变化，增强了模型的可解释性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在与三位学习工程师的为期一个月的实验中，iScore显著提高了模型评分的准确性，具体提升幅度达到三个百分点。这一结果表明，iScore在理解和信任构建方面具有重要价值。

🎯 应用场景

iScore的潜在应用领域包括教育技术、自动评分系统和语言处理工具。通过提升模型的透明性和可解释性，iScore能够帮助教育工作者更有效地利用LLMs进行教学和评估，促进学习效果的提升。未来，该工具可能在更广泛的领域中应用，如内容生成和自动化评估。

📄 摘要（原文）

The recent explosion in popularity of large language models (LLMs) has inspired learning engineers to incorporate them into adaptive educational tools that automatically score summary writing. Understanding and evaluating LLMs is vital before deploying them in critical learning environments, yet their unprecedented size and expanding number of parameters inhibits transparency and impedes trust when they underperform. Through a collaborative user-centered design process with several learning engineers building and deploying summary scoring LLMs, we characterized fundamental design challenges and goals around interpreting their models, including aggregating large text inputs, tracking score provenance, and scaling LLM interpretability methods. To address their concerns, we developed iScore, an interactive visual analytics tool for learning engineers to upload, score, and compare multiple summaries simultaneously. Tightly integrated views allow users to iteratively revise the language in summaries, track changes in the resulting LLM scores, and visualize model weights at multiple levels of abstraction. To validate our approach, we deployed iScore with three learning engineers over the course of a month. We present a case study where interacting with iScore led a learning engineer to improve their LLM's score accuracy by three percentage points. Finally, we conducted qualitative interviews with the learning engineers that revealed how iScore enabled them to understand, evaluate, and build trust in their LLMs during deployment.