Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents

📄 arXiv: 2402.11208v2 📥 PDF

作者: Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, Xu Sun

分类: cs.CR, cs.AI, cs.CL

发布日期: 2024-02-17 (更新: 2024-10-29)

备注: Accepted at NeurIPS 2024, camera ready version. Code and data are available at https://github.com/lancopku/agent-backdoor-attacks

💡 一句话要点

提出针对LLM代理的后门攻击研究以提升安全性

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

📋 核心要点

  1. 当前LLM代理的安全性问题尚未得到充分重视，后门攻击的潜在威胁亟待研究。
  2. 本文提出了一种通用的代理后门攻击框架，并分析了多种后门攻击形式，揭示了其隐蔽性和多样性。
  3. 实验结果表明，LLM代理在后门攻击下表现出显著的脆弱性，现有防御方法无法有效应对，需进一步研究。
  4. method_zh

📝 摘要（中文）

随着大型语言模型（LLMs）的快速发展，基于LLM的代理在金融、医疗和购物等多个实际应用中得到了广泛应用。然而，LLM代理的安全性问题尚未得到充分探索。本文首次研究了LLM代理的后门攻击，提出了一种通用的代理后门攻击框架，并分析了不同形式的后门攻击。研究表明，LLM代理在面对后门攻击时存在严重的脆弱性，现有的文本后门防御算法难以有效缓解这一问题，亟需针对性的防御研究。

🖼️ 关键图片

fig_0
fig_1
fig_2

📄 摘要（原文）

Driven by the rapid development of Large Language Models (LLMs), LLM-based agents have been developed to handle various real-world applications, including finance, healthcare, and shopping, etc. It is crucial to ensure the reliability and security of LLM-based agents during applications. However, the safety issues of LLM-based agents are currently under-explored. In this work, we take the first step to investigate one of the typical safety threats, backdoor attack, to LLM-based agents. We first formulate a general framework of agent backdoor attacks, then we present a thorough analysis of different forms of agent backdoor attacks. Specifically, compared with traditional backdoor attacks on LLMs that are only able to manipulate the user inputs and model outputs, agent backdoor attacks exhibit more diverse and covert forms: (1) From the perspective of the final attacking outcomes, the agent backdoor attacker can not only choose to manipulate the final output distribution, but also introduce the malicious behavior in an intermediate reasoning step only, while keeping the final output correct. (2) Furthermore, the former category can be divided into two subcategories based on trigger locations, in which the backdoor trigger can either be hidden in the user query or appear in an intermediate observation returned by the external environment. We implement the above variations of agent backdoor attacks on two typical agent tasks including web shopping and tool utilization. Extensive experiments show that LLM-based agents suffer severely from backdoor attacks and such backdoor vulnerability cannot be easily mitigated by current textual backdoor defense algorithms. This indicates an urgent need for further research on the development of targeted defenses against backdoor attacks on LLM-based agents. Warning: This paper may contain biased content.