Detecting Malicious Agent Skills in the Wild using Attention
作者: Bacem Etteib, Daniele Lunghi, Tégawendé F. Bissyandé
分类: cs.CR, cs.AI
发布日期: 2026-06-22
💡 一句话要点
提出Locate-and-Judge以检测恶意代理技能
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 恶意技能检测 大型语言模型 安全性检测 技能市场 指令跟随注意力
📋 核心要点
- 现有的提示注入防御方法无法有效应对恶意技能，因为技能本身是指令集合，注入的恶意命令难以被识别。
- 本文提出Locate-and-Judge，通过两阶段检测器集中评估高关注度的指令跨度，从而提高检测效率。
- 实验结果显示，该方法在市场规模上以高精度识别恶意技能，且成本显著低于传统的LLM扫描方法。
📝 摘要（中文）
随着大型语言模型（LLM）代理越来越多地加载由第三方编写的技能，这些技能以文件包的形式分发并在用户权限下执行，恶意技能的风险显著增加。单个恶意技能可能导致数据外泄、代理劫持或作为供应链的潜在威胁，使技能市场成为代理系统的新攻击面。现有的提示注入防御方法在此场景中失效，因为技能本身就是一组指令，注入的命令与许多合法指令混合在一起，继承了它们的权限。为此，本文提出了Locate-and-Judge，一个设计用于此环境的两阶段检测器。该检测器通过关注每个结构跨度的指令跟随注意力来评分，仅保留前K个跨度，随后由评判模块详细检查这些保留的跨度。与直接的LLM扫描相比，该方法在成本上减少了一个数量级，显著提高了可扩展性，同时对召回率的影响较小。经过市场规模的部署，Locate-and-Judge以高精度标记技能，其中大多数被手动确认为恶意，揭示了数十个活跃的恶意技能。
🔬 方法详解
问题定义：本文旨在解决恶意技能在代理系统中的检测问题。现有方法无法有效区分恶意与合法指令，因为恶意指令与合法指令混合在一起，导致检测困难。
核心思路：论文的核心思路是通过两阶段检测器Locate-and-Judge，首先定位高关注度的指令跨度，然后对这些跨度进行详细评估，从而提高检测的效率和准确性。
技术框架：整体架构分为两个主要模块：第一阶段是轻量级定位器，负责评分技能的结构跨度并保留前K个；第二阶段是评判模块，详细检查保留的跨度。
关键创新：最重要的技术创新在于通过关注度评分集中评估高风险指令跨度，避免了对整个技能的全面扫描，从而显著降低了计算成本。
关键设计：在设计中，定位器使用指令跟随注意力评分，评判模块则专注于高关注度的指令跨度。具体参数设置和网络结构细节在论文中进行了详细描述。
🖼️ 关键图片
📊 实验亮点
实验结果表明，Locate-and-Judge在市场规模上以高精度标记恶意技能，成功识别出数十个活跃的恶意技能，且大多数被手动确认为恶意。与传统方法相比，该方法在成本上减少了一个数量级，显著提高了检测的可扩展性。
🎯 应用场景
该研究的潜在应用领域包括安全性检测、技能市场监控和恶意软件防护等。通过有效识别恶意技能，能够保护用户数据安全，提升代理系统的整体安全性，具有重要的实际价值和未来影响。
📄 摘要（原文）
LLM agents increasingly load skills, file-based packages of natural-language instructions written by third parties and distributed through marketplaces, that execute with the user's privileges. A single malicious skill can exfiltrate data, hijack the agent, or persist as a supply-chain foothold, which turns the skill marketplace into a new attack surface for agentic systems. Prompt-injection defenses do not carry over to this setting. They rely on a boundary between trusted instructions and untrusted data, whereas a skill is itself a body of instructions, so an injected command sits among many legitimate ones and inherits their authority. We present Locate-and-Judge, a two-stage detector designed for this regime. A lightweight locator scores the structural spans of a skill by the instruction-following attention each span draws and retains only the top-K. A judge then examines the retained spans in detail. Concentrating the costly judgment on a few high-attention spans lets the detector audit an entire marketplace instead of a sample. Compared to direct LLM-based scanning, this approach offers an order-of-magnitude cost reduction, dramatically increasing its scalability at a small cost to recall, and it dominates keyword and regex baselines at comparable expense. Deployed at marketplace scale and at negligible cost, Locate-and-Judge flags skills with high precision, the majority of which we manually confirmed as malicious, surfacing dozens of live malicious skills, including several disguised as benign functionality and many that SkillSpector and Cisco Skill Scanner fail to detect. We release the resulting labeled dataset.