Repair or Resample? Rethinking Failure Debugging in LLM Multi-Agent Systems

📄 arXiv: 2608.25920v1 📥 PDF

作者: Zhongwen Luan, Xiaoyu Zhang, Ming Hu, Yue Yang, Jiongchi Yu, Xiaohong Chen

分类: cs.AI, cs.SE

发布日期: 2026-08-26

💡 一句话要点

提出SymTrace框架以解决LLM多智能体系统调试问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多智能体系统 故障调试 大型语言模型 SymTrace框架 症状驱动干预 执行轨迹 数据集构建 可靠性提升

📋 核心要点

  1. 现有的多智能体系统调试方法依赖于重跑和重采样，导致故障再现和修复率低，无法有效解决问题。
  2. 本文提出SymTrace框架，通过记录执行轨迹和建立干预锚点，重建故障前的执行状态，从而提高故障再现的可靠性。
  3. 实验结果表明，现有无指导重跑方法的故障再现率仅为67.97%，而基于症状驱动的干预方法成功修复了20.15%的失败案例，显著提升了修复效果。

📝 摘要（中文）

随着基于大型语言模型（LLM）的多智能体系统（MAS）在复杂任务中的应用日益增加，其可靠性已成为阻碍实际部署的核心瓶颈。现有的MAS调试和修复方法通常依赖于重新运行和重新采样整个执行轨迹。然而，究竟这些方法是因果修复MAS故障，还是仅仅通过利用LLM采样的随机性进行随机修复，仍然是一个未解之谜。为评估MAS修复方法的有效性，本文引入了SymTrace，一个记录MAS执行轨迹并建立干预锚点的控制评估框架。通过重放，它有效地重建锚点之前的执行，并仅重新生成下游轨迹，从而可靠地再现MAS故障。我们还构建了SymFail数据集，包含536个人工标注的故障轨迹，具有图链接位置、类别和追踪证据。基于这些基础，我们在三个主流MAS框架上进行了大规模实证研究，发现现有的无指导重跑方法可靠性极低，故障再现和修复率分别仅为67.97%和6.90%。基于这些发现，我们进一步探索了一种症状驱动的干预方法，成功修复了20.15%的失败案例，较现有修复方法提升了191.89%。

🔬 方法详解

问题定义：本文旨在解决现有多智能体系统调试方法的低可靠性问题，特别是重跑和重采样方法在故障再现和修复率上的不足。

核心思路：通过引入SymTrace框架，记录MAS的执行轨迹并建立干预锚点，重建故障前的状态，仅重新生成下游轨迹，从而提高故障再现的可靠性。

技术框架：SymTrace框架包括记录执行轨迹、建立干预锚点、重放执行和重生成下游轨迹四个主要模块。记录模块负责捕捉执行过程，干预模块用于设定干预点，重放模块重建执行状态，重生成模块则负责生成后续轨迹。

关键创新：最重要的创新在于SymTrace框架的设计，使得故障再现不再依赖于全局重跑，而是通过局部重建和干预实现，显著提高了修复的有效性。

关键设计：在SymTrace中，记录的执行轨迹包括状态、动作和环境反馈，干预锚点的选择基于故障发生的关键时刻，重生成的过程采用了基于症状的干预策略，以提高修复率。

🖼️ 关键图片

img_0
img_1
img_2

📊 实验亮点

实验结果显示，现有无指导重跑方法的故障再现率仅为67.97%，修复率仅为6.90%。而基于症状驱动的干预方法成功修复了20.15%的失败案例，较现有修复方法提升了191.89%，显示出显著的性能改进。

🎯 应用场景

该研究的潜在应用领域包括智能机器人、自动驾驶、智能客服等多智能体系统的开发与调试。通过提高故障调试的可靠性，能够加速这些系统的实际部署，提升其在复杂任务中的表现，具有重要的实际价值和未来影响。

📄 摘要（原文）

As large language model (LLM)-based multi-agent systems (MASs) are increasingly applied to long-horizon complex tasks, their reliability has emerged as the core bottleneck hindering their real-world deployment. Existing MAS debugging and repair methods typically rely on rerunning and resampling the entire execution trajectory. However, a fundamental question remains to be answered: do these methods causally repair MAS failures or merely stochastically repair by leveraging the randomness of LLM sampling? To evaluate the effectiveness of MAS repair methods, we introduce SymTrace, a controlled evaluation framework that records the MAS execution trajectory and establishes intervention anchors. During replay, it effectively reconstructs the execution before the anchor using recorded logs and only regenerates the downstream trajectory, thereby enabling the reliable reproduction of MAS failures. We further construct the dataset SymFail, comprising 536 human-annotated failure trajectories with graph-linked locations, categories, and trace evidence. Based on these foundations, we conduct a large-scale empirical study across three mainstream MAS frameworks. Our findings reveal that existing unguided rerun methods are highly unreliable, exhibiting low failure reproduction and repair rates (only 67.97% and 6.90%, respectively). Building upon these findings, we further explore the effectiveness of a symptom-driven intervention method, which successfully repairs 20.15% of the failed cases (a 191.89% improvement to state-of-the-art repair methods). This study aims to provide actionable insights for MAS debugging and repair research, paving the way for the robust deployment of multi-agent systems.