Strictly Causal Streaming Video Anomaly Detection with a Theoretically-Grounded State-Space Core

📄 arXiv: 2608.24810v1 📥 PDF

作者: Yogesh Kumar

分类: cs.AI

发布日期: 2026-08-25

💡 一句话要点

提出严格因果流媒体视频异常检测方法以解决现有技术不足

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 视频异常检测 因果模型 流媒体处理 状态空间模型 自监督学习

📋 核心要点

  1. 现有视频异常检测方法依赖于缓冲片段，缺乏理论支持，导致检测延迟和效率问题。
  2. 本文提出了一种严格因果的流媒体异常检测器，采用O(1)时间更新状态，避免了片段缓冲。
  3. 实验结果显示，该方法在Apple M3 Pro硬件上实现了每帧0.74毫秒的延迟，达到1300 FPS，且在Ped2和Avenue数据集上分别获得67.9%和70.2%的帧级AUC。

📝 摘要（中文）

近期研究将Mamba风格的状态空间模型应用于视频异常检测，但现有方法仍依赖于内部缓冲片段或窗口，缺乏理论支持以解释时间记忆与检测延迟的关系，并且效率仅通过GPU吞吐量进行基准测试。本文提出了一种严格因果的流媒体异常检测器，其固定大小的状态在每帧输入时以O(1)的时间和内存更新，无需前瞻和片段缓冲。其时间核心是一个对角线性状态空间递归，具有输入和状态依赖的衰减门，通过因果下一个嵌入预测在冻结的视觉骨干上进行自监督训练。我们推导了递归衰减谱与检测延迟及其可靠捕获的最短异常之间的闭合关系，并在UCSD Ped2和CUHK Avenue上进行了实证验证。

🔬 方法详解

问题定义：本文旨在解决现有视频异常检测方法在检测延迟和效率上的不足，尤其是对流媒体场景的适应性问题。现有方法通常依赖于缓冲片段，导致实时性差和理论支持不足。

核心思路：提出了一种严格因果的流媒体异常检测器，设计上避免了片段缓冲和前瞻，确保每帧以O(1)时间更新状态，从而提高实时检测能力。

技术框架：该方法的核心是一个对角线性状态空间递归，结合输入和状态依赖的衰减门。通过因果下一个嵌入预测进行自监督训练，整体架构包括状态更新、衰减门控制和异常检测模块。

关键创新：最重要的创新在于提出了一个无缓冲的因果检测框架，利用递归衰减谱与检测延迟之间的闭合关系，显著提高了响应速度和实时性。

关键设计：设计中包括状态大小、衰减率和门控机制的调节，损失函数采用自监督学习方式，确保模型在不同数据集上的适应性。

🖼️ 关键图片

img_0
img_1
img_2

📊 实验亮点

实验结果表明，该方法在Apple M3 Pro硬件上实现了每帧0.74毫秒的延迟，超过1300 FPS的处理速度。尽管在Ped2和Avenue数据集上的帧级AUC分别为67.9%和70.2%，但相较于先前的非因果SSM基线，准确性有所下降，显示出在小数据集上的门控机制影响。

🎯 应用场景

该研究的潜在应用领域包括智能监控、交通监测和安全防范等场景，能够实时检测异常事件，提升系统的响应能力和效率。未来，该方法可扩展到更大规模的数据集和复杂场景，进一步提高检测准确性和实用性。

📄 摘要（原文）

Recent work has applied Mamba style state space models (SSMs) to video anomaly detection, yet existing approaches still rely on buffering clips or windows internally, lack a theoretical account of how temporal memory relates to detection latency, and benchmark efficiency only through GPU throughput rather than the edge hardware these methods are intended to target. We introduce a strictly causal streaming anomaly detector whose fixed size state is updated in O(1) time and memory per incoming frame, with no lookahead and no clip buffering. Its temporal core is a diagonal linear state space recurrence with an input and state dependent decay gate, trained self supervised through causal next embedding prediction on a frozen visual backbone. We derive a closed form relationship between the recurrence decay spectrum and both detection delay and the shortest anomaly it can reliably capture, then validate empirically on UCSD Ped2 and CUHK Avenue. The settling delay bound predicted from the learned base decay (57 to 59 frames) sits far above the measured detection delay (1.6 and 18.4 frames), showing that the event boundary gate, not the base decay, governs responsiveness. We further report end to end latency and throughput measured directly on Apple M3 Pro hardware, 0.74 ms and 0.77 ms per frame (over 1300 FPS), rather than simulated GPU numbers. With an untuned initial configuration the method reaches 67.9 percent and 70.2 percent frame level AUC on Ped2 and Avenue, trailing prior non causal SSM baselines in accuracy. Ablations over decay rate, state size, and gating reveal that the gate contribution is dataset size dependent, hurting accuracy on the smaller Ped2 training set but helping on the larger Avenue one. Closing this accuracy gap and extending evaluation to a third, larger benchmark are immediate next steps.