VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware

📄 arXiv: 2610.34717v1 📥 PDF

作者: Borui Xu, Hongshi Tan, Wenqi Jiang, Bingsheng He, Ming Wu, Jason Zeng, Michael Heinrich

分类: cs.RO, cs.AI

发布日期: 2026-10-26

💡 一句话要点

提出VLA-HWBench以优化边缘设备上的VLA模型推理延迟

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 视觉-语言-动作 边缘计算 推理优化 内核融合 实时控制 机器人技术 多模态学习

📋 核心要点

  1. 现有的VLA模型在推理延迟方面存在显著挑战，影响了机器人控制的实时性。
  2. 本文通过基准测试和七阶段推理分析，提出了优化VLA模型推理的策略，强调动作生成策略的重要性。
  3. 实验结果表明，优化后的模型在NVIDIA Jetson和AMD平台上实现了显著的性能提升，延迟差异可达157倍。

📝 摘要（中文）

随着视觉-语言-动作（VLA）模型在边缘设备上的应用日益增多，推理延迟成为机器人闭环控制的主要限制因素。然而，如何设计VLA模型、推理软件和边缘硬件以实现实时推理仍不明确。本文对18个开放的VLA模型进行了基准测试，揭示了动作生成策略的重要性、内核融合的优势以及内存带宽的限制，提出了优化低延迟边缘VLA的建议。

🔬 方法详解

问题定义：本文旨在解决VLA模型在边缘设备上推理延迟过高的问题。现有方法多集中于任务成功率，缺乏对推理过程的深入分析与优化。

核心思路：通过对18个VLA模型进行基准测试，分析推理延迟的影响因素，提出优化策略，尤其关注动作生成策略和内核融合技术。

技术框架：研究采用七阶段推理流程，涵盖动作生成、内核执行和后处理等模块，重点分析每个阶段的延迟来源。

关键创新：本文的创新在于揭示了动作生成策略对推理延迟的主导作用，并提出了内核融合与低位权重的结合优化，显著提升了推理效率。

关键设计：在实验中，采用4位权重的内核融合技术，优化了GR00T N1.7模型的推理速度，且在Jetson平台上实现了2.6-2.9倍的加速。

📊 实验亮点

实验结果显示，优化后的模型在NVIDIA Jetson平台上，推理延迟的变化范围达到82倍，而在AMD平台上则达到157倍，表明动作生成策略和内核融合技术的有效性。

🎯 应用场景

该研究的潜在应用领域包括机器人控制、智能监控和自动驾驶等场景，能够显著提升边缘设备上VLA模型的实时推理能力，推动相关技术的实际应用与发展。

📄 摘要（原文）

Vision--language--action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. Most VLA models report only task success, while prior studies cover limited action-generation strategies, divide inference into at most three stages, and lack comparisons of inference optimizations on the same models.benchmarking 18 open VLA models across released and alternative backends on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown. Our study reveals three insights. First, action-generation strategy matters more than model size. It dominates 14 of 19 Jetson configurations, latency varies by $82\times$ on Jetson and $157\times$ on AMD, and parameter count explains at most 15% of latency variance. Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. Second, kernel fusion with 4-bit weights helps most, running $π_{0.5}$ and GR00T N1.7 2.6--2.9$\times$ faster than PyTorch with \texttt{torch.compile}, but it exists for only these two models and only on the Jetson. Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12\% to 39\% of the GR00T N1.7 call. Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.