Curriculum reinforcement learning for quantum architecture search under hardware errors

📄 arXiv: 2402.03500v1 📥 PDF

作者: Yash J. Patel, Akash Kundu, Mateusz Ostaszewski, Xavier Bonet-Monroig, Vedran Dunjko, Onur Danaci

分类: quant-ph, cs.AI, cs.LG

发布日期: 2024-02-05

备注: 32 pages, 11 figures, 6 tables. Accepted at ICLR 2024

💡 一句话要点

提出课程强化学习算法以解决量子架构搜索中的噪声问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 量子计算 架构搜索 强化学习 变分量子算法 噪声优化 量子化学 电路设计

📋 核心要点

  1. 当前量子计算面临的主要挑战是如何在噪声环境中找到有效的电路架构，现有方法在这方面的理解不足。
  2. 本文提出的CRLQAS算法通过课程强化学习，结合3D架构编码和环境动态限制，优化电路搜索过程。
  3. 实验结果显示，CRLQAS在量子化学任务中表现优异，超越了多种现有QAS算法，提升了优化效率。

📝 摘要（中文）

在噪声中间规模量子时代，寻找与当前设备限制兼容的有用电路是一个关键挑战。变分量子算法（VQAs）通过固定电路架构并优化单个门参数来提供解决方案。然而，参数优化可能变得不可行，且算法的整体性能高度依赖于初始选择的电路架构。现有的量子架构搜索（QAS）算法在自动设计有用电路架构方面取得了一定进展，但噪声对架构搜索的影响尚不清楚。本文提出了一种基于课程强化学习的QAS算法（CRLQAS），旨在应对现实VQA部署中的挑战。该算法通过3D架构编码、环境动态限制、回合停止机制和新型优化器实现了高效的电路搜索。数值实验表明，CRLQAS在无噪声和有噪声环境中均优于现有QAS算法。

🔬 方法详解

问题定义：本文旨在解决在噪声环境下进行量子架构搜索时的有效电路设计问题。现有方法对噪声影响的理解不足，导致优化性能不佳。

核心思路：CRLQAS算法通过课程强化学习框架，逐步引导智能体探索电路架构，优化搜索过程，旨在提高电路设计的有效性和效率。

技术框架：该算法包括三个主要模块：1) 3D架构编码，便于高效探索电路空间；2) 回合停止机制，帮助智能体找到更短的电路；3) 新型优化器，基于同时扰动随机逼近法，实现更快的收敛。

关键创新：CRLQAS的创新在于结合课程学习与强化学习，系统性地解决了噪声对架构搜索的影响，显著提高了电路设计的有效性。

关键设计：算法中采用了环境动态限制和回合停止机制，优化了电路搜索过程，并使用了保利转移矩阵形式在保利-刘维尔基底中进行高效模拟。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，CRLQAS在多个量子化学任务中表现优异，相较于现有QAS算法，在无噪声和有噪声环境中均实现了显著的性能提升，具体提升幅度未知。

🎯 应用场景

该研究的潜在应用领域包括量子计算、量子化学模拟和量子算法优化。通过提高量子电路设计的效率，CRLQAS有望推动量子计算技术的实际应用，尤其是在处理复杂量子系统时，具有重要的实际价值和未来影响。

📄 摘要（原文）

The key challenge in the noisy intermediate-scale quantum era is finding useful circuits compatible with current device limitations. Variational quantum algorithms (VQAs) offer a potential solution by fixing the circuit architecture and optimizing individual gate parameters in an external loop. However, parameter optimization can become intractable, and the overall performance of the algorithm depends heavily on the initially chosen circuit architecture. Several quantum architecture search (QAS) algorithms have been developed to design useful circuit architectures automatically. In the case of parameter optimization alone, noise effects have been observed to dramatically influence the performance of the optimizer and final outcomes, which is a key line of study. However, the effects of noise on the architecture search, which could be just as critical, are poorly understood. This work addresses this gap by introducing a curriculum-based reinforcement learning QAS (CRLQAS) algorithm designed to tackle challenges in realistic VQA deployment. The algorithm incorporates (i) a 3D architecture encoding and restrictions on environment dynamics to explore the search space of possible circuits efficiently, (ii) an episode halting scheme to steer the agent to find shorter circuits, and (iii) a novel variant of simultaneous perturbation stochastic approximation as an optimizer for faster convergence. To facilitate studies, we developed an optimized simulator for our algorithm, significantly improving computational efficiency in simulating noisy quantum circuits by employing the Pauli-transfer matrix formalism in the Pauli-Liouville basis. Numerical experiments focusing on quantum chemistry tasks demonstrate that CRLQAS outperforms existing QAS algorithms across several metrics in both noiseless and noisy environments.