Reinforcement learning-assisted quantum architecture search for variational quantum algorithms
作者: Akash Kundu
分类: quant-ph, cs.AI, cs.LG
发布日期: 2024-02-21 (更新: 2024-10-01)
备注: With many pages, figures and tables, I, Akash Kundu upload the final version of my thesis! Including reviewers response and a kind of brief overview of recent quantum architecture search methods
💡 一句话要点
提出基于强化学习的量子架构搜索以优化变分量子算法
🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)
关键词: 量子计算 变分量子算法 强化学习 量子架构搜索 噪声优化
📋 核心要点
- 现有的量子架构搜索主要集中在无噪声场景，噪声对搜索过程的影响尚未得到充分研究。
- 本文提出通过强化学习自动化搜索变分电路的最佳结构，利用张量编码和环境动态限制来提高搜索效率。
- 实验结果显示，所提方法在无噪声和有噪声的量子硬件上均优于现有的量子架构搜索方法。
📝 摘要（中文）
在噪声中间规模量子（NISQ）时代，识别功能性量子电路是一个重大挑战，这些电路必须符合当前量子硬件的限制。变分量子算法（VQAs）旨在解决这些问题，但其性能依赖于变分电路的初始化策略、结构和成本函数配置。本文通过强化学习自动搜索变分电路的最佳结构，提升VQAs的性能。我们提出了一种基于张量的量子电路编码，限制环境动态以高效探索电路搜索空间，并引入双深度Q网络（DDQN）以提高稳定性。实验结果表明，基于RL的量子架构搜索在处理各种VQAs时优于现有方法。
🔬 方法详解
问题定义：本文旨在解决在噪声中间规模量子（NISQ）时代，如何有效识别和优化变分量子算法（VQAs）中的量子电路结构的问题。现有方法多集中于无噪声场景，未能充分考虑噪声对电路性能的影响。
核心思路：通过强化学习（RL）自动化搜索变分电路的最佳结构，评估电路的深度、门和参数的总数以及解决特定问题的准确性，从而提升VQAs的整体性能。
技术框架：整体架构包括量子电路的张量编码、环境动态限制、短电路引导的回合停止机制，以及双深度Q网络（DDQN）与ε-贪婪策略的结合，以提高搜索的稳定性和效率。
关键创新：最重要的创新在于引入了张量编码和环境动态限制，使得在噪声环境下的电路搜索更加高效，并且通过DDQN增强了模型的稳定性。这些创新与传统的无噪声搜索方法有本质区别。
关键设计：在技术细节上，采用了双深度Q网络架构，设置了适当的损失函数以优化电路性能，并通过ε-贪婪策略平衡探索与利用，确保搜索过程的有效性。
🖼️ 关键图片
📊 实验亮点
实验结果表明，所提出的基于强化学习的量子架构搜索方法在处理多种变分量子算法时，性能优于现有的量子架构搜索方法，尤其在噪声环境下表现出显著的提升，具体性能数据尚未披露。
🎯 应用场景
该研究的潜在应用领域包括量子计算、量子优化和量子机器学习等。通过优化变分量子算法的电路结构，可以提高量子计算的效率和准确性，推动量子技术在实际应用中的发展，具有重要的实际价值和未来影响。
📄 摘要（原文）
A significant hurdle in the noisy intermediate-scale quantum (NISQ) era is identifying functional quantum circuits. These circuits must also adhere to the constraints imposed by current quantum hardware limitations. Variational quantum algorithms (VQAs), a class of quantum-classical optimization algorithms, were developed to address these challenges in the currently available quantum devices. However, the overall performance of VQAs depends on the initialization strategy of the variational circuit, the structure of the circuit (also known as ansatz), and the configuration of the cost function. Focusing on the structure of the circuit, in this thesis, we improve the performance of VQAs by automating the search for an optimal structure for the variational circuits using reinforcement learning (RL). Within the thesis, the optimality of a circuit is determined by evaluating its depth, the overall count of gates and parameters, and its accuracy in solving the given problem. The task of automating the search for optimal quantum circuits is known as quantum architecture search (QAS). The majority of research in QAS is primarily focused on a noiseless scenario. Yet, the impact of noise on the QAS remains inadequately explored. In this thesis, we tackle the issue by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network (DDQN) with an $ε$-greedy policy for better stability. The numerical experiments on noiseless and noisy quantum hardware show that in dealing with various VQAs, our RL-based QAS outperforms existing QAS. Meanwhile, the methods we propose in the thesis can be readily adapted to address a wide range of other VQAs.