The rapid development of quantum computing (QC) in the NISQ era urgently demands a low-level benchmark suite and insightful evaluation metrics for characterizing the properties of prototype NISQ devices, the efficiency of QC programming compilers, schedulers and assemblers, and the capability of quantum system simulators in a classical computer. In this work, we fill this gap by proposing a low-level, easy-to-use benchmark suite called QASMBench based on the OpenQASM assembly representation. It consolidates commonly used quantum routines and kernels from a variety of domains including chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, cryptography, and so on, trading-off between generality and usability. To analyze these kernels in terms of NISQ device execution, in addition to circuit width and depth, we propose four circuit metrics including gate density, retention lifespan, measurement density, and entanglement variance, to extract more insights about the execution efficiency, the susceptibility to NISQ error, and the potential gain from machine-specific optimizations. Applications in QASMBench can be launched and verified on several NISQ platforms, including IBM-Q, Rigetti, IonQ and Quantinuum. For evaluation, we measure the execution fidelity of a subset of QASMBench applications on 12 IBM-Q machines through density matrix state tomography, comprising 25K circuit evaluations. In addition we also compare the fidelity of executions among the IBM-Q machines, the IonQ QPU and the Rigetti Aspen M-1 system.
Li, Ang, et al. "QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation." ACM Transactions on Quantum Computing, vol. 4, no. 2, Feb. 2023. https://doi.org/10.1145/3550488
Li, Ang, Stein, Samuel, Krishnamoorthy, Sriram, & Ang, James (2023). QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation. ACM Transactions on Quantum Computing, 4(2). https://doi.org/10.1145/3550488
Li, Ang, Stein, Samuel, Krishnamoorthy, Sriram, et al., "QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation," ACM Transactions on Quantum Computing 4, no. 2 (2023), https://doi.org/10.1145/3550488
@article{osti_1969005,
author = {Li, Ang and Stein, Samuel and Krishnamoorthy, Sriram and Ang, James},
title = {QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation},
annote = {The rapid development of quantum computing (QC) in the NISQ era urgently demands a low-level benchmark suite and insightful evaluation metrics for characterizing the properties of prototype NISQ devices, the efficiency of QC programming compilers, schedulers and assemblers, and the capability of quantum system simulators in a classical computer. In this work, we fill this gap by proposing a low-level, easy-to-use benchmark suite called QASMBench based on the OpenQASM assembly representation. It consolidates commonly used quantum routines and kernels from a variety of domains including chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, cryptography, and so on, trading-off between generality and usability. To analyze these kernels in terms of NISQ device execution, in addition to circuit width and depth, we propose four circuit metrics including gate density, retention lifespan, measurement density, and entanglement variance, to extract more insights about the execution efficiency, the susceptibility to NISQ error, and the potential gain from machine-specific optimizations. Applications in QASMBench can be launched and verified on several NISQ platforms, including IBM-Q, Rigetti, IonQ and Quantinuum. For evaluation, we measure the execution fidelity of a subset of QASMBench applications on 12 IBM-Q machines through density matrix state tomography, comprising 25K circuit evaluations. In addition we also compare the fidelity of executions among the IBM-Q machines, the IonQ QPU and the Rigetti Aspen M-1 system.},
doi = {10.1145/3550488},
url = {https://www.osti.gov/biblio/1969005},
journal = {ACM Transactions on Quantum Computing},
issn = {ISSN 2643-6809},
number = {2},
volume = {4},
place = {United States},
publisher = {Association for Computing Machinery},
year = {2023},
month = {02}}
MICRO '52: The 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecturehttps://doi.org/10.1145/3352460.3358265
Murali, Prakash; Baker, Jonathan M.; Javadi-Abhari, Ali
ASPLOS '19: Architectural Support for Programming Languages and Operating Systems, Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systemshttps://doi.org/10.1145/3297858.3304075
Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systemshttps://doi.org/10.1145/3173162.3177152
MICRO '52: The 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecturehttps://doi.org/10.1145/3352460.3358257
Murali, Prakash; Linke, Norbert Matthias; Martonosi, Margaret
ISCA '19: The 46th Annual International Symposium on Computer Architecture, Proceedings of the 46th International Symposium on Computer Architecturehttps://doi.org/10.1145/3307650.3322273
ASPLOS '19: Architectural Support for Programming Languages and Operating Systems, Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systemshttps://doi.org/10.1145/3297858.3304007