ExaTN: Scalable GPU-Accelerated High-Performance Processing of General Tensor Networks at Exascale
Abstract
We present ExaTN (Exascale Tensor Networks), a scalable GPU-accelerated C++ library which can express and process tensor networks on shared- as well as distributed-memory high-performance computing platforms, including those equipped with GPU accelerators. Specifically, ExaTN provides the ability to build, transform, and numerically evaluate tensor networks with arbitrary graph structures and complexity. It also provides algorithmic primitives for the optimization of tensor factors inside a given tensor network in order to find an extremum of a chosen tensor network functional, which is one of the key numerical procedures in quantum many-body theory and quantum-inspired machine learning. Numerical primitives exposed by ExaTN provide the foundation for composing rather complex tensor network algorithms. We enumerate multiple application domains which can benefit from the capabilities of our library, including condensed matter physics, quantum chemistry, quantum circuit simulations, as well as quantum and classical machine learning, for some of which we provide preliminary demonstrations and performance benchmarks just to emphasize a broad utility of our library.
- Authors:
- Publication Date:
- Research Org.:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Basic Energy Sciences (BES); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Laboratory Directed Research and Development (LDRD) Program
- OSTI Identifier:
- 2325498
- Alternate Identifier(s):
- OSTI ID: 1876320
- Grant/Contract Number:
- AC05-00OR22725; ERKCG13/ERKCG23
- Resource Type:
- Published Article
- Journal Name:
- Frontiers in Applied Mathematics and Statistics
- Additional Journal Information:
- Journal Name: Frontiers in Applied Mathematics and Statistics Journal Volume: 8; Journal ID: ISSN 2297-4687
- Publisher:
- Frontiers Media SA
- Country of Publication:
- Switzerland
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; tensor network; quantum many-body theory; quantum computing; quantum circuit; high performance computing; GPU
Citation Formats
Lyakh, Dmitry I., Nguyen, Thien, Claudino, Daniel, Dumitrescu, Eugene, and McCaskey, Alexander J. ExaTN: Scalable GPU-Accelerated High-Performance Processing of General Tensor Networks at Exascale. Switzerland: N. p., 2022.
Web. doi:10.3389/fams.2022.838601.
Lyakh, Dmitry I., Nguyen, Thien, Claudino, Daniel, Dumitrescu, Eugene, & McCaskey, Alexander J. ExaTN: Scalable GPU-Accelerated High-Performance Processing of General Tensor Networks at Exascale. Switzerland. https://doi.org/10.3389/fams.2022.838601
Lyakh, Dmitry I., Nguyen, Thien, Claudino, Daniel, Dumitrescu, Eugene, and McCaskey, Alexander J. Wed .
"ExaTN: Scalable GPU-Accelerated High-Performance Processing of General Tensor Networks at Exascale". Switzerland. https://doi.org/10.3389/fams.2022.838601.
@article{osti_2325498,
title = {ExaTN: Scalable GPU-Accelerated High-Performance Processing of General Tensor Networks at Exascale},
author = {Lyakh, Dmitry I. and Nguyen, Thien and Claudino, Daniel and Dumitrescu, Eugene and McCaskey, Alexander J.},
abstractNote = {We present ExaTN (Exascale Tensor Networks), a scalable GPU-accelerated C++ library which can express and process tensor networks on shared- as well as distributed-memory high-performance computing platforms, including those equipped with GPU accelerators. Specifically, ExaTN provides the ability to build, transform, and numerically evaluate tensor networks with arbitrary graph structures and complexity. It also provides algorithmic primitives for the optimization of tensor factors inside a given tensor network in order to find an extremum of a chosen tensor network functional, which is one of the key numerical procedures in quantum many-body theory and quantum-inspired machine learning. Numerical primitives exposed by ExaTN provide the foundation for composing rather complex tensor network algorithms. We enumerate multiple application domains which can benefit from the capabilities of our library, including condensed matter physics, quantum chemistry, quantum circuit simulations, as well as quantum and classical machine learning, for some of which we provide preliminary demonstrations and performance benchmarks just to emphasize a broad utility of our library.},
doi = {10.3389/fams.2022.838601},
journal = {Frontiers in Applied Mathematics and Statistics},
number = ,
volume = 8,
place = {Switzerland},
year = {Wed Jul 06 00:00:00 EDT 2022},
month = {Wed Jul 06 00:00:00 EDT 2022}
}
https://doi.org/10.3389/fams.2022.838601
Works referenced in this record:
Corner Transfer Matrix Renormalization Group Method
journal, April 1996
- Nishino, Tomotoshi; Okunishi, Kouichi
- Journal of the Physical Society of Japan, Vol. 65, Issue 4
An Ab Initio Exciton Model Including Charge-Transfer Excited States
journal, July 2017
- Li, Xin; Parrish, Robert M.; Liu, Fang
- Journal of Chemical Theory and Computation, Vol. 13, Issue 8
Tensor Network Quantum Virtual Machine for Simulating Quantum Circuits at Exascale
preprint, January 2021
- Nguyen, Thien; Lyakh, Dmitry; Dumitrescu, Eugene
- arXiv
Valence-bond states for quantum computation
journal, December 2004
- Verstraete, F.; Cirac, J. I.
- Physical Review A, Vol. 70, Issue 6
What Limits the Simulation of Quantum Computers?
journal, November 2020
- Zhou, Yiqing; Stoudenmire, E. Miles; Waintal, Xavier
- Physical Review X, Vol. 10, Issue 4
Renormalization and tensor product states in spin chains and lattices
journal, December 2009
- Cirac, J. Ignacio; Verstraete, Frank
- Journal of Physics A: Mathematical and Theoretical, Vol. 42, Issue 50
Class of Quantum Many-Body States That Can Be Efficiently Simulated
journal, September 2008
- Vidal, G.
- Physical Review Letters, Vol. 101, Issue 11
Simulating Quantum Computation by Contracting Tensor Networks
journal, January 2008
- Markov, Igor L.; Shi, Yaoyun
- SIAM Journal on Computing, Vol. 38, Issue 3
Validating quantum-classical programming models with tensor network simulations
journal, December 2018
- McCaskey, Alexander; Dumitrescu, Eugene; Chen, Mengsu
- PLOS ONE, Vol. 13, Issue 12
Efficient tree tensor network states (TTNS) for quantum chemistry: Generalizations of the density matrix renormalization group algorithm
journal, April 2013
- Nakatani, Naoki; Chan, Garnet Kin-Lic
- The Journal of Chemical Physics, Vol. 138, Issue 13
Tensor Completion Algorithms in Big Data Analytics
journal, January 2019
- Song, Qingquan; Ge, Hancheng; Caverlee, James
- ACM Transactions on Knowledge Discovery from Data, Vol. 13, Issue 1
quimb: A python package for quantum information and many-body calculations
journal, September 2018
- Gray, Johnnie
- Journal of Open Source Software, Vol. 3, Issue 29
Efficient Quantum Circuits for Accurate State Preparation of Smooth, Differentiable Functions
conference, October 2020
- Holmes, Adam; Matsuura, A. Y.
- 2020 IEEE International Conference on Quantum Computing and Engineering (QCE)
Hyper-optimized tensor network contraction
journal, March 2021
- Gray, Johnnie; Kourtis, Stefanos
- Quantum, Vol. 5
Tensor Network Renormalization
journal, October 2015
- Evenbly, G.; Vidal, G.
- Physical Review Letters, Vol. 115, Issue 18
Entanglement Renormalization
journal, November 2007
- Vidal, G.
- Physical Review Letters, Vol. 99, Issue 22
cuTT: A High-Performance Tensor Transpose Library for CUDA Compatible GPUs
preprint, January 2017
- Hynninen, Antti-Pekka; Lyakh, Dmitry I.
- arXiv
A practical introduction to tensor networks: Matrix product states and projected entangled pair states
journal, October 2014
- Orús, Román
- Annals of Physics, Vol. 349
Real-Time Evolution Using the Density Matrix Renormalization Group
journal, August 2004
- White, Steven R.; Feiguin, Adrian E.
- Physical Review Letters, Vol. 93, Issue 7
AutoHOOT: Automatic High-Order Optimization for Tensors
conference, September 2020
- Ma, Linjian; Ye, Jiayu; Solomonik, Edgar
- PACT '20: International Conference on Parallel Architectures and Compilation Techniques, Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques
Density matrix formulation for quantum renormalization groups
journal, November 1992
- White, Steven R.
- Physical Review Letters, Vol. 69, Issue 19
Ab Initio Nonadiabatic Dynamics of Multichromophore Complexes: A Scalable Graphical-Processing-Unit-Accelerated Exciton Framework
journal, July 2014
- Sisto, Aaron; Glowacki, David R.; Martinez, Todd J.
- Accounts of Chemical Research, Vol. 47, Issue 9
Simple heuristics for efficient parallel tensor contraction and quantum circuit simulation
journal, December 2020
- Schutski, Roman; Khakhulin, Taras; Oseledets, Ivan
- Physical Review A, Vol. 102, Issue 6
XACC: a system-level software infrastructure for heterogeneous quantum–classical computing
journal, February 2020
- McCaskey, Alexander J.; Lyakh, Dmitry I.; Dumitrescu, Eugene F.
- Quantum Science and Technology, Vol. 5, Issue 2
Classical Simulation of Infinite-Size Quantum Lattice Systems in One Spatial Dimension
journal, February 2007
- Vidal, G.
- Physical Review Letters, Vol. 98, Issue 7
Multi-Tensor Contraction for XEB Verification of Quantum Circuits
preprint, January 2021
- Kalachev, Gleb; Panteleev, Pavel; Yung, Man-Hong
- arXiv
Tensorized Embedding Layers
conference, January 2020
- Hrinchuk, Oleksii; Khrulkov, Valentin; Mirvakhabova, Leyla
- Findings of the Association for Computational Linguistics: EMNLP 2020
A Multi-Scale Tensor Network Architecture for Classification and Regression
preprint, January 2020
- Reyes, Justin; Stoudenmire, Miles
- arXiv
Efficient 2D Tensor Network Simulation of Quantum Systems
conference, November 2020
- Pang, Yuchen; Hao, Tianyi; Dugad, Annika
- SC20: International Conference for High Performance Computing, Networking, Storage and Analysis
The density-matrix renormalization group in the age of matrix product states
journal, January 2011
- Schollwöck, Ulrich
- Annals of Physics, Vol. 326, Issue 1
A flexible high-performance simulator for verifying and benchmarking quantum circuits implemented on real hardware
journal, October 2019
- Villalonga, Benjamin; Boixo, Sergio; Nelson, Bron
- npj Quantum Information, Vol. 5, Issue 1
Generative machine learning with tensor networks: Benchmarks on near-term quantum computers
journal, April 2021
- Wall, Michael L.; Abernathy, Matthew R.; Quiroz, Gregory
- Physical Review Research, Vol. 3, Issue 2
Matrix product operators, matrix product states, and ab initio density matrix renormalization group algorithms
journal, July 2016
- Chan, Garnet Kin-Lic; Keselman, Anna; Nakatani, Naoki
- The Journal of Chemical Physics, Vol. 145, Issue 1
Computational Complexity and Fundamental Limitations to Fermionic Quantum Monte Carlo Simulations
journal, May 2005
- Troyer, Matthias; Wiese, Uwe-Jens
- Physical Review Letters, Vol. 94, Issue 17
Entanglement and Tensor Networks for Supervised Image Classification
preprint, January 2020
- Martyn, John; Vidal, Guifre; Roberts, Chase
- arXiv
The density-matrix renormalization group
journal, April 2005
- Schollwöck, U.
- Reviews of Modern Physics, Vol. 77, Issue 1
Faster identification of optimal contraction sequences for tensor networks
journal, September 2014
- Pfeifer, Robert N. C.; Haegeman, Jutho; Verstraete, Frank
- Physical Review E, Vol. 90, Issue 3
Quantum supremacy using a programmable superconducting processor
journal, October 2019
- Arute, Frank; Arya, Kunal; Babbush, Ryan
- Nature, Vol. 574, Issue 7779
Classical simulation of quantum many-body systems with a tree tensor network
journal, August 2006
- Shi, Y. -Y.; Duan, L. -M.; Vidal, G.
- Physical Review A, Vol. 74, Issue 2
An efficient tensor transpose algorithm for multicore CPU, Intel Xeon Phi, and NVidia Tesla GPU
journal, April 2015
- Lyakh, Dmitry I.
- Computer Physics Communications, Vol. 189
Tensor-Train Decomposition
journal, January 2011
- Oseledets, I. V.
- SIAM Journal on Scientific Computing, Vol. 33, Issue 5
Establishing the quantum supremacy frontier with a 281 Pflop/s simulation
journal, April 2020
- Villalonga, Benjamin; Lyakh, Dmitry; Boixo, Sergio
- Quantum Science and Technology, Vol. 5, Issue 3
NCON: A tensor network contractor for MATLAB
preprint, January 2014
- Pfeifer, Robert N. C.; Evenbly, Glen; Singh, Sukhwinder
- arXiv
Multilevel Algorithms for Multi-Constraint Graph Partitioning
conference, January 1998
- Karypis, G.; Kumar, V.
- SC98 - High Performance Networking and Computing Conference, Proceedings of the IEEE/ACM SC98 Conference
Compressing deep neural networks by matrix product operators
journal, June 2020
- Gao, Ze-Feng; Cheng, Song; He, Rong-Qiang
- Physical Review Research, Vol. 2, Issue 2
Number-State Preserving Tensor Networks as Classifiers for Supervised Learning
preprint, January 2019
- Evenbly, Glen
- arXiv
Advances on tensor network theory: symmetries, fermions, entanglement, and holography
journal, November 2014
- Orús, Román
- The European Physical Journal B, Vol. 87, Issue 11
Efficient Classical Simulation of Slightly Entangled Quantum Computations
journal, October 2003
- Vidal, Guifré
- Physical Review Letters, Vol. 91, Issue 14
The ITensor Software Library for Tensor Network Calculations
preprint, January 2020
- Fishman, Matthew; White, Steven R.; Stoudenmire, E. Miles
- arXiv
Efficient classical simulation of noisy random quantum circuits in one dimension
journal, September 2020
- Noh, Kyungjoo; Jiang, Liang; Fefferman, Bill
- Quantum, Vol. 4