Quantum Perturbation Theory Using Tensor Cores and a Deep Neural Network

Finkelstein, Joshua David; Rubensson, Emanuel H.; Mniszewski, Susan M.; Negre, Christian Francisco Andres; Niklasson, Anders Mauritz N.

doi:10.1021/acs.jctc.2c00274

Title: Quantum Perturbation Theory Using Tensor Cores and a Deep Neural Network

Journal Article · 07 June 2022 · Journal of Chemical Theory and Computation

DOI: https://doi.org/10.1021/acs.jctc.2c00274 · OSTI ID:1883152

^[1];

^[2];

^[1];

^[1]

Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Uppsala Univ. (Sweden)

In this work, time-independent quantum response calculations are performed using Tensor cores. This is achieved by mapping density matrix perturbation theory onto the computational structure of a deep neural network. The main computational cost of each deep layer is dominated by tensor contractions, i.e., dense matrix–matrix multiplications, in mixed-precision arithmetics, which achieves close to peak performance. Quantum response calculations are demonstrated and analyzed using self-consistent charge density-functional tight-binding theory as well as coupled-perturbed Hartree–Fock theory. For linear response calculations, a novel parameter-free convergence criterion is presented that is well-suited for numerically noisy low-precision floating point operations and we demonstrate a peak performance of almost 200 Tflops using the Tensor cores of two Nvidia A100 GPUs.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: 89233218CNA000001

OSTI ID:: 1883152

Report Number(s):: LA-UR-22-21647

Journal Information:: Journal of Chemical Theory and Computation, Journal Name: Journal of Chemical Theory and Computation Journal Issue: 7 Vol. 18; ISSN 1549-9618

Publisher:: American Chemical SocietyCopyright Statement

Country of Publication:: United States

Language:: English

References (67)

A Self-Consistent Charge Density-Functional Based Tight-Binding Method for Predictive Materials Simulations in Physics, Chemistry and Biology Frauenheim, Th.; Seifert, G.; Elsterner, M. physica status solidi (b), Vol. 217, Issue 1 https://doi.org/10.1002/(SICI)1521-3951(200001)217:1<41::AID-PSSB41>3.0.CO;2-V	journal	January 2000
Quantisierung als Eigenwertproblem Schrödinger, E. Annalen der Physik, Vol. 385, Issue 13 https://doi.org/10.1002/andp.19263851302	journal	January 1926
369 Tflop/s molecular dynamics simulations on the petaflop hybrid supercomputer ‘Roadrunner’ Germann, Timothy C.; Kadau, Kai; Swaminarayan, Sriram Concurrency and Computation: Practice and Experience, Vol. 21, Issue 17 https://doi.org/10.1002/cpe.1483	journal	December 2009
Accelerating VASP electronic structure calculations using graphic processing units Hacene, Mohamed; Anciaux-Sedrakian, Ani; Rozanska, Xavier Journal of Computational Chemistry, Vol. 33, Issue 32 https://doi.org/10.1002/jcc.23096	journal	August 2012
Derivative studies in hartree-fock and møller-plesset theories Pople, J. A.; Krishnan, R.; Schlegel, H. B. International Journal of Quantum Chemistry, Vol. 16, Issue S13 https://doi.org/10.1002/qua.560160825	journal	March 1979
Iterative Berechung der reziproken Matrix Schulz, Günther ZAMM - Zeitschrift für Angewandte Mathematik und Mechanik, Vol. 13, Issue 1 https://doi.org/10.1002/zamm.19330130111	journal	January 1933
Density Functional Theory Dreizler, Reiner M.; Gross, Eberhard K. U. Springer Berlin, Heidelberg https://doi.org/10.1007/978-3-642-86105-5	book	January 1990
N�herungsmethode zur L�sung des quantenmechanischen Mehrk�rperproblems Fock, V. Zeitschrift f�r Physik, Vol. 61, Issue 1-2 https://doi.org/10.1007/BF01340294	journal	January 1930
�ber den Grundterm der Zweielektronenprobleme von H?, He, Li+, Be++ usw. Hylleraas, Egil A. Zeitschrift f�r Physik, Vol. 65, Issue 3-4 https://doi.org/10.1007/BF01397032	journal	March 1930
A reformulation of the coupled perturbed self-consistent field equations entirely within a local atomic orbital density matrix-based scheme Ochsenfeld, Christian; Head-Gordon, Martin Chemical Physics Letters, Vol. 270, Issue 5-6 https://doi.org/10.1016/S0009-2614(97)00402-8	journal	May 1997
GPU acceleration of all-electron electronic structure theory using localized numeric atom-centered basis functions Huhn, William P.; Lange, Björn; Yu, Victor Wen-zhe Computer Physics Communications, Vol. 254 https://doi.org/10.1016/j.cpc.2020.107314	journal	September 2020
Efficient parallel linear scaling method to get the response density matrix in all-electron real-space density-functional perturbation theory Shang, Honghui; Liang, WanZhen; Zhang, Yunquan Computer Physics Communications, Vol. 258 https://doi.org/10.1016/j.cpc.2020.107613	journal	January 2021
Many-core acceleration of the first-principles all-electron quantum perturbation calculations Shang, Honghui; Duan, Xiaohui; Li, Fang Computer Physics Communications, Vol. 267 https://doi.org/10.1016/j.cpc.2021.108045	journal	October 2021
GPU-accelerated molecular modeling coming of age Stone, John E.; Hardy, David J.; Ufimtsev, Ivan S. Journal of Molecular Graphics and Modelling, Vol. 29, Issue 2 https://doi.org/10.1016/j.jmgm.2010.06.010	journal	September 2010
Ergo: An open-source program for linear-scaling electronic structure calculations Rudberg, Elias; Rubensson, Emanuel H.; Sałek, Paweł SoftwareX, Vol. 7 https://doi.org/10.1016/j.softx.2018.03.005	journal	January 2018
Editorial: Modern Architectures and Their Impact on Electronic Structure Theory Gordon, Mark S.; Windus, Theresa L. Chemical Reviews, Vol. 120, Issue 17 https://doi.org/10.1021/acs.chemrev.0c00700	journal	September 2020
Graphics Processing Unit-Accelerated Semiempirical Born Oppenheimer Molecular Dynamics Using PyTorch Zhou, Guoqing; Nebgen, Ben; Lubbers, Nicholas Journal of Chemical Theory and Computation, Vol. 16, Issue 8 https://doi.org/10.1021/acs.jctc.0c00243	journal	July 2020
Mixed Precision Fermi-Operator Expansion on Tensor Cores from a Machine Learning Perspective Finkelstein, Joshua; Smith, Justin S.; Mniszewski, Susan M. Journal of Chemical Theory and Computation, Vol. 17, Issue 4 https://doi.org/10.1021/acs.jctc.1c00057	journal	April 2021
Quantum-Based Molecular Dynamics Simulations Using Tensor Cores Finkelstein, Joshua; Smith, Justin S.; Mniszewski, Susan M. Journal of Chemical Theory and Computation, Vol. 17, Issue 10 https://doi.org/10.1021/acs.jctc.1c00726	journal	October 2021
Quantum Chemistry for Solvated Molecules on Graphical Processing Units Using Polarizable Continuum Models Liu, Fang; Luehr, Nathan; Kulik, Heather J. Journal of Chemical Theory and Computation, Vol. 11, Issue 7 https://doi.org/10.1021/acs.jctc.5b00370	journal	June 2015
Recursive Factorization of the Inverse Overlap Matrix in Linear-Scaling Quantum Molecular Dynamics Simulations Negre, Christian F. A.; Mniszewski, Susan M.; Cawkwell, Marc J. Journal of Chemical Theory and Computation, Vol. 12, Issue 7 https://doi.org/10.1021/acs.jctc.6b00154	journal	June 2016
Parameterless Stopping Criteria for Recursive Density Matrix Expansions Kruchinina, Anastasia; Rudberg, Elias; Rubensson, Emanuel H. Journal of Chemical Theory and Computation, Vol. 12, Issue 12 https://doi.org/10.1021/acs.jctc.6b00626	journal	November 2016
Dynamic Precision for Electron Repulsion Integral Evaluation on Graphical Processing Units (GPUs) Luehr, Nathan; Ufimtsev, Ivan S.; Martínez, Todd J. Journal of Chemical Theory and Computation, Vol. 7, Issue 4 https://doi.org/10.1021/ct100701w	journal	March 2011
Nonmonotonic Recursive Polynomial Expansions for Linear Scaling Calculation of the Density Matrix Rubensson, Emanuel H. Journal of Chemical Theory and Computation, Vol. 7, Issue 5 https://doi.org/10.1021/ct2001705	journal	April 2011
GPU Linear Algebra Libraries and GPGPU Programming for Accelerating MOPAC Semiempirical Quantum Chemistry Calculations Maia, Julio Daniel Carvalho; Urquiza Carvalho, Gabriel Aires; Mangueira, Carlos Peixoto Journal of Chemical Theory and Computation, Vol. 8, Issue 9 https://doi.org/10.1021/ct3004645	journal	August 2012
Quantum Chemistry on Graphical Processing Units. 1. Strategies for Two-Electron Integral Evaluation Ufimtsev, Ivan S.; Martínez, Todd J. Journal of Chemical Theory and Computation, Vol. 4, Issue 2 https://doi.org/10.1021/ct700268q	journal	January 2008
Quantum Chemistry on Graphical Processing Units. 2. Direct Self-Consistent-Field Implementation Ufimtsev, Ivan S.; Martinez, Todd J. Journal of Chemical Theory and Computation, Vol. 5, Issue 4 https://doi.org/10.1021/ct800526s	journal	March 2009
Quantum Chemistry on Graphical Processing Units. 3. Analytical Energy Gradients, Geometry Optimization, and First Principles Molecular Dynamics Ufimtsev, Ivan S.; Martinez, Todd J. Journal of Chemical Theory and Computation, Vol. 5, Issue 10 https://doi.org/10.1021/ct9003004	journal	August 2009
Perturbed Hartree—Fock Calculations. I. Magnetic Susceptibility and Shielding in the LiH Molecule Stevens, R. M.; Pitzer, R. M.; Lipscomb, W. N. The Journal of Chemical Physics, Vol. 38, Issue 2 https://doi.org/10.1063/1.1733693	journal	January 1963
Higher-order response in O(N) by perturbed projection Weber, Valéry; Niklasson, Anders M. N.; Challacombe, Matt The Journal of Chemical Physics, Vol. 123, Issue 4 https://doi.org/10.1063/1.1944724	journal	July 2005
Nonorthogonal density-matrix perturbation theory Niklasson, Anders M. N.; Weber, Valéry; Challacombe, Matt The Journal of Chemical Physics, Vol. 123, Issue 4 https://doi.org/10.1063/1.1944725	journal	July 2005
Linear scaling density matrix perturbation theory for basis-set-dependent quantum response calculations: An orthogonal formulation Niklasson, Anders M. N.; Weber, Valéry The Journal of Chemical Physics, Vol. 127, Issue 6 https://doi.org/10.1063/1.2755775	journal	August 2007
Recursive inverse factorization Rubensson, Emanuel H.; Bock, Nicolas; Holmström, Erik The Journal of Chemical Physics, Vol. 128, Issue 10 https://doi.org/10.1063/1.2884921	journal	March 2008
Density functional theory calculation on many-cores hybrid central processing unit-graphic processing unit architectures Genovese, Luigi; Ospici, Matthieu; Deutsch, Thierry The Journal of Chemical Physics, Vol. 131, Issue 3 https://doi.org/10.1063/1.3166140	journal	July 2009
DFTB+, a software package for efficient approximate density functional theory based atomistic simulations Hourahine, B.; Aradi, B.; Blum, V. The Journal of Chemical Physics, Vol. 152, Issue 12 https://doi.org/10.1063/1.5143190	journal	March 2020
Notes on density matrix perturbation theory Truflandier, Lionel A.; Dianzinga, Rivo M.; Bowler, David R. The Journal of Chemical Physics, Vol. 153, Issue 16 https://doi.org/10.1063/5.0022244	journal	October 2020
Linear scaling electronic structure calculations with numerical atomic basis set Shang, Honghui; Xiang, Hongjun; Li, Zhenyu International Reviews in Physical Chemistry, Vol. 29, Issue 4 https://doi.org/10.1080/0144235X.2010.520454	journal	September 2010
\mathcal{O}(N) methods in electronic structure calculations Bowler, D. R.; Miyazaki, T. Reports on Progress in Physics, Vol. 75, Issue 3 https://doi.org/10.1088/0034-4885/75/3/036503	journal	February 2012
Localized inverse factorization Rubensson, Emanuel H.; Artemov, Anton G.; Kruchinina, Anastasia IMA Journal of Numerical Analysis, Vol. 41, Issue 1 https://doi.org/10.1093/imanum/drz075	journal	April 2020
The density matrix in self-consistent field theory I. Iterative construction of the density matrix McWeeny, R. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, Vol. 235, Issue 1203, p. 496-509 https://doi.org/10.1098/rspa.1956.0100	journal	June 1956
Mixed-precision iterative refinement using tensor cores on GPUs to accelerate solution of linear systems Haidar, Azzam; Bayraktar, Harun; Tomov, Stanimire Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 476, Issue 2243 https://doi.org/10.1098/rspa.2020.0110	journal	November 2020
Simulation of Quantum Many-Body Dynamics with Tensor Processing Units: Floquet Prethermalization Morningstar, Alan; Hauru, Markus; Beall, Jackson PRX Quantum, Vol. 3, Issue 2 https://doi.org/10.1103/PRXQuantum.3.020331	journal	May 2022
Perturbation Theory for the Fock-Dirac Density Matrix McWeeny, R. Physical Review, Vol. 126, Issue 3 https://doi.org/10.1103/PhysRev.126.1028	journal	May 1962
Electronic Polarizabilities of Ions from the Hartree-Fock Wave Functions Sternheimer, R. M. Physical Review, Vol. 96, Issue 4 https://doi.org/10.1103/PhysRev.96.951	journal	November 1954
Theory of polarization of crystalline solids King-Smith, R. D.; Vanderbilt, David Physical Review B, Vol. 47, Issue 3 https://doi.org/10.1103/PhysRevB.47.1651	journal	January 1993
Construction of tight-binding-like potentials on the basis of density-functional theory: Application to carbon Porezag, D.; Frauenheim, Th.; Köhler, Th. Physical Review B, Vol. 51, Issue 19 https://doi.org/10.1103/PhysRevB.51.12947	journal	May 1995
Self-consistent-charge density-functional tight-binding method for simulations of complex materials properties Elstner, M.; Porezag, D.; Jungnickel, G. Physical Review B, Vol. 58, Issue 11, p. 7260-7268 https://doi.org/10.1103/PhysRevB.58.7260	journal	September 1998
Expansion algorithm for the density matrix Niklasson, Anders M. N. Physical Review B, Vol. 66, Issue 15 https://doi.org/10.1103/PhysRevB.66.155115	journal	October 2002
Inhomogeneous Electron Gas Rajagopal, A. K.; Callaway, J. Physical Review B, Vol. 7, Issue 5 https://doi.org/10.1103/PhysRevB.7.1912	journal	March 1973
Iterative refinement method for the approximate factorization of a matrix inverse Niklasson, Anders M. N. Physical Review B, Vol. 70, Issue 19 https://doi.org/10.1103/PhysRevB.70.193102	journal	November 2004
Canonical density matrix perturbation theory Niklasson, Anders M. N.; Cawkwell, M. J.; Rubensson, Emanuel H. Physical Review E, Vol. 92, Issue 6 https://doi.org/10.1103/PhysRevE.92.063301	journal	December 2015
Green’s-function approach to linear response in solids Baroni, Stefano; Giannozzi, Paolo; Testa, Andrea Physical Review Letters, Vol. 58, Issue 18 https://doi.org/10.1103/PhysRevLett.58.1861	journal	May 1987
Crystal Structures of Zirconia from First Principles and Self-Consistent Tight Binding Finnis, M. W.; Paxton, A. T.; Methfessel, M. Physical Review Letters, Vol. 81, Issue 23 https://doi.org/10.1103/PhysRevLett.81.5149	journal	December 1998
Density Matrix Perturbation Theory Niklasson, Anders M. N.; Challacombe, Matt Physical Review Letters, Vol. 92, Issue 19 https://doi.org/10.1103/PhysRevLett.92.193001	journal	May 2004
Ab Initio Linear Scaling Response Theory: Electric Polarizability by Perturbed Projection Weber, Valéry; Niklasson, Anders M. N.; Challacombe, Matt Physical Review Letters, Vol. 92, Issue 19 https://doi.org/10.1103/PhysRevLett.92.193002	journal	May 2004
New Developments in Molecular Orbital Theory Roothaan, C. C. J. Reviews of Modern Physics, Vol. 23, Issue 2 https://doi.org/10.1103/RevModPhys.23.69	journal	April 1951
Weak Interactions in Molecular Quantum Mechanics Karplus, Martin Reviews of Modern Physics, Vol. 32, Issue 2 https://doi.org/10.1103/RevModPhys.32.455	journal	April 1960
Phonons and related crystal properties from density-functional perturbation theory Baroni, Stefano; de Gironcoli, Stefano; Dal Corso, Andrea Reviews of Modern Physics, Vol. 73, Issue 2 https://doi.org/10.1103/RevModPhys.73.515	journal	July 2001
Fast Batched Matrix Multiplication for Small Sizes Using Half-Precision Arithmetic on GPUs Abdelfattah, Ahmad; Tomov, Stanimire; Dongarra, Jack 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2019.00022	conference	May 2019
NVIDIA Tensor Core Programmability, Performance & Precision Markidis, Stefano; Chien, Steven Wei Der; Laure, Erwin 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) https://doi.org/10.1109/IPDPSW.2018.00091	conference	May 2018
Functions of Matrices Higham, Nicholas J. Society for Industrial and Applied Mathematics https://doi.org/10.1137/1.9780898717778	book	January 2008
Interior Eigenvalues from Density Matrix Expansions in Quantum Mechanical Molecular Dynamics Rubensson, Emanuel H.; Niklasson, Anders M. N. SIAM Journal on Scientific Computing, Vol. 36, Issue 2 https://doi.org/10.1137/130911585	journal	January 2014
A Chain Rule for Matrix Functions and Applications Mathias, Roy SIAM Journal on Matrix Analysis and Applications, Vol. 17, Issue 3 https://doi.org/10.1137/S0895479895283409	journal	July 1996
A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method Benzi, Michele; Meyer, Carl D.; Tůma, Miroslav SIAM Journal on Scientific Computing, Vol. 17, Issue 5 https://doi.org/10.1137/S1064827594271421	journal	September 1996
Extreme-scale ab initio quantum raman spectra simulations on the leadership HPC system in China Shang, Honghui; Li, Fang; Zhang, Yunquan Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1145/3458817.3487402	conference	November 2021
Macroscopic Electric Polarization as a Geometric Quantum Phase Resta, R. Europhysics Letters (EPL), Vol. 22, Issue 2 https://doi.org/10.1209/0295-5075/22/2/010	journal	April 1993
Numerical behavior of NVIDIA tensor cores Fasi, Massimiliano; Higham, Nicholas J.; Mikaitis, Mantas PeerJ Computer Science, Vol. 7 https://doi.org/10.7717/peerj-cs.330	journal	January 2021

Similar Records

Quantum-Based Molecular Dynamics Simulations Using Tensor Cores

Journal Article · 2021 · Journal of Chemical Theory and Computation · OSTI ID:1825454

Finkelstein, Joshua David; Smith, Justin Steven; Mniszewski, Susan M.; +4 more

Introducing GPU Acceleration into the Python-Based Simulations of Chemistry Framework

Journal Article · 2025 · Journal of Physical Chemistry. A, Molecules, Spectroscopy, Kinetics, Environment, and General Theory · OSTI ID:2505073

Li, Rui; Sun, Qiming; Zhang, Xing; +1 more

Mixed-Precision S/DGEMM Using the TF32 and TF64 Frameworks on Low-Precision AI Tensor Cores

Conference · 2023 · OSTI ID:2438716

Valero Lara, Pedro; Vetter, Jeffrey

Related Subjects

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY
Chemical calculations
Electron correlation
Layers
Mathematics
Neural networks
Order

Title: Quantum Perturbation Theory Using Tensor Cores and a Deep Neural Network

Citation Formats

References (67)

Similar Records

Related Subjects