Coupled‐cluster singles, doubles and perturbative triples with density fitting approximation for massively parallel heterogeneous platforms

Peng, Chong; Calvin, Justus A.; Valeev, Edward F.

doi:10.1002/qua.25894

Coupled‐cluster singles, doubles and perturbative triples with density fitting approximation for massively parallel heterogeneous platforms

Journal Article · Fri Jan 11 23:00:00 EST 2019 · International Journal of Quantum Chemistry

DOI:https://doi.org/10.1002/qua.25894· OSTI ID:1490758

Peng, Chong ^[1]; Calvin, Justus A. ^[1]; ^[1]

Department of Chemistry, Virginia Tech Blacksburg Virginia

Abstract

A high‐performance implementation of the coupled‐cluster singles, doubles, and perturbative triples [CCSD(T)] is developed in the Massively Parallel Quantum Chemistry program. Novel features include: (1) reduced memory requirements via a density‐fitting (DF) CCSD implementation utilizing distributed lazy evaluation for tensors with more than two unoccupied indices and (2) the ability to utilize efficiently many‐core nodes (Intel Xeon Phi) and heterogeneous nodes with multiple NVIDIA GPUs on each node. All data that are greater than quadratic in the system size are distributed among processes. Excellent strong scaling is observed on distributed‐memory computers equipped with conventional CPUs, Intel Xeon Phi processors, and heterogeneous nodes with multiple NVIDIA GPUs Canonical CCSD(T) energies can be evaluated for systems containing 200 electrons and 1000 basis functions in a few days using a small size commodity cluster, with even larger computations possible on leadership‐class computing resources.

View Accepted Manuscript (Publisher)

Sponsoring Organization:: USDOE

OSTI ID:: 1490758

Journal Information:: International Journal of Quantum Chemistry, Journal Name: International Journal of Quantum Chemistry Journal Issue: 12 Vol. 119; ISSN 0020-7608

Publisher:: Wiley Blackwell (John Wiley & Sons)Copyright Statement

Country of Publication:: United States

Language:: English

References (66)

Massively parallel algorithm and implementation of RI-MP2 energy calculation for peta-scale many-core supercomputers Katouda, Michio; Naruse, Akira; Hirano, Yukihiko Journal of Computational Chemistry, Vol. 37, Issue 30 https://doi.org/10.1002/jcc.24491	journal	September 2016
New algorithm for tensor contractions on multi-core CPUs, GPUs, and accelerators enables CCSD and EOM-CCSD calculations with over 1000 basis functions on a single compute node Kaliman, Ilya A.; Krylov, Anna I. Journal of Computational Chemistry, Vol. 38, Issue 11 https://doi.org/10.1002/jcc.24713	journal	March 2017
Efficient parallel algorithm of second-order Møller-Plesset perturbation theory with resolution-of-identity approximation (RI-MP2) Katouda, Michio; Nagase, Shigeru International Journal of Quantum Chemistry, Vol. 109, Issue 10 https://doi.org/10.1002/qua.22068	journal	January 2009
Correlation problems in atomic and molecular systems III. Rederivation of the coupled-pair many-electron theory using the traditional quantum chemical methodst ?i?ek, J.; Paldus, J. International Journal of Quantum Chemistry, Vol. 5, Issue 4 https://doi.org/10.1002/qua.560050402	journal	July 1971
Evaluation of the contribution from triply excited intermediates to the fourth-order perturbation theory energy on Intel distributed memory supercomputers Rendell, Alistair P.; Lee, Timothy J.; Komornicki, Andrew Theoretica Chimica Acta, Vol. 84, Issue 4-5 https://doi.org/10.1007/BF01113267	journal	January 1993
Optimizing tensor contraction expressions for hybrid CPU-GPU execution Ma, Wenjing; Krishnamoorthy, Sriram; Villa, Oreste Cluster Computing, Vol. 16, Issue 1 https://doi.org/10.1007/s10586-011-0179-2	journal	November 2011
A parallel vectorized implementation of triple excitations in CCSD(T): application to the binding energies of the AlH3, AlH2F, AlHF2 and AlF3 dimers Rendell, Alistair P.; Lee, Timothy J.; Komornicki, Andrew Chemical Physics Letters, Vol. 178, Issue 5-6 https://doi.org/10.1016/0009-2614(91)87003-T	journal	April 1991
Use of approximate integrals in ab initio theory. An application in MP2 energy calculations Feyereisen, Martin; Fitzgerald, George; Komornicki, Andrew Chemical Physics Letters, Vol. 208, Issue 5-6 https://doi.org/10.1016/0009-2614(93)87156-W	journal	June 1993
Integral approximations for LCAO-SCF calculations Vahtras, O.; Almlöf, J.; Feyereisen, M. W. Chemical Physics Letters, Vol. 213, Issue 5-6 https://doi.org/10.1016/0009-2614(93)89151-7	journal	October 1993
Large-scale correlated electronic structure calculations: the RI-MP2 method on parallel computers Bernholdt, David E.; Harrison, Robert J. Chemical Physics Letters, Vol. 250, Issue 5-6 https://doi.org/10.1016/0009-2614(96)00054-1	journal	March 1996
A fifth-order perturbation comparison of electron correlation theories Raghavachari, Krishnan; Trucks, Gary W.; Pople, John A. Chemical Physics Letters, Vol. 157, Issue 6 https://doi.org/10.1016/S0009-2614(89)87395-6	journal	May 1989
A direct coupled cluster algorithm for massively parallel computers Kobayashi, Rika; Rendell, Alistair P. Chemical Physics Letters, Vol. 265, Issue 1-2 https://doi.org/10.1016/S0009-2614(96)01387-5	journal	January 1997
Why CCSD(T) works: a different perspective Stanton, John F. Chemical Physics Letters, Vol. 281, Issue 1-3 https://doi.org/10.1016/S0009-2614(97)01144-5	journal	December 1997
RI-MP2: optimized auxiliary basis sets and demonstration of efficiency Weigend, Florian; Häser, Marco; Patzelt, Holger Chemical Physics Letters, Vol. 294, Issue 1-3 https://doi.org/10.1016/S0009-2614(98)00862-8	journal	September 1998
Scalability of correlated electronic structure calculations on parallel computers: A case study of the RI-MP2 method Bernholdt, David E. Parallel Computing, Vol. 26, Issue 7-8 https://doi.org/10.1016/S0167-8191(00)00020-X	journal	July 2000
Coupled cluster algorithms for networks of shared memory parallel processors Bentz, Jonathan L.; Olson, Ryan M.; Gordon, Mark S. Computer Physics Communications, Vol. 176, Issue 9-10 https://doi.org/10.1016/j.cpc.2007.03.001	journal	May 2007
An efficient tensor transpose algorithm for multicore CPU, Intel Xeon Phi, and NVidia Tesla GPU Lyakh, Dmitry I. Computer Physics Communications, Vol. 189 https://doi.org/10.1016/j.cpc.2014.12.013	journal	April 2015
Implementation of the CCSD(T)(F12) method using numerical quadratures Bokhan, Denis; Bernadotte, Stephan; Ten-no, Seiichiro Chemical Physics Letters, Vol. 469, Issue 1-3 https://doi.org/10.1016/j.cplett.2008.12.054	journal	February 2009
Scalable Electron Correlation Methods. 3. Efficient and Accurate Parallel Local Coupled Cluster with Pair Natural Orbitals (PNO-LCCSD) Schwilk, Max; Ma, Qianli; Köppl, Christoph Journal of Chemical Theory and Computation, Vol. 13, Issue 8 https://doi.org/10.1021/acs.jctc.7b00554	journal	July 2017
Scalable Electron Correlation Methods. 4. Parallel Explicitly Correlated Local Coupled Cluster with Pair Natural Orbitals (PNO-LCCSD-F12) Ma, Qianli; Schwilk, Max; Köppl, Christoph Journal of Chemical Theory and Computation, Vol. 13, Issue 10 https://doi.org/10.1021/acs.jctc.7b00799	journal	September 2017
Scalable Electron Correlation Methods. 5. Parallel Perturbative Triples Correction for Explicitly Correlated Local Coupled Cluster with Pair Natural Orbitals Ma, Qianli; Werner, Hans-Joachim Journal of Chemical Theory and Computation, Vol. 14, Issue 1 https://doi.org/10.1021/acs.jctc.7b01141	journal	December 2017
Massively Parallel Implementation of Explicitly Correlated Coupled-Cluster Singles and Doubles Using TiledArray Framework Peng, Chong; Calvin, Justus A.; Pavošević, Fabijan The Journal of Physical Chemistry A, Vol. 120, Issue 51 https://doi.org/10.1021/acs.jpca.6b10150	journal	December 2016
Coupled Cluster Theory on Graphics Processing Units I. The Coupled Cluster Doubles Method DePrince, A. Eugene; Hammond, Jeff R. Journal of Chemical Theory and Computation, Vol. 7, Issue 5 https://doi.org/10.1021/ct100584w	journal	April 2011
GPU-Based Implementations of the Noniterative Regularized-CCSD(T) Corrections: Applications to Strongly Correlated Systems Ma, Wenjing; Krishnamoorthy, Sriram; Villa, Oreste Journal of Chemical Theory and Computation, Vol. 7, Issue 5 https://doi.org/10.1021/ct1007247	journal	April 2011
S66: A Well-balanced Database of Benchmark Interaction Energies Relevant to Biomolecular Structures Řezáč, Jan; Riley, Kevin E.; Hobza, Pavel Journal of Chemical Theory and Computation, Vol. 7, Issue 8 https://doi.org/10.1021/ct2002946	journal	July 2011
Noniterative Multireference Coupled Cluster Methods on Heterogeneous CPU–GPU Systems Bhaskaran-Nair, Kiran; Ma, Wenjing; Krishnamoorthy, Sriram Journal of Chemical Theory and Computation, Vol. 9, Issue 4 https://doi.org/10.1021/ct301130u	journal	March 2013
Fast and Flexible Coupled Cluster Implementation Asadchev, Andrey; Gordon, Mark S. Journal of Chemical Theory and Computation, Vol. 9, Issue 8 https://doi.org/10.1021/ct400054m	journal	July 2013
Accuracy and Efficiency of Coupled-Cluster Theory Using Density Fitting/Cholesky Decomposition, Frozen Natural Orbitals, and a t ₁ -Transformed Hamiltonian DePrince, A. Eugene; Sherrill, C. David Journal of Chemical Theory and Computation, Vol. 9, Issue 6 https://doi.org/10.1021/ct400250u	journal	May 2013
Optimization of the Coupled Cluster Implementation in NWChem on Petascale Parallel Architectures Anisimov, Victor M.; Bauer, Gregory H.; Chadalavada, Kalyana Journal of Chemical Theory and Computation, Vol. 10, Issue 10 https://doi.org/10.1021/ct500404c	journal	September 2014
Scalable Electron Correlation Methods I.: PNO-LMP2 with Linear Scaling in the Molecular Size and Near-Inverse-Linear Scaling in the Number of Processors Werner, Hans-Joachim; Knizia, Gerald; Krause, Christine Journal of Chemical Theory and Computation, Vol. 11, Issue 2 https://doi.org/10.1021/ct500725e	journal	January 2015
A Novel Approach to Parallel Coupled Cluster Calculations: Combining Distributed and Shared Memory Techniques for Modern Cluster Based Systems Olson, Ryan M.; Bentz, Jonathan L.; Kendall, Ricky A. Journal of Chemical Theory and Computation, Vol. 3, Issue 4 https://doi.org/10.1021/ct600366k	journal	May 2007
Efficient Parallel Implementation of the CCSD External Exchange Operator and the Perturbative Triples (T) Energy Calculation Janowski, Tomasz; Pulay, Peter Journal of Chemical Theory and Computation, Vol. 4, Issue 10 https://doi.org/10.1021/ct800142f	journal	September 2008
Comparison of the quadratic configuration interaction and coupled-cluster approaches to electron correlation including the effect of triple excitations Lee, Timothy J.; Rendell, Alistair P.; Taylor, Peter R. The Journal of Physical Chemistry, Vol. 94, Issue 14 https://doi.org/10.1021/j100377a008	journal	July 1990
Tensor Contraction Engine: Abstraction and Automated Parallel Implementation of Configuration-Interaction, Coupled-Cluster, and Many-Body Perturbation Theories Hirata, So The Journal of Physical Chemistry A, Vol. 107, Issue 46 https://doi.org/10.1021/jp034596z	journal	November 2003
Linear scaling local coupled cluster theory with density fitting. Part I: 4-external integrals Schütz, Martin; Manby, Frederick R. Phys. Chem. Chem. Phys., Vol. 5, Issue 16 https://doi.org/10.1039/B304550A	journal	January 2003
Quintuple-ζ quality coupled-cluster correlation energies with triple-ζ basis sets Tew, David P.; Klopper, Wim; Neiss, Christian Phys. Chem. Chem. Phys., Vol. 9, Issue 16 https://doi.org/10.1039/B617230J	journal	January 2007
Variational formulation of perturbative explicitly-correlated coupled-cluster methods Torheyden, Martin; Valeev, Edward F. Physical Chemistry Chemical Physics, Vol. 10, Issue 23 https://doi.org/10.1039/b803620a	journal	January 2008
Efficient use of the correlation consistent basis sets in resolution of the identity MP2 calculations Weigend, Florian; Köhn, Andreas; Hättig, Christof The Journal of Chemical Physics, Vol. 116, Issue 8 https://doi.org/10.1063/1.1445115	journal	February 2002
Fast linear scaling second-order Møller-Plesset perturbation theory (MP2) using local and density fitting approximations Werner, Hans-Joachim; Manby, Frederick R.; Knowles, Peter J. The Journal of Chemical Physics, Vol. 118, Issue 18 https://doi.org/10.1063/1.1564816	journal	May 2003
Coulombic potential energy integrals and approximations Whitten, J. L. The Journal of Chemical Physics, Vol. 58, Issue 10 https://doi.org/10.1063/1.1679012	journal	May 1973
On the Correlation Problem in Atomic and Molecular Systems. Calculation of Wavefunction Components in Ursell‐Type Expansion Using Quantum‐Field Theoretical Methods Čížek, Jiří The Journal of Chemical Physics, Vol. 45, Issue 11 https://doi.org/10.1063/1.1727484	journal	December 1966
Pseudonatural Orbitals as a Basis for the Superposition of Configurations. I. He ₂ ⁺ Edmiston, C.; Krauss, M. The Journal of Chemical Physics, Vol. 45, Issue 5 https://doi.org/10.1063/1.1727841	journal	September 1966
Critical comparison of various connected quadruple excitation approximations in the coupled-cluster treatment of bond breaking Musiał, Monika; Bartlett, Rodney J. The Journal of Chemical Physics, Vol. 122, Issue 22 https://doi.org/10.1063/1.1926273	journal	June 2005
Coupled-cluster methods including noniterative corrections for quadruple excitations Bomble, Yannick J.; Stanton, John F.; Kállay, Mihály The Journal of Chemical Physics, Vol. 123, Issue 5 https://doi.org/10.1063/1.1950567	journal	August 2005
A simple and efficient CCSD(T)-F12 approximation Adler, Thomas B.; Knizia, Gerald; Werner, Hans-Joachim The Journal of Chemical Physics, Vol. 127, Issue 22 https://doi.org/10.1063/1.2817618	journal	December 2007
Systematically convergent basis sets for explicitly correlated wavefunctions: The atoms H, He, B–Ne, and Al–Ar Peterson, Kirk A.; Adler, Thomas B.; Werner, Hans-Joachim The Journal of Chemical Physics, Vol. 128, Issue 8 https://doi.org/10.1063/1.2831537	journal	February 2008
High-accuracy extrapolated ab initio thermochemistry. III. Additional improvements and overview Harding, Michael E.; Vázquez, Juana; Ruscic, Branko The Journal of Chemical Physics, Vol. 128, Issue 11 https://doi.org/10.1063/1.2835612	journal	March 2008
Parallel implementation of electronic structure energy, gradient, and Hessian calculations Lotrich, V.; Flocke, N.; Ponton, M. The Journal of Chemical Physics, Vol. 128, Issue 19 https://doi.org/10.1063/1.2920482	journal	May 2008
Simple coupled-cluster singles and doubles method with perturbative inclusion of triples and explicitly correlated geminals: The CCSD(T)R12¯ model Valeev, Edward F.; Daniel Crawford, T. The Journal of Chemical Physics, Vol. 128, Issue 24 https://doi.org/10.1063/1.2939577	journal	June 2008
Optimized auxiliary basis sets for explicitly correlated methods Yousaf, Kazim E.; Peterson, Kirk A. The Journal of Chemical Physics, Vol. 129, Issue 18 https://doi.org/10.1063/1.3009271	journal	November 2008
Efficient and accurate local approximations to coupled-electron pair approaches: An attempt to revive the pair natural orbital method Neese, Frank; Wennmohs, Frank; Hansen, Andreas The Journal of Chemical Physics, Vol. 130, Issue 11 https://doi.org/10.1063/1.3086717	journal	March 2009
Extrapolating MP2 and CCSD explicitly correlated correlation energies to the complete basis set limit with first and second row correlation consistent basis sets Hill, J. Grant; Peterson, Kirk A.; Knizia, Gerald The Journal of Chemical Physics, Vol. 131, Issue 19 https://doi.org/10.1063/1.3265857	journal	January 2009
On some approximations in applications of X α theory Dunlap, B. I.; Connolly, J. W. D.; Sabin, J. R. The Journal of Chemical Physics, Vol. 71, Issue 8 https://doi.org/10.1063/1.438728	journal	October 1979
Gaussian basis sets for use in correlated molecular calculations. I. The atoms boron through neon and hydrogen Dunning, Thom H. The Journal of Chemical Physics, Vol. 90, Issue 2 https://doi.org/10.1063/1.456153	journal	January 1989
Comparison of coupled‐cluster methods which include the effects of connected triple excitations Scuseria, Gustavo E.; Lee, Timothy J. The Journal of Chemical Physics, Vol. 93, Issue 8 https://doi.org/10.1063/1.459684	journal	October 1990
Electron affinities of the first‐row atoms revisited. Systematic basis sets and wave functions Kendall, Rick A.; Dunning, Thom H.; Harrison, Robert J. The Journal of Chemical Physics, Vol. 96, Issue 9 https://doi.org/10.1063/1.462569	journal	May 1992
Coupled‐cluster theory employing approximate integrals: An approach to avoid the input/output and storage bottlenecks Rendell, Alistair P.; Lee, Timothy J. The Journal of Chemical Physics, Vol. 101, Issue 1 https://doi.org/10.1063/1.468148	journal	July 1994
General implementation of the resolution-of-the-identity and Cholesky representations of electron repulsion integrals within coupled-cluster and equation-of-motion methods: Theory and benchmarks Epifanovsky, Evgeny; Zuev, Dmitry; Feng, Xintian The Journal of Chemical Physics, Vol. 139, Issue 13 https://doi.org/10.1063/1.4820484	journal	October 2013
Sparse maps—A systematic infrastructure for reduced-scaling electronic structure methods. I. An efficient and simple linear scaling local MP2 method that uses an intermediate basis of pair natural orbitals Pinski, Peter; Riplinger, Christoph; Valeev, Edward F. The Journal of Chemical Physics, Vol. 143, Issue 3 https://doi.org/10.1063/1.4926879	journal	July 2015
Sparse maps—A systematic infrastructure for reduced-scaling electronic structure methods. II. Linear scaling domain based pair natural orbital coupled cluster theory Riplinger, Christoph; Pinski, Peter; Becker, Ute The Journal of Chemical Physics, Vol. 144, Issue 2 https://doi.org/10.1063/1.4939030	journal	January 2016
Perturbative triples correction for local pair natural orbital based explicitly correlated CCSD(F12*) using Laplace transformation techniques Schmitz, Gunnar; Hättig, Christof The Journal of Chemical Physics, Vol. 145, Issue 23 https://doi.org/10.1063/1.4972001	journal	December 2016
SparseMaps—A systematic infrastructure for reduced scaling electronic structure methods. V. Linear scaling explicitly correlated coupled-cluster method with pair natural orbitals Pavošević, Fabijan; Peng, Chong; Pinski, Peter The Journal of Chemical Physics, Vol. 146, Issue 17 https://doi.org/10.1063/1.4979993	journal	May 2017
A new near-linear scaling, efficient and accurate, open-shell domain-based local pair natural orbital coupled cluster singles and doubles theory Saitow, Masaaki; Becker, Ute; Riplinger, Christoph The Journal of Chemical Physics, Vol. 146, Issue 16 https://doi.org/10.1063/1.4981521	journal	April 2017
Auxiliary basis expansions for large-scale electronic structure calculations Jung, Y.; Sodt, A.; Gill, P. M. W. Proceedings of the National Academy of Sciences, Vol. 102, Issue 19 https://doi.org/10.1073/pnas.0408475102	journal	April 2005
Density-fitted singles and doubles coupled cluster on graphics processing units DePrince,, A. Eugene; Kennedy, Matthew R.; Sumpter, Bobby G. Molecular Physics, Vol. 112, Issue 5-6 https://doi.org/10.1080/00268976.2013.874599	journal	January 2014
MADNESS: A Multiresolution, Adaptive Numerical Environment for Scientific Simulation Harrison, Robert J.; Beylkin, Gregory; Bischoff, Florian A. SIAM Journal on Scientific Computing, Vol. 38, Issue 5 https://doi.org/10.1137/15M1026171	journal	January 2016

Similar Records

Density-fitted singles and doubles coupled cluster on graphics processing units

Journal Article · Tue Dec 31 23:00:00 EST 2013 · Molecular Physics · OSTI ID:1116479

Performance Analysis of Memory Transfers and GEMM Subroutines on NVIDIA Tesla GPU Cluster

Conference · Mon Aug 31 00:00:00 EDT 2009 · OSTI ID:965387

Demonstrating GPU code portability and scalability for radiative heat transfer computations

Journal Article · Sun Jul 01 00:00:00 EDT 2018 · Journal of Computational Science · OSTI ID:1565656

Coupled‐cluster singles, doubles and perturbative triples with density fitting approximation for massively parallel heterogeneous platforms

Citation Formats

References (66)

Similar Records

Related Subjects