U.S. Department of EnergyOffice of Scientific and Technical Information
Double-buffered, heterogeneous CPU + GPU integral digestion algorithm for single-excitation calculations involving a large number of excited states
Abstract
The most widely used quantum-chemical models for excited states are single-excitation theories, a category that includes configuration interaction with single substitutions, time-dependent density functional theory, and also a recently developed ab initio exciton model. When a large number of excited states are desired, these calculations incur a significant bottleneck in the “digestion” step in which two-electron integrals are contracted with density or density-like matrices. We present an implementation that moves this step onto graphical processing units (GPUs), and introduce a double-buffer scheme that minimizes latency by computing integrals on the central processing units (CPUs) concurrently with their digestion on the GPUs. An automatic code generation scheme simplifies the implementation of high-performance GPU kernels. For the exciton model, which requires separate excited-state calculations on each electronically coupled chromophore, the heterogeneous implementation described here results in speedups of 2–6× versus a CPU-only implementation. For traditional time-dependent density functional theory calculations, we obtain speedups of up to 5× when a large number of excited states is computed.
- Authors:
-
[3]
- The Ohio State Univ., Columbus, OH (United States); Q‐Chem Inc., Pleasanton, CA (United States)
- Q‐Chem Inc., Pleasanton, CA (United States)
- The Ohio State Univ., Columbus, OH (United States)
- Publication Date:
- Research Org.:
- The Ohio State Univ., Columbus, OH (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Basic Energy Sciences (BES). Chemical Sciences, Geosciences, and Biosciences Division
- OSTI Identifier:
- 1604460
- Alternate Identifier(s):
- OSTI ID: 1479532
- Grant/Contract Number:
- SC0008850; SC0008550
- Resource Type:
- Journal Article: Accepted Manuscript
- Journal Name:
- Journal of Computational Chemistry
- Additional Journal Information:
- Journal Volume: 39; Journal Issue: 26; Journal ID: ISSN 0192-8651
- Publisher:
- Wiley
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Morrison, Adrian F., Epifanovsky, Evgeny, and Herbert, John M. Double-buffered, heterogeneous CPU + GPU integral digestion algorithm for single-excitation calculations involving a large number of excited states. United States: N. p., 2018.
Web. doi:10.1002/jcc.25531.
Morrison, Adrian F., Epifanovsky, Evgeny, & Herbert, John M. Double-buffered, heterogeneous CPU + GPU integral digestion algorithm for single-excitation calculations involving a large number of excited states. United States. https://doi.org/10.1002/jcc.25531
Morrison, Adrian F., Epifanovsky, Evgeny, and Herbert, John M. 2018.
"Double-buffered, heterogeneous CPU + GPU integral digestion algorithm for single-excitation calculations involving a large number of excited states". United States. https://doi.org/10.1002/jcc.25531. https://www.osti.gov/servlets/purl/1604460.
@article{osti_1604460,
title = {Double-buffered, heterogeneous CPU + GPU integral digestion algorithm for single-excitation calculations involving a large number of excited states},
author = {Morrison, Adrian F. and Epifanovsky, Evgeny and Herbert, John M.},
abstractNote = {The most widely used quantum-chemical models for excited states are single-excitation theories, a category that includes configuration interaction with single substitutions, time-dependent density functional theory, and also a recently developed ab initio exciton model. When a large number of excited states are desired, these calculations incur a significant bottleneck in the “digestion” step in which two-electron integrals are contracted with density or density-like matrices. We present an implementation that moves this step onto graphical processing units (GPUs), and introduce a double-buffer scheme that minimizes latency by computing integrals on the central processing units (CPUs) concurrently with their digestion on the GPUs. An automatic code generation scheme simplifies the implementation of high-performance GPU kernels. For the exciton model, which requires separate excited-state calculations on each electronically coupled chromophore, the heterogeneous implementation described here results in speedups of 2–6× versus a CPU-only implementation. For traditional time-dependent density functional theory calculations, we obtain speedups of up to 5× when a large number of excited states is computed.},
doi = {10.1002/jcc.25531},
url = {https://www.osti.gov/biblio/1604460},
journal = {Journal of Computational Chemistry},
issn = {0192-8651},
number = 26,
volume = 39,
place = {United States},
year = {Wed Oct 03 00:00:00 EDT 2018},
month = {Wed Oct 03 00:00:00 EDT 2018}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 5 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Ab Initio Implementation of the Frenkel–Davydov Exciton Model: A Naturally Parallelizable Approach to Computing Collective Excitations in Crystals and Aggregates
journal, November 2014
- Morrison, Adrian F.; You, Zhi-Qiang; Herbert, John M.
- Journal of Chemical Theory and Computation, Vol. 10, Issue 12
Low-Scaling Quantum Chemistry Approach to Excited-State Properties via an ab Initio Exciton Model: Application to Excitation Energy Transfer in a Self-Assembled Nanotube
journal, October 2015
- Morrison, Adrian F.; Herbert, John M.
- The Journal of Physical Chemistry Letters, Vol. 6, Issue 21
Evidence for Singlet Fission Driven by Vibronic Coherence in Crystalline Tetracene
journal, March 2017
- Morrison, Adrian F.; Herbert, John M.
- The Journal of Physical Chemistry Letters, Vol. 8, Issue 7
Analytic derivative couplings and first-principles exciton/phonon coupling constants for an ab initio Frenkel-Davydov exciton model: Theory, implementation, and application to compute triplet exciton mobility parameters for crystalline tetracene
journal, June 2017
- Morrison, Adrian F.; Herbert, John M.
- The Journal of Chemical Physics, Vol. 146, Issue 22
Beyond Time-Dependent Density Functional Theory Using Only Single Excitations: Methods for Computational Studies of Excited States in Complex Systems
journal, April 2016
- Herbert, John M.; Zhang, Xing; Morrison, Adrian F.
- Accounts of Chemical Research, Vol. 49, Issue 5
Energy Transfer Mechanisms and the Molecular Exciton Model for Molecular Aggregates
journal, September 1963
- Kasha, Michael
- Radiation Research, Vol. 20, Issue 1
The exciton model in molecular spectroscopy
journal, January 1965
- Kasha, M.; Rawls, H. R.; Ashraf El-Bayoumi, M.
- Pure and Applied Chemistry, Vol. 11, Issue 3-4
Charge Transfer and Polarization in Solvated Proteins from Ab Initio Molecular Dynamics
journal, June 2011
- Ufimtsev, Ivan S.; Luehr, Nathan; Martinez, Todd J.
- The Journal of Physical Chemistry Letters, Vol. 2, Issue 14
Ab Initio Quantum Chemistry for Protein Structures
journal, October 2012
- Kulik, Heather J.; Luehr, Nathan; Ufimtsev, Ivan S.
- The Journal of Physical Chemistry B, Vol. 116, Issue 41
Nanoscale Multireference Quantum Chemistry: Full Configuration Interaction on Graphical Processing Units
journal, September 2015
- Fales, B. Scott; Levine, Benjamin G.
- Journal of Chemical Theory and Computation, Vol. 11, Issue 10
GROMACS-the road ahead: GROMACS-the road ahead
journal, April 2011
- van der Spoel, David; Hess, Berk
- Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 1, Issue 5
Two-electron integral evaluation on the graphics processor unit
journal, January 2007
- Yasuda, Koji
- Journal of Computational Chemistry, Vol. 29, Issue 3
Quantum Chemistry on Graphical Processing Units. 1. Strategies for Two-Electron Integral Evaluation
journal, January 2008
- Ufimtsev, Ivan S.; Martínez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 4, Issue 2
Automated Code Engine for Graphical Processing Units: Application to the Effective Core Potential Integrals and Gradients
journal, December 2015
- Song, Chenchen; Wang, Lee-Ping; Martínez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 12, Issue 1
Uncontracted Rys Quadrature Implementation of up to G Functions on Graphical Processing Units
journal, February 2010
- Asadchev, Andrey; Allada, Veerendra; Felder, Jacob
- Journal of Chemical Theory and Computation, Vol. 6, Issue 3
Acceleration of High Angular Momentum Electron Repulsion Integrals and Integral Derivatives on Graphics Processing Units
journal, March 2015
- Miao, Yipu; Merz, Kenneth M.
- Journal of Chemical Theory and Computation, Vol. 11, Issue 4
Arbitrary Angular Momentum Electron Repulsion Integrals with Graphical Processing Units: Application to the Resolution of Identity Hartree–Fock Method
journal, June 2017
- Kalinowski, Jaroslaw; Wennmohs, Frank; Neese, Frank
- Journal of Chemical Theory and Computation, Vol. 13, Issue 7
Quantum supercharger library: Hyper-parallel integral derivatives algorithms for ab initio QM/MM dynamics
journal, May 2015
- Renison, C. Alicia; Fernandes, Kyle D.; Naidoo, Kevin J.
- Journal of Computational Chemistry, Vol. 36, Issue 18
Quantum supercharger library: Hyper-parallelism of the Hartree-Fock method
journal, May 2015
- Fernandes, Kyle D.; Renison, C. Alicia; Naidoo, Kevin J.
- Journal of Computational Chemistry, Vol. 36, Issue 18
Accelerating Density Functional Calculations with Graphics Processing Unit
journal, July 2008
- Yasuda, Koji
- Journal of Chemical Theory and Computation, Vol. 4, Issue 8
Quantum Chemistry on Graphical Processing Units. 2. Direct Self-Consistent-Field Implementation
journal, March 2009
- Ufimtsev, Ivan S.; Martinez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 5, Issue 4
Quantum Chemistry on Graphical Processing Units. 3. Analytical Energy Gradients, Geometry Optimization, and First Principles Molecular Dynamics
journal, August 2009
- Ufimtsev, Ivan S.; Martinez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 5, Issue 10
Linear-scaling self-consistent field calculations based on divide-and-conquer method using resolution-of-identity approximation on graphical processing units
journal, November 2014
- Yoshikawa, Takeshi; Nakai, Hiromi
- Journal of Computational Chemistry, Vol. 36, Issue 3
Excited-State Electronic Structure with Configuration Interaction Singles and Tamm–Dancoff Time-Dependent Density Functional Theory on Graphical Processing Units
journal, May 2011
- Isborn, Christine M.; Luehr, Nathan; Ufimtsev, Ivan S.
- Journal of Chemical Theory and Computation, Vol. 7, Issue 6
The prism algorithm for two-electron integrals
journal, December 1991
- Gill, Peter M. W.; Pople, John A.
- International Journal of Quantum Chemistry, Vol. 40, Issue 6
Molecular integrals Over Gaussian Basis Functions
book, January 1994
- Gill, Peter M. W.
- Advances in Quantum Chemistry
A tensor approach to two-electron matrix elements
journal, July 1997
- Adams, Terry R.; Adamson, Ross D.; Gill, Peter M. W.
- The Journal of Chemical Physics, Vol. 107, Issue 1
Accelerating Resolution-of-the-Identity Second-Order Møller−Plesset Quantum Chemistry Calculations with Graphical Processing Units †
journal, March 2008
- Vogt, Leslie; Olivares-Amaya, Roberto; Kermes, Sean
- The Journal of Physical Chemistry A, Vol. 112, Issue 10
Accelerating Correlated Quantum Chemistry Calculations Using Graphical Processing Units and a Mixed Precision Matrix Multiplication Library
journal, December 2009
- Olivares-Amaya, Roberto; Watson, Mark A.; Edgar, Richard G.
- Journal of Chemical Theory and Computation, Vol. 6, Issue 1
A new approach for second-order perturbation theory
journal, March 2016
- Tomlinson, David G.; Asadchev, Andrey; Gordon, Mark S.
- Journal of Computational Chemistry, Vol. 37, Issue 14
Atomic orbital-based SOS-MP2 with tensor hypercontraction. I. GPU-based tensor construction and exploiting sparsity
journal, May 2016
- Song, Chenchen; Martínez, Todd J.
- The Journal of Chemical Physics, Vol. 144, Issue 17
Large-Scale Functional Group Symmetry-Adapted Perturbation Theory on Graphical Processing Units
journal, January 2018
- Parrish, Robert M.; Thompson, Keiran C.; Martínez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 14, Issue 3
GPU-Based Implementations of the Noniterative Regularized-CCSD(T) Corrections: Applications to Strongly Correlated Systems
journal, April 2011
- Ma, Wenjing; Krishnamoorthy, Sriram; Villa, Oreste
- Journal of Chemical Theory and Computation, Vol. 7, Issue 5
Fast and Flexible Coupled Cluster Implementation
journal, July 2013
- Asadchev, Andrey; Gordon, Mark S.
- Journal of Chemical Theory and Computation, Vol. 9, Issue 8
An atomic orbital-based formulation of the complete active space self-consistent field method on graphical processing units
journal, June 2015
- Hohenstein, Edward G.; Luehr, Nathan; Ufimtsev, Ivan S.
- The Journal of Chemical Physics, Vol. 142, Issue 22
A direct-compatible formulation of the coupled perturbed complete active space self-consistent field equations on graphical processing units
journal, May 2017
- Snyder, James W.; Fales, B. Scott; Hohenstein, Edward G.
- The Journal of Chemical Physics, Vol. 146, Issue 17
Complete active space configuration interaction from state-averaged configuration interaction singles natural orbitals: Analytic first derivatives and derivative coupling vectors
journal, September 2017
- Fales, B. Scott; Shu, Yinan; Levine, Benjamin G.
- The Journal of Chemical Physics, Vol. 147, Issue 9
New implementation of high-level correlated methods using a general block tensor library for high-performance electronic structure calculations
journal, July 2013
- Epifanovsky, Evgeny; Wormit, Michael; Kuś, Tomasz
- Journal of Computational Chemistry, Vol. 34, Issue 26
New algorithm for tensor contractions on multi-core CPUs, GPUs, and accelerators enables CCSD and EOM-CCSD calculations with over 1000 basis functions on a single compute node
journal, March 2017
- Kaliman, Ilya A.; Krylov, Anna I.
- Journal of Computational Chemistry, Vol. 38, Issue 11
Single-Reference ab Initio Methods for the Calculation of Excited States of Large Molecules
journal, November 2005
- Dreuw, Andreas; Head-Gordon, Martin
- Chemical Reviews, Vol. 105, Issue 11
A Structural Model for a Self-Assembled Nanotube Provides Insight into Its Exciton Dynamics
journal, June 2015
- Gao, Min; Paul, Subhradip; Schwieters, Charles D.
- The Journal of Physical Chemistry C, Vol. 119, Issue 24
Sum-over-States Calculation of the Specific Rotations of Some Substituted Oxiranes, Chloropropionitrile, Ethane, and Norbornenone
journal, December 2006
- Wiberg, Kenneth B.; Wang, Yi-gui; Wilson, Shaun M.
- The Journal of Physical Chemistry A, Vol. 110, Issue 51
Calculation of the Term of Magnetic Circular Dichroism. A Time-Dependent Density Functional Theory Approach
journal, December 2006
- Seth, Michael; Autschbach, Jochen; Ziegler, Tom
- Journal of Chemical Theory and Computation, Vol. 3, Issue 2
Origin-independent sum over states simulations of magnetic and electronic circular dichroism spectra via the localized orbital/local origin method
journal, February 2015
- Štěpánek, Petr; Bouř, Petr
- Journal of Computational Chemistry, Vol. 36, Issue 10
Computation of magnetic circular dichroism by sum-over-states summations
journal, March 2013
- Štěpánek, Petr; Bouř, Petr
- Journal of Computational Chemistry, Vol. 34, Issue 18
Electronic Excitations and Spectra in Single-Stranded DNA
journal, June 2008
- Tonzani, Stefano; Schatz, George C.
- Journal of the American Chemical Society, Vol. 130, Issue 24
On the magnetic circular dichroism of benzene. A density-functional study
journal, April 2017
- Kaminský, Jakub; Kříž, Jan; Bouř, Petr
- The Journal of Chemical Physics, Vol. 146, Issue 14
Advances in molecular quantum chemistry contained in the Q-Chem 4 program package
journal, September 2014
- Shao, Yihan; Gan, Zhengting; Epifanovsky, Evgeny
- Molecular Physics, Vol. 113, Issue 2
The analytical gradient of dual-basis resolution-of-the-identity second-order Møller–Plesset perturbation theory
journal, October 2007
- Distasio, Robert A.; Steele, Ryan P.; Head-Gordon, Martin
- Molecular Physics, Vol. 105, Issue 19-22
Double-hybrid density functional theory for excited electronic states of molecules
journal, October 2007
- Grimme, Stefan; Neese, Frank
- The Journal of Chemical Physics, Vol. 127, Issue 15
Efficient calculation of integrals in mixed ramp-Gaussian basis sets
journal, April 2015
- McKemmish, Laura K.
- The Journal of Chemical Physics, Vol. 142, Issue 13
The impact of the resolution of the identity approximate integral method on modern ab initio algorithm development
journal, October 1997
- Kendall, Rick A.; Früchtl, Herbert A.
- Theoretical Chemistry Accounts: Theory, Computation, and Modeling (Theoretica Chimica Acta), Vol. 97, Issue 1-4
The Coulomb operator in a Gaussian product basis
journal, December 1995
- Reza Ahmadi, G.; Almlöf, Jan
- Chemical Physics Letters, Vol. 246, Issue 4-5
Calculation of excitation energies within time-dependent density functional theory using auxiliary basis set expansions
journal, January 1997
- Bauernschmitt, Rüdiger; Häser, Marco; Treutler, Oliver
- Chemical Physics Letters, Vol. 264, Issue 6
Efficient use of the resolution of the identity approximation in time-dependent density functional calculations with hybrid density functionals
journal, August 2002
- Neese, Frank; Olbrich, Gottfried
- Chemical Physics Letters, Vol. 362, Issue 1-2
Resolution-of-identity approach to Hartree–Fock, hybrid density functionals, RPA, MP2 and GW with numeric atom-centered orbital basis functions
journal, May 2012
- Ren, Xinguo; Rinke, Patrick; Blum, Volker
- New Journal of Physics, Vol. 14, Issue 5
Efficient Implementation of the Pair Atomic Resolution of the Identity Approximation for Exact Exchange for Hybrid and Range-Separated Density Functionals
journal, January 2015
- Manzer, Samuel F.; Epifanovsky, Evgeny; Head-Gordon, Martin
- Journal of Chemical Theory and Computation, Vol. 11, Issue 2
Fast, accurate evaluation of exact exchange: The occ-RI-K algorithm
journal, July 2015
- Manzer, Samuel; Horn, Paul R.; Mardirossian, Narbe
- The Journal of Chemical Physics, Vol. 143, Issue 2
Excitation energies from an auxiliary-function formulation of time-dependent density-functional response theory with charge conservation constraint
journal, April 2006
- Ipatov, Andrei; Fouqueau, Antony; Valle, Carlos Perez del
- Journal of Molecular Structure: THEOCHEM, Vol. 762, Issue 1-3
Ab Initio Nonadiabatic Dynamics of Multichromophore Complexes: A Scalable Graphical-Processing-Unit-Accelerated Exciton Framework
journal, July 2014
- Sisto, Aaron; Glowacki, David R.; Martinez, Todd J.
- Accounts of Chemical Research, Vol. 47, Issue 9
Atomistic non-adiabatic dynamics of the LH2 complex with a GPU-accelerated ab initio exciton model
journal, January 2017
- Sisto, Aaron; Stross, Clem; van der Kamp, Marc W.
- Physical Chemistry Chemical Physics, Vol. 19, Issue 23
An Ab Initio Exciton Model Including Charge-Transfer Excited States
journal, July 2017
- Li, Xin; Parrish, Robert M.; Liu, Fang
- Journal of Chemical Theory and Computation, Vol. 13, Issue 8
An efficient implementation of time-dependent density-functional theory for the calculation of excitation energies of large molecules
journal, November 1998
- Stratmann, R. Eric; Scuseria, Gustavo E.; Frisch, Michael J.
- The Journal of Chemical Physics, Vol. 109, Issue 19
The iterative calculation of a few of the lowest eigenvalues and corresponding eigenvectors of large real-symmetric matrices
journal, January 1975
- Davidson, Ernest R.
- Journal of Computational Physics, Vol. 17, Issue 1
Spectroscopic and Computational Studies of Co 2+ Corrinoids: Spectral and Electronic Properties of the Biologically Relevant Base-On and Base-Off Forms of Co 2+ Cobalamin
journal, August 2004
- Stich, Troy A.; Buan, Nicole R.; Brunold, Thomas C.
- Journal of the American Chemical Society, Vol. 126, Issue 31
Electronically Excited States of Vitamin B 12 : Benchmark Calculations Including Time-Dependent Density Functional Theory and Correlated ab Initio Methods
journal, February 2011
- Kornobis, Karina; Kumar, Neeraj; Wong, Bryan M.
- The Journal of Physical Chemistry A, Vol. 115, Issue 7
Electronically Excited States of Vitamin B 12 and Methylcobalamin: Theoretical Analysis of Absorption, CD, and MCD Data
journal, February 2011
- Solheim, Harald; Kornobis, Karina; Ruud, Kenneth
- The Journal of Physical Chemistry B, Vol. 115, Issue 4
Single-Reference ab initio Methods for the Calculation of Excited States of Large Molecules
journal, January 2006
- Dreuw, Andreas; Head-Gordon, Martin
- ChemInform, Vol. 37, Issue 5
Works referencing / citing this record:
GPU‐Accelerated Large‐Scale Excited‐State Simulation Based on Divide‐and‐Conquer Time‐Dependent Density‐Functional Tight‐Binding
journal, August 2019
- Yoshikawa, Takeshi; Komoto, Nana; Nishimura, Yoshifumi
- Journal of Computational Chemistry, Vol. 40, Issue 31