Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: GPU acceleration of all-electron electronic structure theory using localized numeric atom-centered basis functions

Abstract

We present an implementation of all-electron density-functional theory for massively parallel GPU-based platforms, using localized atom-centered basis functions and real-space integration grids. Special attention is paid to domain decomposition of the problem on non-uniform grids, which enables compute- and memory-parallel execution across thousands of nodes for real-space operations, e.g. the update of the electron density, the integration of the real-space Hamiltonian matrix, and calculation of Pulay forces. To assess the performance of our GPU implementation, we performed benchmarks on three different architectures using a 103-material test set. We find that operations which rely on dense serial linear algebra show dramatic speedups from GPU acceleration: in particular, SCF iterations including force and stress calculations exhibit speedups ranging from 4.5 to 6.6. For the architectures and problem types investigated here, this translates to an expected overall speedup between 3–4 for the entire calculation (including non-GPU accelerated parts), for problems featuring several tens to hundreds of atoms. Additional calculations for a 375-atom Bi2Se3 bilayer show that the present GPU strategy scales for large-scale distributed-parallel simulations.

Authors:
ORCiD logo [1];  [1]; ORCiD logo [1];  [2]; ORCiD logo [1]
+ Show Author Affiliations
  1. Duke Univ., Durham, NC (United States)
  2. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Sciences (CNMS)
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES)
OSTI Identifier:
1631232
Alternate Identifier(s):
OSTI ID: 1619314
Grant/Contract Number:  
AC05-00OR22725; AC52-07NA27344
Resource Type:
Accepted Manuscript
Journal Name:
Computer Physics Communications
Additional Journal Information:
Journal Volume: 254; Journal Issue: C; Journal ID: ISSN 0010-4655
Publisher:
Elsevier
Country of Publication:
United States
Language:
English
Subject:
42 ENGINEERING; GPU acceleration; High performance computing; Electronic structure; Density functional theory; Localized basis sets; Domain decomposition

Citation Formats

Huhn, William P., Lange, Björn, Yu, Victor Wen-zhe, Yoon, Mina, and Blum, Volker. GPU acceleration of all-electron electronic structure theory using localized numeric atom-centered basis functions. United States: N. p., 2020. Web. doi:10.1016/j.cpc.2020.107314.
Copy to clipboard
Huhn, William P., Lange, Björn, Yu, Victor Wen-zhe, Yoon, Mina, & Blum, Volker. GPU acceleration of all-electron electronic structure theory using localized numeric atom-centered basis functions. United States. https://doi.org/10.1016/j.cpc.2020.107314
Copy to clipboard
Huhn, William P., Lange, Björn, Yu, Victor Wen-zhe, Yoon, Mina, and Blum, Volker. Thu . "GPU acceleration of all-electron electronic structure theory using localized numeric atom-centered basis functions". United States. https://doi.org/10.1016/j.cpc.2020.107314. https://www.osti.gov/servlets/purl/1631232.
Copy to clipboard
@article{osti_1631232,
title = {GPU acceleration of all-electron electronic structure theory using localized numeric atom-centered basis functions},
author = {Huhn, William P. and Lange, Björn and Yu, Victor Wen-zhe and Yoon, Mina and Blum, Volker},
abstractNote = {We present an implementation of all-electron density-functional theory for massively parallel GPU-based platforms, using localized atom-centered basis functions and real-space integration grids. Special attention is paid to domain decomposition of the problem on non-uniform grids, which enables compute- and memory-parallel execution across thousands of nodes for real-space operations, e.g. the update of the electron density, the integration of the real-space Hamiltonian matrix, and calculation of Pulay forces. To assess the performance of our GPU implementation, we performed benchmarks on three different architectures using a 103-material test set. We find that operations which rely on dense serial linear algebra show dramatic speedups from GPU acceleration: in particular, SCF iterations including force and stress calculations exhibit speedups ranging from 4.5 to 6.6. For the architectures and problem types investigated here, this translates to an expected overall speedup between 3–4 for the entire calculation (including non-GPU accelerated parts), for problems featuring several tens to hundreds of atoms. Additional calculations for a 375-atom Bi2Se3 bilayer show that the present GPU strategy scales for large-scale distributed-parallel simulations.},
doi = {10.1016/j.cpc.2020.107314},
journal = {Computer Physics Communications},
number = C,
volume = 254,
place = {United States},
year = {Thu Apr 09 00:00:00 EDT 2020},
month = {Thu Apr 09 00:00:00 EDT 2020}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1016/j.cpc.2020.107314
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 27 works
Citation information provided by
Web of Science

Figures / Tables:

Figure 1 Figure 1: Program flow for a typical electronic structure calculation involving geometry relaxation or molecular dynamics. Shaded boxes indicate steps contributing to the actual computational workload. Yellow shading indicates steps that are subject to real-space GPU acceleration in this work, whereas gray shading indicates steps that are GPU-accelerated only partly,more » not at all, or that are handled by separate software components outside the scope of this work.« less
All figures and tables (13 total)
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Recent developments in the ABINIT software package
dataset, January 2019

Inhomogeneous Electron Gas
journal, November 1964

Self-Consistent Equations Including Exchange and Correlation Effects
journal, November 1965

Perspective on density functional theory
journal, April 2012

  • Burke, Kieron
  • The Journal of Chemical Physics, Vol. 136, Issue 15
  • DOI: 10.1063/1.4704546

Perspective: Fifty years of density-functional theory in chemical physics
journal, May 2014

  • Becke, Axel D.
  • The Journal of Chemical Physics, Vol. 140, Issue 18
  • DOI: 10.1063/1.4869598

Self‐consistent molecular orbital methods 25. Supplementary functions for Gaussian basis sets
journal, April 1984

  • Frisch, Michael J.; Pople, John A.; Binkley, J. Stephen
  • The Journal of Chemical Physics, Vol. 80, Issue 7
  • DOI: 10.1063/1.447079

Gaussian basis sets for use in correlated molecular calculations. I. The atoms boron through neon and hydrogen
journal, January 1989

  • Dunning, Thom H.
  • The Journal of Chemical Physics, Vol. 90, Issue 2
  • DOI: 10.1063/1.456153

Gaussian basis sets for use in correlated molecular calculations. VI. Sextuple zeta correlation consistent basis sets for boron through neon
journal, December 1996

Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy
journal, January 2005

  • Weigend, Florian; Ahlrichs, Reinhart
  • Physical Chemistry Chemical Physics, Vol. 7, Issue 18, p. 3297-3305
  • DOI: 10.1039/b508541a

NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations
journal, September 2010

  • Valiev, M.; Bylaska, E. J.; Govind, N.
  • Computer Physics Communications, Vol. 181, Issue 9, p. 1477-1489
  • DOI: 10.1016/j.cpc.2010.04.018

cp2k: atomistic simulations of condensed matter systems
journal, June 2013

  • Hutter, Jürg; Iannuzzi, Marcella; Schiffmann, Florian
  • Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 4, Issue 1
  • DOI: 10.1002/wcms.1159

Turbomole
journal, July 2013

  • Furche, Filipp; Ahlrichs, Reinhart; Hättig, Christof
  • Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 4, Issue 2
  • DOI: 10.1002/wcms.1162

Advances in molecular quantum chemistry contained in the Q-Chem 4 program package
journal, September 2014

The Elephant in the Room of Density Functional Theory Calculations
journal, March 2017

  • Jensen, Stig Rune; Saha, Santanu; Flores-Livas, José A.
  • The Journal of Physical Chemistry Letters, Vol. 8, Issue 7
  • DOI: 10.1021/acs.jpclett.7b00255

Atomic Shielding Constants
journal, July 1930

Chemistry with ADF
journal, January 2001

  • te Velde, G.; Bickelhaupt, F. M.; Baerends, E. J.
  • Journal of Computational Chemistry, Vol. 22, Issue 9, p. 931-967
  • DOI: 10.1002/jcc.1056

Optimized Slater-type basis sets for the elements 1-118
journal, May 2003

  • Van Lenthe, E.; Baerends, E. J.
  • Journal of Computational Chemistry, Vol. 24, Issue 9
  • DOI: 10.1002/jcc.10255

An efficient numerical multicenter basis set for molecular orbital calculations: Application to FeCl 4
journal, December 1973

  • Averill, F. W.; Ellis, D. E.
  • The Journal of Chemical Physics, Vol. 59, Issue 12
  • DOI: 10.1063/1.1680020

Efficient and accurate expansion methods for molecules in local density models
journal, February 1982

  • Delley, B.; Ellis, D. E.
  • The Journal of Chemical Physics, Vol. 76, Issue 4
  • DOI: 10.1063/1.443168

Ab initio multicenter tight-binding model for molecular-dynamics simulations and other applications in covalent systems
journal, August 1989

An all‐electron numerical method for solving the local density functional for polyatomic molecules
journal, January 1990

  • Delley, B.
  • The Journal of Chemical Physics, Vol. 92, Issue 1
  • DOI: 10.1063/1.458452

Efficient ab initio tight binding
journal, September 1997

Full-potential nonorthogonal local-orbital minimum-basis band-structure scheme
journal, January 1999

Ab initio molecular simulations with numeric atom-centered orbitals
journal, November 2009

  • Blum, Volker; Gehrke, Ralf; Hanke, Felix
  • Computer Physics Communications, Vol. 180, Issue 11
  • DOI: 10.1016/j.cpc.2009.06.022

Achieving linear scaling in exchange-correlation density functional quadratures
journal, July 1996

  • Stratmann, R. Eric; Scuseria, Gustavo E.; Frisch, Michael J.
  • Chemical Physics Letters, Vol. 257, Issue 3-4
  • DOI: 10.1016/0009-2614(96)00600-8

Linear Scaling Density Functional Calculations with Gaussian Orbitals
journal, June 1999

  • Scuseria, Gustavo E.
  • The Journal of Physical Chemistry A, Vol. 103, Issue 25
  • DOI: 10.1021/jp990629s

Efficient integration for all-electron electronic structure calculation using numeric basis functions
journal, December 2009

Quantum Chemistry on Graphical Processing Units. 1. Strategies for Two-Electron Integral Evaluation
journal, January 2008

  • Ufimtsev, Ivan S.; Martínez, Todd J.
  • Journal of Chemical Theory and Computation, Vol. 4, Issue 2
  • DOI: 10.1021/ct700268q

Quantum Chemistry on Graphical Processing Units. 2. Direct Self-Consistent-Field Implementation
journal, March 2009

  • Ufimtsev, Ivan S.; Martinez, Todd J.
  • Journal of Chemical Theory and Computation, Vol. 5, Issue 4
  • DOI: 10.1021/ct800526s

Quantum Chemistry on Graphical Processing Units. 2. Direct Self-Consistent-Field (SCF) Implementation
journal, October 2009

  • Ufimtsev, Ivan S.; Martinez, Todd J.
  • Journal of Chemical Theory and Computation, Vol. 5, Issue 11
  • DOI: 10.1021/ct900433g

Quantum Chemistry on Graphical Processing Units. 3. Analytical Energy Gradients, Geometry Optimization, and First Principles Molecular Dynamics
journal, August 2009

  • Ufimtsev, Ivan S.; Martinez, Todd J.
  • Journal of Chemical Theory and Computation, Vol. 5, Issue 10
  • DOI: 10.1021/ct9003004

Two-electron integral evaluation on the graphics processor unit
journal, January 2007

  • Yasuda, Koji
  • Journal of Computational Chemistry, Vol. 29, Issue 3
  • DOI: 10.1002/jcc.20779

Accelerating Density Functional Calculations with Graphics Processing Unit
journal, July 2008

  • Yasuda, Koji
  • Journal of Chemical Theory and Computation, Vol. 4, Issue 8
  • DOI: 10.1021/ct8001046

Recent developments in the ABINIT software package
journal, August 2016

Density functional theory calculation on many-cores hybrid central processing unit-graphic processing unit architectures
journal, July 2009

  • Genovese, Luigi; Ospici, Matthieu; Deutsch, Thierry
  • The Journal of Chemical Physics, Vol. 131, Issue 3
  • DOI: 10.1063/1.3166140

Graphics Processing Unit acceleration of the Random Phase Approximation in the projector augmented wave method
journal, December 2013

  • Yan, Jun; Li, Lin; O’Grady, Christopher
  • Computer Physics Communications, Vol. 184, Issue 12
  • DOI: 10.1016/j.cpc.2013.07.014

GPU implementation of the linear scaling three dimensional fragment method for large scale electronic structure calculations
journal, February 2017

Real-Space Density Functional Theory on Graphical Processing Units: Computational Approach and Comparison to Gaussian Basis Set Methods
journal, September 2013

  • Andrade, Xavier; Aspuru-Guzik, Alán
  • Journal of Chemical Theory and Computation, Vol. 9, Issue 10
  • DOI: 10.1021/ct400520e

Porting ONETEP to graphical processing unit-based coprocessors. 1. FFT box operations
journal, August 2013

  • Wilkinson, Karl; Skylaris, Chris-Kriton
  • Journal of Computational Chemistry, Vol. 34, Issue 28
  • DOI: 10.1002/jcc.23410

The analysis of a plane wave pseudopotential density functional theory code on a GPU machine
journal, January 2013

Fast plane wave density functional theory molecular dynamics calculations on multi-GPU machines
journal, October 2013

Accelerating Resolution-of-the-Identity Second-Order Møller−Plesset Quantum Chemistry Calculations with Graphical Processing Units †
journal, March 2008

  • Vogt, Leslie; Olivares-Amaya, Roberto; Kermes, Sean
  • The Journal of Physical Chemistry A, Vol. 112, Issue 10
  • DOI: 10.1021/jp0776762

Accelerating Correlated Quantum Chemistry Calculations Using Graphical Processing Units and a Mixed Precision Matrix Multiplication Library
journal, December 2009

  • Olivares-Amaya, Roberto; Watson, Mark A.; Edgar, Richard G.
  • Journal of Chemical Theory and Computation, Vol. 6, Issue 1
  • DOI: 10.1021/ct900543q

Speeding up plane-wave electronic-structure calculations using graphics-processing units
journal, July 2011

  • Maintz, Stefan; Eck, Bernhard; Dronskowski, Richard
  • Computer Physics Communications, Vol. 182, Issue 7
  • DOI: 10.1016/j.cpc.2011.03.010

VASP on a GPU: Application to exact-exchange calculations of the stability of elemental boron
journal, July 2012

Accelerating VASP electronic structure calculations using graphic processing units
journal, August 2012

  • Hacene, Mohamed; Anciaux-Sedrakian, Ani; Rozanska, Xavier
  • Journal of Computational Chemistry, Vol. 33, Issue 32
  • DOI: 10.1002/jcc.23096

Optimizations of the eigensolvers in the ELPA library
journal, July 2019

Towards dense linear algebra for hybrid GPU accelerated manycore systems
journal, June 2010

All-electron formalism for total energy strain derivatives and stress tensor components for numeric atom-centered orbitals
journal, May 2015

Thermodynamic Equilibrium Conditions of Graphene Films on SiC
journal, August 2013

Hybrid functionals for large periodic systems in an all-electron, numeric atom-centered basis framework
journal, July 2015

  • Levchenko, Sergey V.; Ren, Xinguo; Wieferink, Jürgen
  • Computer Physics Communications, Vol. 192
  • DOI: 10.1016/j.cpc.2015.02.021

ELSI: A unified software interface for Kohn–Sham electronic structure solvers
journal, January 2018

  • Yu, Victor Wen-zhe; Corsetti, Fabiano; García, Alberto
  • Computer Physics Communications, Vol. 222
  • DOI: 10.1016/j.cpc.2017.09.007

Reproducibility in density functional theory calculations of solids
journal, March 2016

Accurate localized resolution of identity approach for linear-scaling hybrid density functionals and for many-body perturbation theory
journal, September 2015

GW 100: Benchmarking G 0 W 0 for Molecular Systems
journal, November 2015

  • van Setten, Michiel J.; Caruso, Fabio; Sharifzadeh, Sahar
  • Journal of Chemical Theory and Computation, Vol. 11, Issue 12
  • DOI: 10.1021/acs.jctc.5b00453

Parallel solution of partial symmetric eigenvalue problems from electronic structure calculations
journal, December 2011

Linear scaling electronic structure methods
journal, July 1999

\mathcal{O}(N) methods in electronic structure calculations
journal, February 2012

A multicenter numerical integration scheme for polyatomic molecules
journal, February 1988

  • Becke, A. D.
  • The Journal of Chemical Physics, Vol. 88, Issue 4
  • DOI: 10.1063/1.454033

Generalized Gradient Approximation Made Simple
journal, October 1996

  • Perdew, John P.; Burke, Kieron; Ernzerhof, Matthias
  • Physical Review Letters, Vol. 77, Issue 18, p. 3865-3868
  • DOI: 10.1103/PhysRevLett.77.3865

Fast Calculation of Electrostatics in Crystals and Large Molecules
journal, January 1996

  • Delley, Bernard
  • The Journal of Physical Chemistry, Vol. 100, Issue 15
  • DOI: 10.1021/jp952713n

High order integration schemes on the unit sphere
journal, July 1996

Optimized Slater-type basis sets for the elements 1-118
journal, May 2003

  • Van Lenthe, E.; Baerends, E. J.
  • Journal of Computational Chemistry, Vol. 24, Issue 9
  • DOI: 10.1002/jcc.10255

An algorithm for 3D numerical integration that scales linearly with the size of the molecule
journal, July 1995

NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations
journal, September 2010

  • Valiev, M.; Bylaska, E. J.; Govind, N.
  • Computer Physics Communications, Vol. 181, Issue 9, p. 1477-1489
  • DOI: 10.1016/j.cpc.2010.04.018

Fast Calculation of Electrostatics in Crystals and Large Molecules
journal, January 1996

  • Delley, Bernard
  • The Journal of Physical Chemistry, Vol. 100, Issue 15
  • DOI: 10.1021/jp952713n

Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy
journal, January 2005

  • Weigend, Florian; Ahlrichs, Reinhart
  • Physical Chemistry Chemical Physics, Vol. 7, Issue 18, p. 3297-3305
  • DOI: 10.1039/b508541a

phiGEMM: A CPU-GPU Library for Porting Quantum ESPRESSO on Hybrid Systems
conference, February 2012

  • Spiga, Filippo; Girotto, Ivan
  • 2012 20th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)
  • DOI: 10.1109/pdp.2012.72
    Sort by:
    [ × clear filter / sort ]

    Figures / Tables found in this record:

    Figure 1 (p. 6)figure
    Figure 2 (p. 12)figure
    Figure 3 (p. 15)figure
    Figure 4 (p. 17)figure
    Figure 5 (p. 20)figure
    Figure 6 (p. 23)figure
    Table 1 (p. 27)table
    Table 2 (p. 29)table
    Table 3 (p. 30)table
    Figure 7 (p. 31)figure
    Figure 8 (p. 32)figure
    Table 4 (p. 33)table
    Figure 9 (p. 34)figure
      [ × clear filter / sort ]
      Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

      Similar Records in DOE PAGES and OSTI.GOV collections: