Solvers for $\mathcal{O} (N)$ Electronic Structure in the Strong Scaling Limit

Bock, Nicolas; Challacombe, William M.; Kale, Laxmikant

doi:10.1137/140974602

Title: Solvers for $$\mathcal{O} (N)$$ Electronic Structure in the Strong Scaling Limit

Journal Article · Tue Jan 26 00:00:00 EST 2016 · SIAM Journal on Scientific Computing

DOI:https://doi.org/10.1137/140974602· OSTI ID:1329852

Bock, Nicolas ^[1]; Challacombe, William M. ^[1]; Kale, Laxmikant ^[2]

Los Alamos National Lab. (LANL), Los Alamos, NM (United States). Theoretical Division
Univ. of Illinois, Urbana-Champaign, IL (United States). Parallel Programming Lab.

Here we present a hybrid OpenMP/Charm\tt++ framework for solving the $$\mathcal{O} (N)$$ self-consistent-field eigenvalue problem with parallelism in the strong scaling regime, $$P\gg{N}$$, where $$P$$ is the number of cores, and $$N$$ is a measure of system size, i.e., the number of matrix rows/columns, basis functions, atoms, molecules, etc. This result is achieved with a nested approach to spectral projection and the sparse approximate matrix multiply [Bock and Challacombe, SIAM J. Sci. Comput., 35 (2013), pp. C72--C98], and involves a recursive, task-parallel algorithm, often employed by generalized $$N$$-Body solvers, to occlusion and culling of negligible products in the case of matrices with decay. Lastly, employing classic technologies associated with generalized $$N$$-Body solvers, including overdecomposition, recursive task parallelism, orderings that preserve locality, and persistence-based load balancing, we obtain scaling beyond hundreds of cores per molecule for small water clusters ([H$${}_2$$O]$${}_N$$, $$N \in \{ 30, 90, 150 \}$$, $$P/N \approx \{ 819, 273, 164 \}$$) and find support for an increasingly strong scalability with increasing system size $$N$$.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); USDOE Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: AC52-06NA25396; 20110230ER

OSTI ID:: 1329852

Report Number(s):: LA-UR-14-22050

Journal Information:: SIAM Journal on Scientific Computing, Vol. 38, Issue 1; ISSN 1064-8275

Publisher:: SIAMCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 7 works

Citation information provided by
Web of Science

References (50)

Decay Properties of Spectral Projectors with Applications to Electronic Structure Benzi, Michele; Boito, Paola; Razouk, Nader SIAM Review, Vol. 55, Issue 1 https://doi.org/10.1137/100814019	journal	January 2013
Bounds for the Entries of Matrix Functions with Applications to Preconditioning Benzi, Michele; Golub, Gene H. Bit Numerical Mathematics, Vol. 39, Issue 3, p. 417-438 https://doi.org/10.1023/A:1022362401426	journal	January 1999
Orderings for Factorized Sparse Approximate Inverse Preconditioners Benzi, Michele; Tuma, Miroslav SIAM Journal on Scientific Computing, Vol. 21, Issue 5 https://doi.org/10.1137/S1064827598339372	journal	January 2000
Stability of fast algorithms for matrix multiplication Bini, Dario; Lotti, Grazia Numerische Mathematik, Vol. 36, Issue 1 https://doi.org/10.1007/BF01395989	journal	March 1980
An Optimized Sparse Approximate Matrix Multiply for Matrices with Decay Bock, Nicolas; Challacombe, Matt SIAM Journal on Scientific Computing, Vol. 35, Issue 1 https://doi.org/10.1137/120870761	journal	January 2013
Sparse matrix multiplication: The distributed block-compressed sparse row library Borštnik, Urban; VandeVondele, Joost; Weber, Valéry Parallel Computing, Vol. 40, Issue 5-6 https://doi.org/10.1016/j.parco.2014.03.012	journal	May 2014
Calculations for millions of atoms with density functional theory: linear scaling shows its potential Bowler, D. R.; Miyazaki, T. Journal of Physics: Condensed Matter, Vol. 22, Issue 7 https://doi.org/10.1088/0953-8984/22/7/074207	journal	February 2010
\mathcal{O}(N) methods in electronic structure calculations Bowler, D. R.; Miyazaki, T. Reports on Progress in Physics, Vol. 75, Issue 3 https://doi.org/10.1088/0034-4885/75/3/036503	journal	February 2012
Recent progress in linear scaling ab initio electronic structure techniques Bowler, D. R.; Miyazaki, T.; Gillan, M. J. Journal of Physics: Condensed Matter, Vol. 14, Issue 11 https://doi.org/10.1088/0953-8984/14/11/303	journal	March 2002
Automatic data distribution and load balancing with space-filling curves: implementation in CONQUEST Brázdová, V.; Bowler, D. R. Journal of Physics: Condensed Matter, Vol. 20, Issue 27 https://doi.org/10.1088/0953-8984/20/27/275223	journal	June 2008
A linear scaling method for Hartree–Fock exchange calculations of large molecules Burant, John C.; Scuseria, Gustavo E.; Frisch, Michael J. The Journal of Chemical Physics, Vol. 105, Issue 19 https://doi.org/10.1063/1.472627	journal	November 1996
Hard scaling challenges for ab initio molecular dynamics capabilities in NWChem: Using 100,000 CPUs per second Bylaska, Eric J.; Glass, Kevin; Baxter, Doug Journal of Physics: Conference Series, Vol. 180 https://doi.org/10.1088/1742-6596/180/1/012028	journal	July 2009
A simplified density matrix minimization for linear scaling self-consistent field theory Challacombe, Matt The Journal of Chemical Physics, Vol. 110, Issue 5 https://doi.org/10.1063/1.477969	journal	February 1999
Linear scaling computation of the Fock matrix. V. Hierarchical Cubature for numerical integration of the exchange-correlation matrix Challacombe, Matt The Journal of Chemical Physics, Vol. 113, Issue 22 https://doi.org/10.1063/1.1316012	journal	December 2000
Linear Scaling Solution of the Time-Dependent Self-Consistent-Field Equations Challacombe, Matt Computation, Vol. 2, Issue 1 https://doi.org/10.3390/computation2010001	journal	March 2014
Linear scaling computation of the Fock matrix Challacombe, Matt; Schwegler, Eric The Journal of Chemical Physics, Vol. 106, Issue 13 https://doi.org/10.1063/1.473575	journal	April 1997
Fast assembly of the Coulomb matrix: A quantum chemical tree code Challacombe, Matt; Schwegler, Eric; Almlöf, Jan The Journal of Chemical Physics, Vol. 104, Issue 12 https://doi.org/10.1063/1.471163	journal	March 1996
Parallel scalability of Hartree–Fock calculations Chow, Edmond; Liu, Xing; Smelyanskiy, Mikhail The Journal of Chemical Physics, Vol. 142, Issue 10 https://doi.org/10.1063/1.4913961	journal	March 2015
Semiempirical methods with conjugate gradient density matrix search to replace diagonalization for molecular systems containing thousands of atoms Daniels, Andrew D.; Millam, John M.; Scuseria, Gustavo E. The Journal of Chemical Physics, Vol. 107, Issue 2 https://doi.org/10.1063/1.474404	journal	July 1997
Model for energetics of solids based on the density matrix Daw, Murray S. Physical Review B, Vol. 47, Issue 16 https://doi.org/10.1103/PhysRevB.47.10895	journal	April 1993
Decay rates for inverses of band matrices Demko, Stephen; Moss, William F.; Smith, Philip W. Mathematics of Computation, Vol. 43, Issue 168 https://doi.org/10.1090/S0025-5718-1984-0758197-9	journal	January 1984
Auto-blocking matrix-multiplication or tracking BLAS3 performance from source code Frens, Jeremy D.; Wise, David S. ACM SIGPLAN Notices, Vol. 32, Issue 7 https://doi.org/10.1145/263767.263789	journal	July 1997
QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials Giannozzi, Paolo; Baroni, Stefano; Bonini, Nicola Journal of Physics: Condensed Matter, Vol. 21, Issue 39, Article No. 395502 https://doi.org/10.1088/0953-8984/21/39/395502	journal	September 2009
Linear-scaling DFT-pseudopotential calculations on parallel computers Goringe, C. M.; Hernández, E.; Gillan, M. J. Computer Physics Communications, Vol. 102, Issue 1-3 https://doi.org/10.1016/S0010-4655(97)00029-5	journal	May 1997
The GAMESS-UK electronic structure package: algorithms, developments and applications Guest *, Martyn F.; Bush, Ian J.; Van Dam, Huub J. J. Molecular Physics, Vol. 103, Issue 6-8 https://doi.org/10.1080/00268970512331340592	journal	March 2005
Two Fast Algorithms for Sparse Matrices: Multiplication and Permuted Transposition Gustavson, Fred G. ACM Transactions on Mathematical Software, Vol. 4, Issue 3 https://doi.org/10.1145/355791.355796	journal	September 1978
Ab-initio simulations of materials using VASP: Density-functional theory and beyond Hafner, Jürgen Journal of Computational Chemistry, Vol. 29, Issue 13 https://doi.org/10.1002/jcc.21057	journal	October 2008
The Do-It-Yourself Supercomputer Hargrove, William W.; Hoffman, Forrest M.; Sterling, Thomas Scientific American, Vol. 285, Issue 2 https://doi.org/10.1038/scientificamerican0801-72	journal	August 2001
Performance evaluation of ultra-large-scale first-principles electronic structure calculation code on the K computer Hasegawa, Yukihiro; Iwata, Jun-Ichi; Tsuji, Miwako The International Journal of High Performance Computing Applications, Vol. 28, Issue 3 https://doi.org/10.1177/1094342013508163	journal	October 2013
High precision valence bond potential curve for the Cl2 molecule Heil, Timothy G.; O'Neil, Stephen V.; Schaefer, Henry F. Chemical Physics Letters, Vol. 5, Issue 5 https://doi.org/10.1016/0009-2614(70)85132-6	journal	April 1970
Inhomogeneous Electron Gas Hohenberg, P.; Kohn, W. Physical Review, Vol. 136, Issue 3B, p. B864-B871 https://doi.org/10.1103/PhysRev.136.B864	journal	November 1964
Iterative spatial join Jacox, Edwin H.; Samet, Hanan ACM Transactions on Database Systems, Vol. 28, Issue 3 https://doi.org/10.1145/937598.937600	journal	September 2003
Self-Consistent Equations Including Exchange and Correlation Effects Kohn, W.; Sham, L. J. Physical Review, Vol. 140, Issue 4A, p. A1133-A1138 https://doi.org/10.1103/PhysRev.140.A1133	journal	November 1965
Density-matrix electronic-structure method with linear system-size scaling Li, X. -P.; Nunes, R. W.; Vanderbilt, David Physical Review B, Vol. 47, Issue 16 https://doi.org/10.1103/PhysRevB.47.10891	journal	April 1993
GRAPE‐4: A Massively Parallel Special‐Purpose Computer for Collisional N ‐Body Simulations Makino, Junichiro; Taiji, Makoto; Ebisuzaki, Toshikazu The Astrophysical Journal, Vol. 480, Issue 1 https://doi.org/10.1086/303972	journal	May 1997
The density matrix in self-consistent field theory I. Iterative construction of the density matrix McWeeny, R. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, Vol. 235, Issue 1203, p. 496-509 https://doi.org/10.1098/rspa.1956.0100	journal	June 1956
Linear scaling conjugate gradient density matrix search as an alternative to diagonalization for first principles electronic structure calculations Millam, John M.; Scuseria, Gustavo E. The Journal of Chemical Physics, Vol. 106, Issue 13 https://doi.org/10.1063/1.473579	journal	April 1997
Join processing in relational databases Mishra, Priti; Eich, Margaret H. ACM Computing Surveys, Vol. 24, Issue 1 https://doi.org/10.1145/128762.128764	journal	March 1992
Trace resetting density matrix purification in O(N) self-consistent-field theory Niklasson, Anders M. N.; Tymczak, C. J.; Challacombe, Matt The Journal of Chemical Physics, Vol. 118, Issue 19 https://doi.org/10.1063/1.1559913	journal	May 2003
Linear and sublinear scaling formation of Hartree–Fock-type exchange matrices Ochsenfeld, Christian; White, Christopher A.; Head-Gordon, Martin The Journal of Chemical Physics, Vol. 109, Issue 5 https://doi.org/10.1063/1.476741	journal	August 1998
A comparative performance study of common and popular task-centric programming frameworks: PERFORMANCE IN TASK-CENTRIC PROGRAMMING FRAMEWORKS Podobas, Artur; Brorsson, Mats; Faxén, Karl-Filip Concurrency and Computation: Practice and Experience, Vol. 27, Issue 1 https://doi.org/10.1002/cpe.3186	journal	December 2013
Linear scaling computation of the Fock matrix. II. Rigorous bounds on exchange integrals and incremental Fock build Schwegler, Eric; Challacombe, Matt; Head-Gordon, Martin The Journal of Chemical Physics, Vol. 106, Issue 23 https://doi.org/10.1063/1.473833	journal	June 1997
Introducing ONETEP : Linear-scaling density functional simulations on parallel computers Skylaris, Chris-Kriton; Haynes, Peter D.; Mostofi, Arash A. The Journal of Chemical Physics, Vol. 122, Issue 8 https://doi.org/10.1063/1.1839852	journal	February 2005
NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations Valiev, M.; Bylaska, E. J.; Govind, N. Computer Physics Communications, Vol. 181, Issue 9, p. 1477-1489 https://doi.org/10.1016/j.cpc.2010.04.018	journal	September 2010
SUMMA: scalable universal matrix multiplication algorithm Van De Geijn, R. A.; Watts, J. Concurrency: Practice and Experience, Vol. 9, Issue 4 https://doi.org/10.1002/(SICI)1096-9128(199704)9:4<255::AID-CPE250>3.0.CO;2-2	journal	April 1997
Linear Scaling Self-Consistent Field Calculations with Millions of Atoms in the Condensed Phase VandeVondele, Joost; Borštnik, Urban; Hutter, Jürg Journal of Chemical Theory and Computation, Vol. 8, Issue 10 https://doi.org/10.1021/ct200897x	journal	March 2012
Representing matrices as quadtrees for parallel processors: extended abstract Wise, David S. ACM SIGSAM Bulletin, Vol. 18, Issue 3 https://doi.org/10.1145/1089389.1089398	journal	August 1984
Costs of quadtree representation of nondense matrices Wise, David S.; Franco, John Journal of Parallel and Distributed Computing, Vol. 9, Issue 3 https://doi.org/10.1016/0743-7315(90)90079-5	journal	July 1990
Language support for Morton-order matrices Wise, David S.; Frens, Jeremy D.; Gu, Yuhong ACM SIGPLAN Notices, Vol. 36, Issue 7 https://doi.org/10.1145/568014.379559	journal	July 2001
Mature HIV-1 capsid structure by cryo-electron microscopy and all-atom molecular dynamics Zhao, Gongpu; Perilla, Juan R.; Yufenyuy, Ernest L. Nature, Vol. 497, Issue 7451 https://doi.org/10.1038/nature12162	journal	May 2013

Cited By (2)

Graph-based linear scaling electronic structure theory Niklasson, Anders M. N.; Mniszewski, Susan M.; Negre, Christian F. A. The Journal of Chemical Physics, Vol. 144, Issue 23 https://doi.org/10.1063/1.4952650	journal	June 2016
Graph-based linear scaling electronic structure theory Niklasson, Anders M. N.; Mniszewski, Susan M.; Negre, Christian F. A. arXiv https://doi.org/10.48550/arxiv.1603.00937	text	January 2016

Similar Records

Symmetric Indefinite Linear Solver Using OpenMP Task on Multicore Architectures

Journal Article · Wed Aug 01 00:00:00 EDT 2018 · IEEE Transactions on Parallel and Distributed Systems · OSTI ID:1329852

Yamazaki, Ichitaro; Kurzak, Jakub; Wu, Panruo; +2 more

Deploy threading in Nalu solver stack

Technical Report · Mon Oct 01 00:00:00 EDT 2018 · OSTI ID:1329852

Prokopenko, Andrey; Thomas, Stephen; Swirydowicz, Kasia; +4 more

Hybrid programming-model strategies for GPU offloading of electronic structure calculation kernels

Journal Article · Fri Mar 29 00:00:00 EDT 2024 · Journal of Chemical Physics · OSTI ID:1329852

Fattebert, Jean-Luc; Negre, Christian A.; Finkelstein, Joshua; +6 more

Related Subjects

97 MATHEMATICS AND COMPUTING
sparse approximate matrix multiply
sparse linear algebra
SpAMM
reduced complexity algorithm
linear scaling
quantum chemistry
spectral projection
N-Body
Charm++
matrices with decay
parallel irregular
space filling curve
persistence load balancing
overdecomposition

Title: Solvers for $$\mathcal{O} (N)$$ Electronic Structure in the Strong Scaling Limit

Citation Formats

References (50)

Cited By (2)

Similar Records

Related Subjects