skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Shift-and-invert parallel spectral transformation eigensolver: Massively parallel performance for density-functional based tight-binding

Abstract

The Shift-and-invert parallel spectral transformations (SIPs), a computational approach to solve sparse eigenvalue problems, is developed for massively parallel architectures with exceptional parallel scalability and robustness. The capabilities of SIPs are demonstrated by diagonalization of density-functional based tight-binding (DFTB) Hamiltonian and overlap matrices for single-wall metallic carbon nanotubes, diamond nanowires, and bulk diamond crystals. The largest (smallest) example studied is a 128,000 (2000) atom nanotube for which ~330,000 (~5600) eigenvalues and eigenfunctions are obtained in ~190 (~5) seconds when parallelized over 266,144 (16,384) Blue Gene/Q cores. Weak scaling and strong scaling of SIPs are analyzed and the performance of SIPs is compared with other novel methods. Different matrix ordering methods are investigated to reduce the cost of the factorization step, which dominates the time-to-solution at the strong scaling limit. As a result, a parallel implementation of assembling the density matrix from the distributed eigenvectors is demonstrated.

Authors:
 [1];  [1];  [2];  [1];  [1]
  1. Argonne National Lab. (ANL), Argonne, IL (United States)
  2. Univ. of Alabama, Tuscaloosa, AL (United States)
Publication Date:
Research Org.:
Argonne National Laboratory (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES). Chemical Sciences, Geosciences, and Biosciences Division
OSTI Identifier:
1248169
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Journal of Computational Chemistry
Additional Journal Information:
Journal Volume: 37; Journal Issue: 4; Journal ID: ISSN 0192-8651
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; 36 MATERIALS SCIENCE; 37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY; sparse matrices; spectrum slicing; generalized eigenvalue problem; semi-empirical methods; tight-binding DFT

Citation Formats

Zhang, Hong, Zapol, Peter, Dixon, David A., Wagner, Albert F., and Keceli, Murat. Shift-and-invert parallel spectral transformation eigensolver: Massively parallel performance for density-functional based tight-binding. United States: N. p., 2015. Web. doi:10.1002/jcc.24254.
Zhang, Hong, Zapol, Peter, Dixon, David A., Wagner, Albert F., & Keceli, Murat. Shift-and-invert parallel spectral transformation eigensolver: Massively parallel performance for density-functional based tight-binding. United States. https://doi.org/10.1002/jcc.24254
Zhang, Hong, Zapol, Peter, Dixon, David A., Wagner, Albert F., and Keceli, Murat. 2015. "Shift-and-invert parallel spectral transformation eigensolver: Massively parallel performance for density-functional based tight-binding". United States. https://doi.org/10.1002/jcc.24254. https://www.osti.gov/servlets/purl/1248169.
@article{osti_1248169,
title = {Shift-and-invert parallel spectral transformation eigensolver: Massively parallel performance for density-functional based tight-binding},
author = {Zhang, Hong and Zapol, Peter and Dixon, David A. and Wagner, Albert F. and Keceli, Murat},
abstractNote = {The Shift-and-invert parallel spectral transformations (SIPs), a computational approach to solve sparse eigenvalue problems, is developed for massively parallel architectures with exceptional parallel scalability and robustness. The capabilities of SIPs are demonstrated by diagonalization of density-functional based tight-binding (DFTB) Hamiltonian and overlap matrices for single-wall metallic carbon nanotubes, diamond nanowires, and bulk diamond crystals. The largest (smallest) example studied is a 128,000 (2000) atom nanotube for which ~330,000 (~5600) eigenvalues and eigenfunctions are obtained in ~190 (~5) seconds when parallelized over 266,144 (16,384) Blue Gene/Q cores. Weak scaling and strong scaling of SIPs are analyzed and the performance of SIPs is compared with other novel methods. Different matrix ordering methods are investigated to reduce the cost of the factorization step, which dominates the time-to-solution at the strong scaling limit. As a result, a parallel implementation of assembling the density matrix from the distributed eigenvectors is demonstrated.},
doi = {10.1002/jcc.24254},
url = {https://www.osti.gov/biblio/1248169}, journal = {Journal of Computational Chemistry},
issn = {0192-8651},
number = 4,
volume = 37,
place = {United States},
year = {Tue Nov 17 00:00:00 EST 2015},
month = {Tue Nov 17 00:00:00 EST 2015}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 13 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Simplified LCAO Method for the Periodic Potential Problem
journal, June 1954


Self-consistent-charge density-functional tight-binding method for simulations of complex materials properties
journal, September 1998


Tight-Binding Density Functional Theory:  An Approximate Kohn−Sham DFT Scheme
journal, July 2007


Approximate Self‐Consistent Molecular Orbital Theory. I. Invariant Procedures
journal, November 1965


Ground states of molecules. 38. The MNDO method. Approximations and parameters
journal, June 1977


Optimization of parameters for semiempirical methods I. Method
journal, March 1989


Semiempirical quantum–chemical methods
journal, July 2013


Inhomogeneous Electron Gas
journal, November 1964


Self-Consistent Equations Including Exchange and Correlation Effects
journal, November 1965


Assessment of Gaussian-3 and Density Functional Theories for Enthalpies of Formation of C 1 −C 16 Alkanes
journal, June 2000


Triplet states via intermediate neglect of differential overlap: Benzene, pyridine and the diazines
journal, January 1976


Semiempirical Molecular Orbital Methods
book, January 2007


SIPs: Shift-and-invert parallel spectral transformations
journal, June 2007


Strategies for spectrum slicing based on restarted Lanczos methods
journal, March 2012


The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and computational science
journal, May 2014


New algorithms for iterative matrix-free eigensolvers in quantum chemistry
journal, December 2014


Density functional tight binding
journal, March 2014

  • Elstner, Marcus; Seifert, Gotthard
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 372, Issue 2011
  • https://doi.org/10.1098/rsta.2012.0483

Direct calculation of electron density in density-functional theory
journal, March 1991


What is the best alternative to diagonalization of the Hamiltonian in large scale semiempirical calculations?
journal, January 1999


SIESTA-PEXSI: massively parallel method for efficient and accurate ab initio materials simulation without matrix diagonalization
journal, July 2014


An iteration method for the solution of the eigenvalue problem of linear differential and integral operators
journal, October 1950


A Jacobi--Davidson Iteration Method for Linear Eigenvalue Problems
journal, January 2000


SLEPc: A scalable and flexible toolkit for the solution of eigenvalue problems
journal, September 2005


A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling
journal, January 2001


Hybrid scheduling for the parallel solution of linear systems
journal, February 2006


A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs
journal, January 1998


A Parallel Algorithm for Multilevel Graph Partitioning and Sparse Matrix Ordering
journal, January 1998


PT-Scotch: A tool for efficient parallel graph ordering
journal, July 2008


DFTB+, a Sparse Matrix-Based Implementation of the DFTB Method
journal, July 2007


Computing the Minimum Fill-In is NP-Complete
journal, March 1981


Reducing the bandwidth of sparse symmetric matrices
conference, January 1969


Reading Althusser
journal, April 1971


Sparse Matrices in MATLAB: Design and Implementation
journal, January 1992


Nested Dissection of a Regular Finite Element Mesh
journal, April 1973


A Sparse Self-Consistent Field Algorithm and Its Parallel Implementation: Application to Density-Functional-Based Tight Binding
journal, May 2014


Parallel Computing in Quantum Chemistry
reference-book, January 2008


High-Performance Solvers for Dense Hermitian Eigenproblems
journal, January 2013


Elemental: A New Framework for Distributed Memory Dense Matrix Computations
journal, February 2013


Performance of Dense Eigensolvers on BlueGene/Q
book, January 2014


ScaLAPACK: a portable linear algebra library for distributed memory computers — design issues and performance
journal, August 1996


Fast semiempirical calculations: Fast Semiempirical Calculations
journal, June 1982


Calculations for millions of atoms with density functional theory: linear scaling shows its potential
journal, February 2010


Kohn−Sham Density Functional Theory Electronic Structure Calculations with Linearly Scaling Computational Time and Memory Usage
journal, December 2010


Linear Scaling Self-Consistent Field Calculations with Millions of Atoms in the Condensed Phase
journal, March 2012


Sparse Projected-Gradient Method As a Linear-Scaling Low-Memory Alternative to Diagonalization in Self-Consistent Field Electronic Structure Calculations
journal, December 2012


Systematic sparse matrix error control for linear scaling electronic structure calculations
journal, January 2005


Numerical Methods for Electronic Structure Calculations of Materials
journal, January 2010


Pole-Based approximation of the Fermi-Dirac function
journal, August 2009


SelInv---An Algorithm for Selected Inversion of a Sparse Symmetric Matrix
journal, February 2011


A Fast Parallel Algorithm for Selected Inversion of Structured Sparse Matrices with Application to 2D Electronic Structure Calculations
journal, January 2011


Accelerating atomic orbital-based electronic structure calculation via pole expansion and selected inversion
journal, June 2013


A spectrum slicing method for the Kohn–Sham problem
journal, March 2012


Improving the scalability of a symmetric iterative eigensolver for multi-core platforms: IMPROVING THE SCALABILITY OF A SYMMETRIC ITERATIVE EIGENSOLVER
journal, September 2013


Works referencing / citing this record:

Three-body expansion of the fragment molecular orbital method combined with density-functional tight-binding
journal, January 2017


SIESTA-SIPs: Massively parallel spectrum-slicing eigensolver for an ab initio molecular dynamics package
journal, August 2018


On the achievement of high fidelity and scalability for large-scale diagonalizations in grid-based DFT simulations
journal, March 2018


Recent advances in quantum‐mechanical molecular dynamics simulations of proton transfer mechanism in various water‐based environments
journal, May 2019