## Task-based Parallel Computation of the Density Matrix in Quantum-based Molecular Dynamics using Graph Partitioning

## Abstract

Quantum-based molecular dynamics (QMD) is a highly accurate and transferable method for material science simulations. However, the time scales and system sizes accessible to QMD are typically limited to picoseconds and a few hundred atoms. These constraints arise due to expensive self-consistent ground-state electronic structure calculations that can often scale cubically with the number of atoms. Linearly scaling methods depend on computing the density matrix P from the Hamiltonian matrix H by exploiting the sparsity in both matrices. The second-order spectral projection (SP2) algorithm is an O(N) algorithm that computes P with a sequence of 40 - 50 matrix-matrix multiplications. In this paper, we present task-based implementations of a recently developed data-parallel graph-based approach to the SP2 algorithm, G-SP2. We represent the density matrix P as an undirected graph and use graph partitioning techniques to divide the computation into smaller independent tasks. The partitions thus obtained are generally not of equal size and give rise to undesirable load imbalances in standard MPI-based implementations. This load-balancing challenge can be mitigated by dynamically scheduling parallel computations at runtime using task-based programming models. We develop task-based implementations of the data-parallel G-SP2 algorithm using both Intel's Concurrent Collections (CnC) as well as the Charm++more »

- Authors:

- Univ. of Illinois, Urbana-Champaign, IL (United States)
- Univ. of Missouri, Columbia, MO (United States)
- Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
- Univ. of Delaware, Newark, DE (United States)
- Univ. of Iowa, Iowa City, IA (United States)
- Univ. of Georgia, Athens, GA (United States)
- Columbia Univ., New York, NY (United States)

- Publication Date:

- Research Org.:
- Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

- Sponsoring Org.:
- USDOE Office of Science (SC). Advanced Scientific Computing Research (ASCR) (SC-21)

- OSTI Identifier:
- 1501801

- Report Number(s):
- LA-UR-16-24908

Journal ID: ISSN 1064-8275

- Grant/Contract Number:
- 89233218CNA000001

- Resource Type:
- Accepted Manuscript

- Journal Name:
- SIAM Journal on Scientific Computing

- Additional Journal Information:
- Journal Volume: 39; Journal Issue: 6; Journal ID: ISSN 1064-8275

- Publisher:
- SIAM

- Country of Publication:
- United States

- Language:
- English

- Subject:
- 97 MATHEMATICS AND COMPUTING; Computer Science; Mathematics

### Citation Formats

```
Ghale, Purnima, Kroonblawd, Matthew P., Mniszewski, Sue, Negre, Christian F. A., Pavel, Robert, Pino, Sergio, Sardeshmukh, Vivek, Shi, Guangjie, and Hahn, Georg. Task-based Parallel Computation of the Density Matrix in Quantum-based Molecular Dynamics using Graph Partitioning. United States: N. p., 2017.
Web. doi:10.1137/16M109404X.
```

```
Ghale, Purnima, Kroonblawd, Matthew P., Mniszewski, Sue, Negre, Christian F. A., Pavel, Robert, Pino, Sergio, Sardeshmukh, Vivek, Shi, Guangjie, & Hahn, Georg. Task-based Parallel Computation of the Density Matrix in Quantum-based Molecular Dynamics using Graph Partitioning. United States. doi:10.1137/16M109404X.
```

```
Ghale, Purnima, Kroonblawd, Matthew P., Mniszewski, Sue, Negre, Christian F. A., Pavel, Robert, Pino, Sergio, Sardeshmukh, Vivek, Shi, Guangjie, and Hahn, Georg. Thu .
"Task-based Parallel Computation of the Density Matrix in Quantum-based Molecular Dynamics using Graph Partitioning". United States. doi:10.1137/16M109404X. https://www.osti.gov/servlets/purl/1501801.
```

```
@article{osti_1501801,
```

title = {Task-based Parallel Computation of the Density Matrix in Quantum-based Molecular Dynamics using Graph Partitioning},

author = {Ghale, Purnima and Kroonblawd, Matthew P. and Mniszewski, Sue and Negre, Christian F. A. and Pavel, Robert and Pino, Sergio and Sardeshmukh, Vivek and Shi, Guangjie and Hahn, Georg},

abstractNote = {Quantum-based molecular dynamics (QMD) is a highly accurate and transferable method for material science simulations. However, the time scales and system sizes accessible to QMD are typically limited to picoseconds and a few hundred atoms. These constraints arise due to expensive self-consistent ground-state electronic structure calculations that can often scale cubically with the number of atoms. Linearly scaling methods depend on computing the density matrix P from the Hamiltonian matrix H by exploiting the sparsity in both matrices. The second-order spectral projection (SP2) algorithm is an O(N) algorithm that computes P with a sequence of 40 - 50 matrix-matrix multiplications. In this paper, we present task-based implementations of a recently developed data-parallel graph-based approach to the SP2 algorithm, G-SP2. We represent the density matrix P as an undirected graph and use graph partitioning techniques to divide the computation into smaller independent tasks. The partitions thus obtained are generally not of equal size and give rise to undesirable load imbalances in standard MPI-based implementations. This load-balancing challenge can be mitigated by dynamically scheduling parallel computations at runtime using task-based programming models. We develop task-based implementations of the data-parallel G-SP2 algorithm using both Intel's Concurrent Collections (CnC) as well as the Charm++ programming model and evaluate these implementations for future use. Scaling and performance results of our implementations are investigated for representative segments of QMD simulations for solvated protein systems containing more than 10,000 atoms.},

doi = {10.1137/16M109404X},

journal = {SIAM Journal on Scientific Computing},

number = 6,

volume = 39,

place = {United States},

year = {2017},

month = {12}

}