skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: PSelInv—A Distributed Memory Parallel Algorithm for Selected Inversion: The Symmetric Case

Abstract

We describe an efficient parallel implementation of the selected inversion algorithm for distributed memory computer systems, which we call PSelInv. The PSelInv method computes selected elements of a general sparse matrix A that can be decomposed as A = LU, where L is lower triangular and U is upper triangular. The implementation described in this article focuses on the case of sparse symmetric matrices. It contains an interface that is compatible with the distributed memory parallel sparse direct factorization SuperLU_DIST. However, the underlying data structure and design of PSelInv allows it to be easily combined with other factorization routines, such as PARDISO. We discuss general parallelization strategies such as data and task distribution schemes. In particular, we describe how to exploit the concurrency exposed by the elimination tree associated with the LU factorization of A. We demonstrate the efficiency and accuracy of PSelInv by presenting several numerical experiments. In particular, we show that PSelInv can run efficiently on more than 4,000 cores for a modestly sized matrix. We also demonstrate how PSelInv can be used to accelerate large-scale electronic structure calculations.

Authors:
 [1];  [2];  [1]
  1. Lawrence Berkeley National Laboratory, Berkeley CA USA
  2. University of California, Berkeley and Lawrence Berkeley National Laboratory, Berkeley CA
Publication Date:
Research Org.:
Lawrence Berkeley National Laboratory-National Energy Research Scientific Computing Center
Sponsoring Org.:
USDOE
OSTI Identifier:
1524063
DOE Contract Number:  
AC02-05CH11231
Resource Type:
Journal Article
Journal Name:
ACM Transactions on Mathematical Software
Additional Journal Information:
Journal Volume: 43; Journal Issue: 3; Journal ID: ISSN 0098-3500
Country of Publication:
United States
Language:
English

Citation Formats

Jacquelin, Mathias, Lin, Lin, and Yang, Chao. PSelInv—A Distributed Memory Parallel Algorithm for Selected Inversion: The Symmetric Case. United States: N. p., 2016. Web. doi:10.1145/2786977.
Jacquelin, Mathias, Lin, Lin, & Yang, Chao. PSelInv—A Distributed Memory Parallel Algorithm for Selected Inversion: The Symmetric Case. United States. doi:10.1145/2786977.
Jacquelin, Mathias, Lin, Lin, and Yang, Chao. Wed . "PSelInv—A Distributed Memory Parallel Algorithm for Selected Inversion: The Symmetric Case". United States. doi:10.1145/2786977.
@article{osti_1524063,
title = {PSelInv—A Distributed Memory Parallel Algorithm for Selected Inversion: The Symmetric Case},
author = {Jacquelin, Mathias and Lin, Lin and Yang, Chao},
abstractNote = {We describe an efficient parallel implementation of the selected inversion algorithm for distributed memory computer systems, which we call PSelInv. The PSelInv method computes selected elements of a general sparse matrix A that can be decomposed as A = LU, where L is lower triangular and U is upper triangular. The implementation described in this article focuses on the case of sparse symmetric matrices. It contains an interface that is compatible with the distributed memory parallel sparse direct factorization SuperLU_DIST. However, the underlying data structure and design of PSelInv allows it to be easily combined with other factorization routines, such as PARDISO. We discuss general parallelization strategies such as data and task distribution schemes. In particular, we describe how to exploit the concurrency exposed by the elimination tree associated with the LU factorization of A. We demonstrate the efficiency and accuracy of PSelInv by presenting several numerical experiments. In particular, we show that PSelInv can run efficiently on more than 4,000 cores for a modestly sized matrix. We also demonstrate how PSelInv can be used to accelerate large-scale electronic structure calculations.},
doi = {10.1145/2786977},
journal = {ACM Transactions on Mathematical Software},
issn = {0098-3500},
number = 3,
volume = 43,
place = {United States},
year = {2016},
month = {12}
}

Works referenced in this record:

On Computing Inverse Entries of a Sparse Matrix in an Out-of-Core Environment
journal, January 2012

  • Amestoy, Patrick R.; Duff, Iain S.; L'Excellent, Jean-Yves
  • SIAM Journal on Scientific Computing, Vol. 34, Issue 4
  • DOI: 10.1137/100799411

On computing certain elements of the inverse of a sparse matrix
journal, March 1975


Accelerating atomic orbital-based electronic structure calculation via pole expansion and selected inversion
journal, June 2013


The influence of relaxed supernode partitions on the multifrontal method
journal, December 1989

  • Ashcraft, Cleve; Grimes, Roger
  • ACM Transactions on Mathematical Software, Vol. 15, Issue 4
  • DOI: 10.1145/76909.76910

SelInv---An Algorithm for Selected Inversion of a Sparse Symmetric Matrix
journal, February 2011

  • Lin, Lin; Yang, Chao; Meza, Juan C.
  • ACM Transactions on Mathematical Software, Vol. 37, Issue 4
  • DOI: 10.1145/1916461.1916464

ScaLAPACK's MRRR algorithm
journal, January 2010


A Fast Parallel Algorithm for Selected Inversion of Structured Sparse Matrices with Application to 2D Electronic Structure Calculations
journal, January 2011

  • Lin, Lin; Yang, Chao; Lu, Jianfeng
  • SIAM Journal on Scientific Computing, Vol. 33, Issue 3
  • DOI: 10.1137/09077432X

A hybrid method for the parallel computation of Green’s functions
journal, August 2009

  • Petersen, Dan Erik; Li, Song; Stokbro, Kurt
  • Journal of Computational Physics, Vol. 228, Issue 14
  • DOI: 10.1016/j.jcp.2009.03.035

SuperLU_DIST: A scalable distributed-memory sparse direct solver for unsymmetric linear systems
journal, June 2003

  • Li, Xiaoye S.; Demmel, James W.
  • ACM Transactions on Mathematical Software, Vol. 29, Issue 2
  • DOI: 10.1145/779359.779361

Pole-Based approximation of the Fermi-Dirac function
journal, August 2009

  • Lin, Lin; Lu, Jianfeng; Ying, Lexing
  • Chinese Annals of Mathematics, Series B, Vol. 30, Issue 6
  • DOI: 10.1007/s11401-009-0201-7

A Parallel Algorithm for Multilevel Graph Partitioning and Sparse Matrix Ordering
journal, January 1998

  • Karypis, George; Kumar, Vipin
  • Journal of Parallel and Distributed Computing, Vol. 48, Issue 1
  • DOI: 10.1006/jpdc.1997.1403

The SIESTA method for ab initio order- N materials simulation
journal, March 2002

  • Soler, José M.; Artacho, Emilio; Gale, Julian D.
  • Journal of Physics: Condensed Matter, Vol. 14, Issue 11
  • DOI: 10.1088/0953-8984/14/11/302

Some stable methods for calculating inertia and solving symmetric linear systems
journal, January 1977


A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling
journal, January 2001

  • Amestoy, Patrick R.; Duff, Iain S.; L'Excellent, Jean-Yves
  • SIAM Journal on Matrix Analysis and Applications, Vol. 23, Issue 1
  • DOI: 10.1137/S0895479899358194

The university of Florida sparse matrix collection
journal, November 2011

  • Davis, Timothy A.; Hu, Yifan
  • ACM Transactions on Mathematical Software, Vol. 38, Issue 1
  • DOI: 10.1145/2049662.2049663

A probing method for computing the diagonal of a matrix inverse
journal, April 2011

  • Tang, Jok M.; Saad, Yousef
  • Numerical Linear Algebra with Applications, Vol. 19, Issue 3
  • DOI: 10.1002/nla.779

A two-dimensional domain decomposition technique for the simulation of quantum-scale devices
journal, February 2012

  • Cauley, Stephen; Balakrishnan, Venkataramanan; Klimeck, Gerhard
  • Journal of Computational Physics, Vol. 231, Issue 4
  • DOI: 10.1016/j.jcp.2011.10.006

Inhomogeneous Electron Gas
journal, November 1964


An iteration method for the solution of the eigenvalue problem of linear differential and integral operators
journal, October 1950

  • Lanczos, C.
  • Journal of Research of the National Bureau of Standards, Vol. 45, Issue 4
  • DOI: 10.6028/jres.045.026

Extension and optimization of the FIND algorithm: Computing Green’s and less-than Green’s functions
journal, February 2012


Rational approximation to the Fermi–Dirac function with applications in density functional theory
journal, May 2010


Electronic structure calculations with dynamical mean-field theory
journal, August 2006


The Role of Elimination Trees in Sparse Factorization
journal, January 1990

  • Liu, Joseph W. H.
  • SIAM Journal on Matrix Analysis and Applications, Vol. 11, Issue 1
  • DOI: 10.1137/0611010

An estimator for the diagonal of a matrix
journal, November 2007


Finding off-diagonal entries of the inverse of a large symmetric sparse matrix: FINDING OFF-DIAGONAL ENTRIES OF THE INVERSE OF A LARGE SPARSE MATRIX
journal, February 2012

  • Eastwood, Shawn; Wan, Justin W. L.
  • Numerical Linear Algebra with Applications, Vol. 20, Issue 1
  • DOI: 10.1002/nla.1826

A nested dissection approach to modeling transport in nanodevices: Algorithms and applications: NESTED DISSECTION APPROACH TO MODELING TRANSPORT IN NANODEVICES
journal, June 2013

  • Hetmaniuk, U.; Zhao, Y.; Anantram, M. P.
  • International Journal for Numerical Methods in Engineering, Vol. 95, Issue 7
  • DOI: 10.1002/nme.4518

Adaptive local basis set for Kohn–Sham density functional theory in a discontinuous Galerkin framework I: Total energy calculation
journal, February 2012


A fast algorithm for sparse matrix computations related to inversion
journal, June 2013


An Efficient Block-Oriented Approach to Parallel Sparse Cholesky Factorization
journal, November 1994

  • Rothberg, Edward; Gupta, Anoop
  • SIAM Journal on Scientific Computing, Vol. 15, Issue 6
  • DOI: 10.1137/0915085

A Shifted Block Lanczos Algorithm for Solving Sparse Symmetric Generalized Eigenproblems
journal, January 1994

  • Grimes, Roger G.; Lewis, John G.; Simon, Horst D.
  • SIAM Journal on Matrix Analysis and Applications, Vol. 15, Issue 1
  • DOI: 10.1137/S0895479888151111

Computing entries of the inverse of a sparse matrix using the FIND algorithm
journal, November 2008


PT-Scotch: A tool for efficient parallel graph ordering
journal, July 2008


Self-Consistent Equations Including Exchange and Correlation Effects
journal, November 1965