skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on January 20, 2021

Title: Preparing sparse solvers for exascale computing

Abstract

Sparse solvers provide essential functionality for a wide variety of scientific applications. Highly parallel sparse solvers are essential for continuing advances in high-fidelity, multi-physics and multi-scale simulations, especially as we target exascale platforms. This paper describes the challenges, strategies and progress of the US Department of Energy Exascale Computing project towards providing sparse solvers for exascale computing platforms. We address the demands of systems with thousands of high-performance node devices where exposing concurrency, hiding latency and creating alternative algorithms become essential. The efforts described here are works in progress, highlighting current success and upcoming challenges.

Authors:
 [1];  [2];  [3];  [4]; ORCiD logo [2];  [4];  [5];  [5];  [2];  [6];  [5];  [2];  [3]
  1. Univ. of Tennessee, Knoxville, TN (United States)
  2. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
  3. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  4. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  5. Argonne National Lab. (ANL), Argonne, IL (United States)
  6. Vienna University of Technology, Wien, Wien, Austria
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC)
OSTI Identifier:
1601440
Alternate Identifier(s):
OSTI ID: 1604740
Report Number(s):
[SAND-2019-10821J]
[Journal ID: ISSN 1364-503X; 679361]
Grant/Contract Number:  
[AC04-94AL85000; AC02-05CH11231]
Resource Type:
Accepted Manuscript
Journal Name:
Philosophical Transactions of the Royal Society. A, Mathematical, Physical and Engineering Sciences
Additional Journal Information:
[ Journal Volume: 378; Journal Issue: 2166]; Journal ID: ISSN 1364-503X
Publisher:
The Royal Society Publishing
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; sparse solvers; mathematical libraries

Citation Formats

Anzt, Hartwig, Boman, Erik, Falgout, Rob, Ghysels, Pieter, Heroux, Michael, Li, Xiaoye, Curfman McInnes, Lois, Tran Mills, Richard, Rajamanickam, Sivasankaran, Rupp, Karl, Smith, Barry, Yamazaki, Ichitaro, and Meier Yang, Ulrike. Preparing sparse solvers for exascale computing. United States: N. p., 2020. Web. doi:10.1098/rsta.2019.0053.
Anzt, Hartwig, Boman, Erik, Falgout, Rob, Ghysels, Pieter, Heroux, Michael, Li, Xiaoye, Curfman McInnes, Lois, Tran Mills, Richard, Rajamanickam, Sivasankaran, Rupp, Karl, Smith, Barry, Yamazaki, Ichitaro, & Meier Yang, Ulrike. Preparing sparse solvers for exascale computing. United States. doi:10.1098/rsta.2019.0053.
Anzt, Hartwig, Boman, Erik, Falgout, Rob, Ghysels, Pieter, Heroux, Michael, Li, Xiaoye, Curfman McInnes, Lois, Tran Mills, Richard, Rajamanickam, Sivasankaran, Rupp, Karl, Smith, Barry, Yamazaki, Ichitaro, and Meier Yang, Ulrike. Mon . "Preparing sparse solvers for exascale computing". United States. doi:10.1098/rsta.2019.0053.
@article{osti_1601440,
title = {Preparing sparse solvers for exascale computing},
author = {Anzt, Hartwig and Boman, Erik and Falgout, Rob and Ghysels, Pieter and Heroux, Michael and Li, Xiaoye and Curfman McInnes, Lois and Tran Mills, Richard and Rajamanickam, Sivasankaran and Rupp, Karl and Smith, Barry and Yamazaki, Ichitaro and Meier Yang, Ulrike},
abstractNote = {Sparse solvers provide essential functionality for a wide variety of scientific applications. Highly parallel sparse solvers are essential for continuing advances in high-fidelity, multi-physics and multi-scale simulations, especially as we target exascale platforms. This paper describes the challenges, strategies and progress of the US Department of Energy Exascale Computing project towards providing sparse solvers for exascale computing platforms. We address the demands of systems with thousands of high-performance node devices where exposing concurrency, hiding latency and creating alternative algorithms become essential. The efforts described here are works in progress, highlighting current success and upcoming challenges.},
doi = {10.1098/rsta.2019.0053},
journal = {Philosophical Transactions of the Royal Society. A, Mathematical, Physical and Engineering Sciences},
number = [2166],
volume = [378],
place = {United States},
year = {2020},
month = {1}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on January 20, 2021
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Updating incomplete factorization preconditioners for model order reduction
journal, February 2016


Kokkos: Enabling manycore performance portability through polymorphic memory access patterns
journal, December 2014

  • Carter Edwards, H.; Trott, Christian R.; Sunderland, Daniel
  • Journal of Parallel and Distributed Computing, Vol. 74, Issue 12
  • DOI: 10.1016/j.jpdc.2014.07.003

Fine-Grained Parallel Incomplete LU Factorization
journal, January 2015

  • Chow, Edmond; Patel, Aftab
  • SIAM Journal on Scientific Computing, Vol. 37, Issue 2
  • DOI: 10.1137/140968896

ParILUT---A New Parallel Threshold ILU Factorization
journal, January 2018

  • Anzt, Hartwig; Chow, Edmond; Dongarra, Jack
  • SIAM Journal on Scientific Computing, Vol. 40, Issue 4
  • DOI: 10.1137/16M1079506

Parallel Graph Coloring for Manycore Architectures
conference, May 2016

  • Deveci, Mehmet; Boman, Erik G.; Devine, Karen D.
  • 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
  • DOI: 10.1109/IPDPS.2016.54

A New Paradigm for Parallel Adaptive Meshing Algorithms
journal, January 2000


Towards Extreme-Scale Simulations for Low Mach Fluids with Second-Generation Trilinos
journal, December 2014

  • Lin, Paul; Bettencourt, Matthew; Domino, Stefan
  • Parallel Processing Letters, Vol. 24, Issue 04
  • DOI: 10.1142/S0129626414420055

Multithreaded sparse matrix-matrix multiplication for many-core and GPU architectures
journal, October 2018


ViennaCL---Linear Algebra Library for Multi- and Many-Core Architectures
journal, January 2016

  • Rupp, Karl; Tillet, Philippe; Rudolf, Florian
  • SIAM Journal on Scientific Computing, Vol. 38, Issue 5
  • DOI: 10.1137/15M1026419

An overview of the Trilinos project
journal, September 2005

  • Heroux, Michael A.; Phipps, Eric T.; Salinger, Andrew G.
  • ACM Transactions on Mathematical Software, Vol. 31, Issue 3
  • DOI: 10.1145/1089014.1089021

A Distributed-Memory Package for Dense Hierarchically Semi-Separable Matrix Computations Using Randomization
journal, June 2016

  • Rouet, François-Henry; Li, Xiaoye S.; Ghysels, Pieter
  • ACM Transactions on Mathematical Software, Vol. 42, Issue 4
  • DOI: 10.1145/2930660

ParILUT - A Parallel Threshold ILU for GPUs
conference, May 2019

  • Anzt, Hartwig; Ribizel, Tobias; Flegar, Goran
  • 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
  • DOI: 10.1109/IPDPS.2019.00033

Reducing communication in algebraic multigrid using additive variants: REDUCING COMMUNICATION IN AMG WITH ADDITIVE VARIANTS
journal, February 2014

  • Vassilevski, Panayot S.; Yang, Ulrike Meier
  • Numerical Linear Algebra with Applications, Vol. 21, Issue 2
  • DOI: 10.1002/nla.1928

Communication Avoiding ILU0 Preconditioner
journal, January 2015

  • Grigori, Laura; Moufawad, Sophie
  • SIAM Journal on Scientific Computing, Vol. 37, Issue 2
  • DOI: 10.1137/130930376

Adaptive precision in block-Jacobi preconditioning for iterative sparse linear system solvers: Adaptive precision in block-Jacobi preconditioning for iterative solvers
journal, March 2018

  • Anzt, Hartwig; Dongarra, Jack; Flegar, Goran
  • Concurrency and Computation: Practice and Experience, Vol. 31, Issue 6
  • DOI: 10.1002/cpe.4460

ShyLU: A Hybrid-Hybrid Solver for Multicore Platforms
conference, May 2012

  • Rajamanickam, Sivasankaran; Boman, Erik G.; Heroux, Michael A.
  • 2012 IEEE International Symposium on Parallel & Distributed Processing (IPDPS), 2012 IEEE 26th International Parallel and Distributed Processing Symposium
  • DOI: 10.1109/IPDPS.2012.64

Reducing Parallel Communication in Algebraic Multigrid through Sparsification
journal, January 2016

  • Bienz, Amanda; Falgout, Robert D.; Gropp, William
  • SIAM Journal on Scientific Computing, Vol. 38, Issue 5
  • DOI: 10.1137/15M1026341

Exposing Fine-Grained Parallelism in Algebraic Multigrid Methods
journal, January 2012

  • Bell, Nathan; Dalton, Steven; Olson, Luke N.
  • SIAM Journal on Scientific Computing, Vol. 34, Issue 4
  • DOI: 10.1137/110838844

A fast adaptive solver for hierarchically semiseparable representations
journal, December 2005


Improving Multifrontal Methods by Means of Block Low-Rank Representations
journal, January 2015

  • Amestoy, Patrick; Ashcraft, Cleve; Boiteau, Olivier
  • SIAM Journal on Scientific Computing, Vol. 37, Issue 3
  • DOI: 10.1137/120903476

A new parallel domain decomposition method for the adaptive finite element solution of elliptic partial differential equations
journal, January 2001

  • Bank, Randolph E.; Jimack, Peter K.
  • Concurrency and Computation: Practice and Experience, Vol. 13, Issue 5
  • DOI: 10.1002/cpe.569

Non-Galerkin Coarse Grids for Algebraic Multigrid
journal, January 2014

  • Falgout, Robert D.; Schroder, Jacob B.
  • SIAM Journal on Scientific Computing, Vol. 36, Issue 3
  • DOI: 10.1137/130931539

A Study of Clustering Techniques and Hierarchical Matrix Formats for Kernel Ridge Regression
conference, May 2018

  • Rebrova, Elizaveta; Chavez, Gustavo; Liu, Yang
  • 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)
  • DOI: 10.1109/IPDPSW.2018.00140

Fast linear algebra-based triangle counting with KokkosKernels
conference, September 2017

  • Wolf, Michael M.; Deveci, Mehmet; Berry, Jonathan W.
  • 2017 IEEE High-Performance Extreme Computing Conference (HPEC), 2017 IEEE High Performance Extreme Computing Conference (HPEC)
  • DOI: 10.1109/HPEC.2017.8091043

Designing vector-friendly compact BLAS and LAPACK kernels
conference, January 2017

  • Kim, Kyungjoo; Costa, Timothy B.; Deveci, Mehmet
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '17
  • DOI: 10.1145/3126908.3126941

Distance-two interpolation for parallel algebraic multigrid
journal, January 2008

  • De Sterck, Hans; Falgout, Robert D.; Nolting, Joshua W.
  • Numerical Linear Algebra with Applications, Vol. 15, Issue 2-3
  • DOI: 10.1002/nla.559

Stencil computations for PDE-based applications with examples from DUNE and hypre: Stencil Computations for PDE-based Applications
journal, February 2017

  • Engwer, C.; Falgout, R. D.; Yang, U. M.
  • Concurrency and Computation: Practice and Experience, Vol. 29, Issue 17
  • DOI: 10.1002/cpe.4097

Fast Triangle Counting Using Cilk
conference, September 2018

  • Yasar, Abdurrahman; Rajamanickam, Sivasankaran; Wolf, Michael
  • 2018 IEEE High Performance Extreme Computing Conference (HPEC)
  • DOI: 10.1109/HPEC.2018.8547563

A low-communication, parallel algorithm for solving PDEs based on range decomposition: RANGE DECOMPOSITION: A LOW COMMUNICATION ALGORITHM FOR SOLVING PDES
journal, March 2016

  • Appelhans, David J.; Manteuffel, Tom; McCormick, Steve
  • Numerical Linear Algebra with Applications, Vol. 24, Issue 3
  • DOI: 10.1002/nla.2041

Basker: Parallel sparse LU factorization utilizing hierarchical parallelism and data layouts
journal, October 2017


A Communication-Avoiding 3D LU Factorization Algorithm for Sparse Matrices
conference, May 2018

  • Sao, Piyush; Li, Xiaoye Sherry; Vuduc, Richard
  • 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
  • DOI: 10.1109/IPDPS.2018.00100

A communication-avoiding 3D algorithm for sparse LU factorization on heterogeneous systems
journal, September 2019

  • Sao, Piyush; Li, Xiaoye S.; Vuduc, Richard
  • Journal of Parallel and Distributed Computing, Vol. 131
  • DOI: 10.1016/j.jpdc.2019.03.004

Domain Decomposition Preconditioners for Communication-Avoiding Krylov Methods on a Hybrid CPU/GPU Cluster
conference, November 2014

  • Yamazaki, Ichitaro; Rajamanickam, Sivasankaran; Boman, Erik G.
  • SC14: International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2014.81

A Parallel Multigrid Preconditioned Conjugate Gradient Algorithm for Groundwater Flow Simulations
journal, September 1996

  • Ashby, Steven F.; Falgout, Robert D.
  • Nuclear Science and Engineering, Vol. 124, Issue 1
  • DOI: 10.13182/NSE96-A24230

Robust and Accurate Stopping Criteria for Adaptive Randomized Sampling in Matrix-Free Hierarchically Semiseparable Construction
journal, January 2019

  • Gorman, Christopher; Chávez, Gustavo; Ghysels, Pieter
  • SIAM Journal on Scientific Computing, Vol. 41, Issue 5
  • DOI: 10.1137/18M1194961

Tacho: Memory-Scalable Task Parallel Sparse Cholesky Factorization
conference, May 2018

  • Kim, Kyungjoo; Edwards, H. Carter; Rajamanickam, Sivasankaran
  • 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)
  • DOI: 10.1109/IPDPSW.2018.00094

Algebraic Multigrid Domain and Range Decomposition (AMG-DD/AMG-RD)
journal, January 2015

  • Bank, R.; Falgout, R.; Jones, T.
  • SIAM Journal on Scientific Computing, Vol. 37, Issue 5
  • DOI: 10.1137/140974717

An HSS Matrix-Inspired Butterfly-Based Direct Solver for Analyzing Scattering From Two-Dimensional Objects
journal, January 2017

  • Liu, Yang; Guo, Han; Michielssen, Eric
  • IEEE Antennas and Wireless Propagation Letters, Vol. 16
  • DOI: 10.1109/LAWP.2016.2626786

A communication-avoiding 3D sparse triangular solver
conference, January 2019

  • Sao, Piyush; Kannan, Ramakrishnan; Li, Xiaoye Sherry
  • Proceedings of the ACM International Conference on Supercomputing - ICS '19
  • DOI: 10.1145/3330345.3330357