Scalable line and plane relaxation in a parallel structured multigrid solver
Abstract
The efficient solution of sparse, linear systems that arise through the discretization of partial differential equations remains a key challenge for a range of high performance scientific simulations. One approach for reducing data movement and improving performance is by exposing and exploiting structure in a problem through the use of robust structured multilevel solvers. By choosing coarsening that preserves the structure of the problem, these methods maintain efficient structured computation and communication throughout the multigrid hierarchy. However, when coarsening is not permitted to be dependent on the operator, anisotropy must be addressed by the smoother — producing error compatible for coarse-grid correction with structured coarsening. Here, the components required in a scalable parallel structured solver are described with a focus on memory and communication efficiency of robust smoothers. While the implementation of communication and memory reduction techniques in smoothers integrated in a complete 3D solver present a significant engineering challenge, a novel approach is proposed that addresses these challenges systematically through a change to the solver’s execution model. Enabled by user-level threading paired with a set of data and communication abstractions, this approach permits seamless aggregation of communication in plane smoothers — directly reusing code for a 2D distributed multilevelmore »
- Authors:
-
- Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
- Univ. of Illinois at Urbana-Champaign, IL (United States)
- Publication Date:
- Research Org.:
- Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF)
- OSTI Identifier:
- 1688768
- Alternate Identifier(s):
- OSTI ID: 1776467
- Report Number(s):
- LA-UR-20-23447
Journal ID: ISSN 0167-8191
- Grant/Contract Number:
- 89233218CNA000001; NA0002374; AC52-06NA25396; OCI-0725070; ACI-1238993
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Parallel Computing
- Additional Journal Information:
- Journal Volume: 100; Journal ID: ISSN 0167-8191
- Publisher:
- Elsevier
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; parallel; line relaxation; plane relaxation communication; multigrid; structured
Citation Formats
Reisner, Andrew Ray, Moulton, John David, Berndt, Markus, and Olson, Luke N. Scalable line and plane relaxation in a parallel structured multigrid solver. United States: N. p., 2020.
Web. doi:10.1016/j.parco.2020.102705.
Reisner, Andrew Ray, Moulton, John David, Berndt, Markus, & Olson, Luke N. Scalable line and plane relaxation in a parallel structured multigrid solver. United States. https://doi.org/10.1016/j.parco.2020.102705
Reisner, Andrew Ray, Moulton, John David, Berndt, Markus, and Olson, Luke N. Tue .
"Scalable line and plane relaxation in a parallel structured multigrid solver". United States. https://doi.org/10.1016/j.parco.2020.102705. https://www.osti.gov/servlets/purl/1688768.
@article{osti_1688768,
title = {Scalable line and plane relaxation in a parallel structured multigrid solver},
author = {Reisner, Andrew Ray and Moulton, John David and Berndt, Markus and Olson, Luke N.},
abstractNote = {The efficient solution of sparse, linear systems that arise through the discretization of partial differential equations remains a key challenge for a range of high performance scientific simulations. One approach for reducing data movement and improving performance is by exposing and exploiting structure in a problem through the use of robust structured multilevel solvers. By choosing coarsening that preserves the structure of the problem, these methods maintain efficient structured computation and communication throughout the multigrid hierarchy. However, when coarsening is not permitted to be dependent on the operator, anisotropy must be addressed by the smoother — producing error compatible for coarse-grid correction with structured coarsening. Here, the components required in a scalable parallel structured solver are described with a focus on memory and communication efficiency of robust smoothers. While the implementation of communication and memory reduction techniques in smoothers integrated in a complete 3D solver present a significant engineering challenge, a novel approach is proposed that addresses these challenges systematically through a change to the solver’s execution model. Enabled by user-level threading paired with a set of data and communication abstractions, this approach permits seamless aggregation of communication in plane smoothers — directly reusing code for a 2D distributed multilevel cycle. Results show an effective reduction in communication costs for coarse-grid problems, and result in a speedup of 8.7x in smoothing routines shown in Fig. 12 using this approach. This produces a significant improvement to strong scalability while maintaining favorable weak scaling behavior. Finally, a parallel scaling study using a series of refined meshes is included that demonstrates the effectiveness of this approach in an application of interest.},
doi = {10.1016/j.parco.2020.102705},
journal = {Parallel Computing},
number = ,
volume = 100,
place = {United States},
year = {Tue Oct 20 00:00:00 EDT 2020},
month = {Tue Oct 20 00:00:00 EDT 2020}
}
Works referenced in this record:
A parallel solver for tridiagonal linear systems for distributed memory parallel computers
journal, November 1991
- Brugnano, L.
- Parallel Computing, Vol. 17, Issue 9
Argobots: A Lightweight Low-Level Threading and Tasking Framework
journal, March 2018
- Seo, Sangmin; Amer, Abdelhalim; Balaji, Pavan
- IEEE Transactions on Parallel and Distributed Systems, Vol. 29, Issue 3
Reducing Parallel Communication in Algebraic Multigrid through Sparsification
journal, January 2016
- Bienz, Amanda; Falgout, Robert D.; Gropp, William
- SIAM Journal on Scientific Computing, Vol. 38, Issue 5
A multilevel parallel solver for block tridiagonal and banded linear systems
journal, September 1990
- Hajj, Ibrahim N.; Skelboe, Stig
- Parallel Computing, Vol. 15, Issue 1-3
On Stable Parallel Linear System Solvers
journal, January 1978
- Sameh, A. H.; Kuck, D. J.
- Journal of the ACM, Vol. 25, Issue 1
Black box multigrid
journal, December 1982
- Dendy, J. E.
- Journal of Computational Physics, Vol. 48, Issue 3
An Efficient Parallel Algorithm for the Solution of a Tridiagonal Linear System of Equations
journal, January 1973
- Stone, Harold S.
- Journal of the ACM, Vol. 20, Issue 1
Parallelization and vectorization aspects of the solution of tridiagonal linear systems
journal, May 1990
- Krechel, Arno; Plum, Hans-Joachim; Stüben, Klaus
- Parallel Computing, Vol. 14, Issue 1
A Fast Direct Solution of Poisson's Equation Using Fourier Analysis
journal, January 1965
- Hockney, R. W.
- Journal of the ACM, Vol. 12, Issue 1
A Parallel Method for Tridiagonal Equations
journal, June 1981
- Wang, H. H.
- ACM Transactions on Mathematical Software, Vol. 7, Issue 2
Scaling Structured Multigrid to 500K+ Cores Through Coarse-Grid Redistribution
journal, January 2018
- Reisner, Andrew; Olson, Luke N.; Moulton, J. David
- SIAM Journal on Scientific Computing, Vol. 40, Issue 4
A parallel hybrid banded system solver: the SPIKE algorithm
journal, February 2006
- Polizzi, Eric; Sameh, Ahmed H.
- Parallel Computing, Vol. 32, Issue 2
A Semicoarsening Multigrid Method for Elliptic Partial Differential Equations with Highly Discontinuous and Anisotropic Coefficients
journal, January 1998
- Schaffer, Steve
- SIAM Journal on Scientific Computing, Vol. 20, Issue 1
Parallel Factorizations for Tridiagonal Matrices
journal, June 1993
- Amodio, P.; Brugnano, L.; Politi, T.
- SIAM Journal on Numerical Analysis, Vol. 30, Issue 3