Evaluating asynchronous Schwarz solvers on GPUs

Nayak, Pratik; Cojean, Terry; Anzt, Hartwig

doi:10.1177/1094342020946814

Evaluating asynchronous Schwarz solvers on GPUs

Journal Article · Mon Aug 10 00:00:00 EDT 2020 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/1094342020946814· OSTI ID:1778413

^[1]; Cojean, Terry ^[1]; Anzt, Hartwig ^[2]

Karlsruhe Institute of Technology, Karlsruhe, Germany
Karlsruhe Institute of Technology, Karlsruhe, Germany, University of Tennessee, Knoxville, USA

With the commencement of the exascale computing era, we realize that the majority of the leadership supercomputers are heterogeneous and massively parallel. Even a single node can contain multiple co-processors such as GPUs and multiple CPU cores. For example, ORNL’s Summit accumulates six NVIDIA Tesla V100 GPUs and 42 IBM Power9 cores on each node. Synchronizing across compute resources of multiple nodes can be prohibitively expensive. Hence, it is necessary to develop and study asynchronous algorithms that circumvent this issue of bulk-synchronous computing. In this study, we examine the asynchronous version of the abstract Restricted Additive Schwarz method as a solver. We do not explicitly synchronize, but allow the communication between the sub-domains to be completely asynchronous, thereby removing the bulk synchronous nature of the algorithm.

We accomplish this by using the one-sided Remote Memory Access (RMA) functions of the MPI standard. We study the benefits of using such an asynchronous solver over its synchronous counterpart. We also study the communication patterns governed by the partitioning and the overlap between the sub-domains on the global solver. Finally, we show that this concept can render attractive performance benefits over the synchronous counterparts even for a well-balanced problem.

View Journal Article

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 1778413

Journal Information:: International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 3 Vol. 35; ISSN 1094-3420

Publisher:: SAGE PublicationsCopyright Statement

Country of Publication:: United States

Language:: English

References (21)

Algorithms for distributed termination detection Mattern, Friedemann Distributed Computing, Vol. 2, Issue 3 https://doi.org/10.1007/BF01782776	journal	September 1987
Asynchronous optimized Schwarz methods with and without overlap Magoulès, Frédéric; Szyld, Daniel B.; Venet, Cédric Numerische Mathematik, Vol. 137, Issue 1 https://doi.org/10.1007/s00211-017-0872-z	journal	March 2017
Weighted max norms, splittings, and overlapping additive Schwarz iterations Frommer, Andreas; Szyld, Daniel B. Numerische Mathematik, Vol. 83, Issue 2 https://doi.org/10.1007/s002110050449	journal	August 1999
Algebraic theory of multiplicative Schwarz methods Benzi, Michele; Frommer, Andreas; Nabben, Reinhard Numerische Mathematik, Vol. 89, Issue 4 https://doi.org/10.1007/s002110100275	journal	October 2001
Towards Optimized Schwarz Methods for the Navier–Stokes Equations Blayo, Eric; Cherel, David; Rousseau, Antoine Journal of Scientific Computing, Vol. 66, Issue 1 https://doi.org/10.1007/s10915-015-0020-9	journal	April 2015
Derivation of a termination detection algorithm for distributed computations Dijkstra, Edsger W.; Feijen, W. H. J.; van Gasteren, A. J. M. Information Processing Letters, Vol. 16, Issue 5 https://doi.org/10.1016/0020-0190(83)90092-3	journal	June 1983
On asynchronous iterations Frommer, Andreas; Szyld, Daniel B. Journal of Computational and Applied Mathematics, Vol. 123, Issue 1-2 https://doi.org/10.1016/S0377-0427(00)00409-X	journal	November 2000
Asynchronous iterative sub-structuring methods Magoulès, Frédéric; Venet, Cédric Mathematics and Computers in Simulation, Vol. 145 https://doi.org/10.1016/j.matcom.2016.05.009	journal	March 2018
Performance of asynchronous optimized Schwarz with one-sided communication Yamazaki, Ichitaro; Chow, Edmond; Bouteiller, Aurelien Parallel Computing, Vol. 86 https://doi.org/10.1016/j.parco.2019.05.004	journal	August 2019
2.1 Summit and Sierra: Designing AI/HPC Supercomputers Kahle, James A.; Moreno, Jaime; Dreps, Dan 2019 IEEE International Solid- State Circuits Conference - (ISSCC) https://doi.org/10.1109/ISSCC.2019.8662426	conference	February 2019
Ultra-Performance Pascal GPU and NVLink Interconnect Foley, Denis; Danskin, John IEEE Micro, Vol. 37, Issue 2 https://doi.org/10.1109/MM.2017.37	journal	March 2017
A decentralized convergence detection algorithm for asynchronous parallel iterative algorithms Bahi, J. M.; Contassot-Vivier, S.; Couturier, R. IEEE Transactions on Parallel and Distributed Systems, Vol. 16, Issue 1 https://doi.org/10.1109/TPDS.2005.2	journal	January 2005
An Algebraic Convergence Theory for Restricted Additive Schwarz Methods Using Weighted Max Norms Frommer, Andreas; Szyld, Daniel B. SIAM Journal on Numerical Analysis, Vol. 39, Issue 2 https://doi.org/10.1137/S0036142900370824	journal	January 2001
A Restricted Additive Schwarz Preconditioner for General Sparse Linear Systems Cai, Xiao-Chuan; Sarkis, Marcus SIAM Journal on Scientific Computing, Vol. 21, Issue 2 https://doi.org/10.1137/S106482759732678X	journal	January 1999
Algorithm 887: CHOLMOD, Supernodal Sparse Cholesky Factorization and Update/Downdate Chen, Yanqing; Davis, Timothy A.; Hager, William W. ACM Transactions on Mathematical Software, Vol. 35, Issue 3 https://doi.org/10.1145/1391989.1391995	journal	October 2008
Asynchronous Iterative Methods for Multiprocessors Baudet, Gérard M. Journal of the ACM, Vol. 25, Issue 2 https://doi.org/10.1145/322063.322067	journal	April 1978
Load-balancing Sparse Matrix Vector Product Kernels on GPUs Anzt, Hartwig; Cojean, Terry; Yen-Chen, Chen ACM Transactions on Parallel Computing, Vol. 7, Issue 1 https://doi.org/10.1145/3380930	journal	April 2020
Distributed Termination Francez, Nissim ACM Transactions on Programming Languages and Systems, Vol. 2, Issue 1 https://doi.org/10.1145/357084.357087	journal	January 1980
The deal.II library, Version 9.0 Alzetta, Giovanni; Arndt, Daniel; Bangerth, Wolfgang Journal of Numerical Mathematics, Vol. 26, Issue 4 https://doi.org/10.1515/jnma-2018-0054	journal	December 2018
Convergence of the multiplicative Schwarz method for singularly perturbed convection-diffusion problems discretized on a Shishkin mesh Echeverría, Carlos; Liesen, Jörg; Szyld, Daniel B. ETNA - Electronic Transactions on Numerical Analysis, Vol. 48 https://doi.org/10.1553/etna_vol48s40	journal	January 2018
Restricted Additive Schwarz Preconditioner for Elliptic Equations with Jump Coefficients Liu, Zhiyong; He, Yinnian Advances in Applied Mathematics and Mechanics, Vol. 8, Issue 6 https://doi.org/10.4208/aamm.2014.m669	journal	September 2016

Similar Records

Performance of asynchronous optimized Schwarz with one-sided communication

Journal Article · Wed May 15 00:00:00 EDT 2019 · Parallel Computing · OSTI ID:1577477

Asynchronous Iterative Solvers for Extreme-Scale Computing

Technical Report · Wed Feb 03 23:00:00 EST 2021 · OSTI ID:1764239

Evaluating asynchronous Schwarz solvers on GPUs

Citation Formats

References (21)

Similar Records

Related Subjects