Scalable Asynchronous Domain Decomposition Solvers

Glusa, Christian; Boman, Erik G.; Chow, Edmond; Rajamanickam, Sivasankaran; Szyld, Daniel B.

doi:10.1137/19m1291303

Scalable Asynchronous Domain Decomposition Solvers

Journal Article · 13 December 2020 · SIAM Journal on Scientific Computing

DOI:https://doi.org/10.1137/19m1291303· OSTI ID:1769906

^[1]; Boman, Erik G. ^[1]; ^[2]; Rajamanickam, Sivasankaran ^[1]; ^[3]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Georgia Inst. of Technology, Atlanta, GA (United States)
Temple Univ., Philadelphia, PA (United States)

We discuss how parallel implementations of linear iterative solvers generally alternate between phases of data exchange and phases of local computation. Increasingly large problem sizes and more heterogeneous compute architectures make load balancing and the design of low latency network interconnects that are able to satisfy the communication requirements of linear solvers very challenging tasks. In particular, global communication patterns such as inner products become increasingly limiting at scale. We explore the use of asynchronous communication based on one-sided Message Passing Interface primitives in the context of domain decomposition solvers. In particular, a scalable asynchronous two-level Schwarz method is presented. We discuss practical issues encountered in the development of a scalable solver and show experimental results obtained on a state-of-the-art supercomputer system that illustrate the benefits of asynchronous solvers in load balanced as well as load imbalanced scenarios. Using the novel method, we can observe speedups of up to four times over its classical synchronous equivalent.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1769906

Alternate ID(s):: OSTI ID: 1832745

Report Number(s):: SAND--2020-8220J; 689827

Journal Information:: SIAM Journal on Scientific Computing, Journal Name: SIAM Journal on Scientific Computing Journal Issue: 6 Vol. 42; ISSN 1064-8275

Publisher:: SIAMCopyright Statement

Country of Publication:: United States

Language:: English

References (21)

Hiding Global Communication Latency in the GMRES Algorithm on Massively Parallel Machines Ghysels, P.; Ashby, T. J.; Meerbergen, K. SIAM Journal on Scientific Computing, Vol. 35, Issue 1 https://doi.org/10.1137/12086563X	journal	January 2013
Synchronous and asynchronous optimized Schwarz methods for one‐way subdivision of bounded domains El Haddad, Mireille; Garay, José C.; Magoulès, Frédéric Numerical Linear Algebra with Applications, Vol. 27, Issue 2 https://doi.org/10.1002/nla.2279	journal	December 2019
Asynchronous Iterative Methods for Multiprocessors Baudet, Gérard M. Journal of the ACM, Vol. 25, Issue 2 https://doi.org/10.1145/322063.322067	journal	April 1978
On the efficient implementation of preconditioned s-step conjugate gradient methods on multiprocessors with memory hierarchy Chronopoulos, A. T.; Gear, C. W. Parallel Computing, Vol. 11, Issue 1 https://doi.org/10.1016/0167-8191(89)90062-8	journal	July 1989
A Restricted Additive Schwarz Preconditioner for General Sparse Linear Systems Cai, Xiao-Chuan; Sarkis, Marcus SIAM Journal on Scientific Computing, Vol. 21, Issue 2 https://doi.org/10.1137/S106482759732678X	journal	January 1999
Performance of asynchronous optimized Schwarz with one-sided communication Yamazaki, Ichitaro; Chow, Edmond; Bouteiller, Aurelien Parallel Computing, Vol. 86 https://doi.org/10.1016/j.parco.2019.05.004	journal	August 2019
Algorithm 887: CHOLMOD, Supernodal Sparse Cholesky Factorization and Update/Downdate Chen, Yanqing; Davis, Timothy A.; Hager, William W. ACM Transactions on Mathematical Software, Vol. 35, Issue 3 https://doi.org/10.1145/1391989.1391995	journal	October 2008
Distributed Convergence Detection Based on Global Residual Error Under Asynchronous Iterations Magoules, Frederic; Gbikpi-Benissan, Guillaume IEEE Transactions on Parallel and Distributed Systems, Vol. 29, Issue 4 https://doi.org/10.1109/TPDS.2017.2780856	journal	April 2018
High Performance RDMA-Based MPI Implementation over InfiniBand Liu, Jiuxing; Wu, Jiesheng; Panda, Dhabaleswar K. International Journal of Parallel Programming, Vol. 32, Issue 3 https://doi.org/10.1023/B:IJPP.0000029272.69895.c1	journal	June 2004
An Algebraic Convergence Theory for Restricted Additive Schwarz Methods Using Weighted Max Norms Frommer, Andreas; Szyld, Daniel B. SIAM Journal on Numerical Analysis, Vol. 39, Issue 2 https://doi.org/10.1137/S0036142900370824	journal	January 2001
Toward Exascale Resilience Cappello, Franck; Geist, Al; Gropp, Bill The International Journal of High Performance Computing Applications, Vol. 23, Issue 4 https://doi.org/10.1177/1094342009347767	journal	September 2009
A Supernodal Approach to Sparse Partial Pivoting Demmel, James W.; Eisenstat, Stanley C.; Gilbert, John R. SIAM Journal on Matrix Analysis and Applications, Vol. 20, Issue 3 https://doi.org/10.1137/S0895479895291765	journal	January 1999
A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs Karypis, George; Kumar, Vipin SIAM Journal on Scientific Computing, Vol. 20, Issue 1 https://doi.org/10.1137/S1064827595287997	journal	January 1998
Restricted Additive Schwarz Preconditioners with Harmonic Overlap for Symmetric Positive Definite Linear Systems Cai, Xiao-Chuan; Dryja, Maksymilian; Sarkis, Marcus SIAM Journal on Numerical Analysis, Vol. 41, Issue 4 https://doi.org/10.1137/S0036142901389621	journal	January 2003
Asynchronous optimized Schwarz methods with and without overlap Magoulès, Frédéric; Szyld, Daniel B.; Venet, Cédric Numerische Mathematik, Vol. 137, Issue 1 https://doi.org/10.1007/s00211-017-0872-z	journal	March 2017
s-step iterative methods for symmetric linear systems Chronopoulos, A. T.; Gear, C. W. Journal of Computational and Applied Mathematics, Vol. 25, Issue 2 https://doi.org/10.1016/0377-0427(89)90045-9	journal	February 1989
On asynchronous iterations Frommer, Andreas; Szyld, Daniel B. Journal of Computational and Applied Mathematics, Vol. 123, Issue 1-2 https://doi.org/10.1016/S0377-0427(00)00409-X	journal	November 2000
Distributed asynchronous computation of fixed points Bertsekas, Dimitri P. Mathematical Programming, Vol. 27, Issue 1 https://doi.org/10.1007/BF02591967	journal	September 1983
ARock: An Algorithmic Framework for Asynchronous Parallel Coordinate Updates Peng, Zhimin; Xu, Yangyang; Yan, Ming SIAM Journal on Scientific Computing, Vol. 38, Issue 5 https://doi.org/10.1137/15M1024950	journal	January 2016
A decentralized convergence detection algorithm for asynchronous parallel iterative algorithms Bahi, J. M.; Contassot-Vivier, S.; Couturier, R. IEEE Transactions on Parallel and Distributed Systems, Vol. 16, Issue 1 https://doi.org/10.1109/TPDS.2005.2	journal	January 2005
Chaotic relaxation Chazan, D.; Miranker, W. Linear Algebra and its Applications, Vol. 2, Issue 2 https://doi.org/10.1016/0024-3795(69)90028-7	journal	April 1969

Similar Records

Highly scalable distributed-memory sparse triangular solution algorithms.

Conference · 2018 · OSTI ID:1602817

Liu, Yang; Jacquelin, Mathias; Ghysels, Pieter; +1 more

Evaluating asynchronous Schwarz solvers on GPUs

Journal Article · 2020 · International Journal of High Performance Computing Applications · OSTI ID:1778413

Nayak, Pratik; Cojean, Terry; Anzt, Hartwig

Related Subjects

97 MATHEMATICS AND COMPUTING
Asynchronous iteration
domain decomposition
Schwarz methods
chaotic relaxation

Scalable Asynchronous Domain Decomposition Solvers

Citation Formats

References (21)

Similar Records

Related Subjects