Fast synchronization‐free algorithms for parallel sparse triangular solves with multiple right‐hand sides
- Niels Bohr Institute University of Copenhagen Copenhagen Denmark, Scientific Computing Department STFC Rutherford Appleton Laboratory UK, Department of Computer Science Norwegian University of Science and Technology Trondheim Norway
- Pacific Northwest National Lab Richland USA
- Scientific Computing Department STFC Rutherford Appleton Laboratory UK
- Niels Bohr Institute University of Copenhagen Copenhagen Denmark
The sparse triangular solve kernels, SpTRSV and SpTRSM, are important building blocks for a number of numerical linear algebra routines. Parallelizing SpTRSV and SpTRSM on today's manycore platforms, such as GPUs, is not an easy task since computing a component of the solution may depend on previously computed components, enforcing a degree of sequential processing. As a consequence, most existing work introduces a preprocessing stage to partition the components into a group of level‐sets or colour‐sets so that components within a set are independent and can be processed simultaneously during the subsequent solution stage. However, this class of methods requires a long preprocessing time as well as significant runtime synchronization overheads between the sets. To address this, we propose in this paper novel approaches for SpTRSV and SpTRSM in which the ordering between components is naturally enforced within the solution stage. In this way, the cost for preprocessing can be greatly reduced, and the synchronizations between sets are completely eliminated. To further exploit the data‐parallelism, we also develop an adaptive scheme for efficiently processing multiple right‐hand sides in SpTRSM. A comparison with a state‐of‐the‐art library supplied by the GPU vendor, using 20 sparse matrices on the latest GPU device, shows that the proposed approach obtains an average speedup of over two for SpTRSV and up to an order of magnitude speedup for SpTRSM. In addition, our method is up to two orders of magnitude faster for the preprocessing stage than existing SpTRSV and SpTRSM methods.
- Sponsoring Organization:
- USDOE
- OSTI ID:
- 1398070
- Alternate ID(s):
- OSTI ID: 1557091
- Journal Information:
- Concurrency and Computation. Practice and Experience, Journal Name: Concurrency and Computation. Practice and Experience Journal Issue: 21 Vol. 29; ISSN 1532-0626
- Publisher:
- Wiley Blackwell (John Wiley & Sons)Copyright Statement
- Country of Publication:
- United Kingdom
- Language:
- English
Similar Records
Unified Communication Optimization Strategies for Sparse Triangular Solver on CPU and GPU Clusters
Fast and Scalable Sparse Triangular Solver for Multi-GPU Based HPC Architectures