Addressing Global Data Dependencies in Heterogeneous Asynchronous Runtime Systems on GPUs

Peterson, Brad; Humphrey, Alan; Schmidt, John; Berzins, Martin

doi:10.1145/3152041.3152082

Addressing Global Data Dependencies in Heterogeneous Asynchronous Runtime Systems on GPUs

Conference · 01 November 2017 · Proceedings of the 3rd International IEEE Workshop on Extreme Scale Programming Models and Middleware

DOI:https://doi.org/10.1145/3152041.3152082· OSTI ID:1582428

Peterson, Brad ^[1]; Humphrey, Alan ^[1]; Schmidt, John ^[1]; Berzins, Martin ^[1]

Univ. of Utah, Salt Lake City, UT (United States)

Large-scale parallel applications with complex global data dependencies beyond those of reductions pose significant scalability challenges in an asynchronous runtime system. Internodal challenges include identifying the all-to-all communication of data dependencies among the nodes. Intranodal challenges include gathering together these data dependencies into usable data objects while avoiding data duplication. This paper addresses these challenges within the context of a large-scale, industrial coal boiler simulation using the Uintah asynchronous many-task runtime system on GPU architectures. We show significant reduction in time spent analyzing data dependencies through refinements in our dependency search algorithm. Multiple task graphs are used to eliminate subsequent analysis when task graphs change in predictable and repeatable ways. Using a combined data store and task scheduler redesign reduces data dependency duplication ensuring that problems fit within host and GPU memory. Furthermore, these modifications did not require any changes to application code or sweeping changes to the Uintah runtime system. We report results running on the DOE Titan system on 119K CPU cores and 7.5K GPUs simultaneously. Our solutions can be generalized to other task dependency problems with global dependencies among thousands of nodes which must be processed efficiently at large scale.

Research Organization:: Univ. of Utah, Salt Lake City, UT (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC)

DOE Contract Number:: NA0002375; AC05-00OR22725

OSTI ID:: 1582428

Journal Information:: Proceedings of the 3rd International IEEE Workshop on Extreme Scale Programming Models and Middleware, Conference: 3.International IEEE Workshop on Extreme Scale Programming Models and Middleware (ESPM2'17), Denver, CO (United States), 12 Nov 2017

Country of Publication:: United States

Language:: English

References (13)

Dynamic task scheduling for the Uintah framework No authors listed 2010 3rd Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) https://doi.org/10.1109/MTAGS.2010.5699431		November 2010
Reducing overhead in the Uintah framework to support short-lived tasks on GPU-heterogeneous architectures No authors listed Proceedings of the 5th International Workshop on Domain-Specific Languages and High-Level Frameworks for High Performance Computing - WOLFHPC '15 https://doi.org/10.1145/2830018.2830023		January 2015
Investigating applications portability with the Uintah DAG-based runtime system on PetaScale supercomputers No authors listed Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13 https://doi.org/10.1145/2503210.2503250		January 2013
The cosmological simulation code gadget-2 No authors listed Monthly Notices of the Royal Astronomical Society https://doi.org/10.1111/j.1365-2966.2005.09655.x		December 2005
Large Scale Parallel Solution of Incompressible Flow Problems Using Uintah and Hypre No authors listed 2013 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing https://doi.org/10.1109/CCGrid.2013.10		May 2013
Spatial Domain-Based Parallelism in Large-Scale, Participating-Media, Radiative Transport Applications No authors listed Numerical Heat Transfer, Part B: Fundamentals https://doi.org/10.1080/10407799708915117		June 1997
Parallelization of the P-1 Radiation Model No authors listed Numerical Heat Transfer, Part B: Fundamentals https://doi.org/10.1080/10407790500344068		January 2006
Efficient Methods for Handling Long-Range Forces in Particle–Particle Simulations No authors listed Journal of Computational Physics https://doi.org/10.1006/jcph.2000.6541		August 2000
The Design and Implementation of hypre, a Library of Parallel High Performance Preconditioners No authors listed Lecture Notes in Computational Science and Engineering https://doi.org/10.1007/3-540-31619-1_8		January 2006
A Scalable Algorithm for Radiative Heat Transfer Using Reverse Monte Carlo Ray Tracing No authors listed Lecture Notes in Computer Science https://doi.org/10.1007/978-3-319-20119-1_16		January 2015
Radiative Heat Transfer Calculation on 16384 GPUs Using a Reverse Monte Carlo Ray Tracing Approach with Adaptive Mesh Refinement No authors listed 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) https://doi.org/10.1109/IPDPSW.2016.93		May 2016
CHARM++: a portable concurrent object oriented system based on C++ No authors listed ACM SIGPLAN Notices https://doi.org/10.1145/167962.165874		October 1993
The uintah framework: a unified heterogeneous task scheduling and runtime system No authors listed 2012 SC Companion: High Performance Computing, Networking Storage and Analysis https://doi.org/10.1109/SCC.2012.6674233		November 2012

Cited By (1)

Node failure resiliency for Uintah without checkpointing No authors listed Concurrency and Computation: Practice and Experience https://doi.org/10.1002/cpe.5340		June 2019

Similar Records

An Integrated Approach to Scaling Task-Based Runtime Systems for Next Generation Engineering problems

Journal Article · 2017 · International Conference for High Performance Computing, Networking, Storage and Analysis · OSTI ID:1755953

Humphrey, Alan; Peterson, Brad; Schmidt, John; +9 more

Automatic Halo Management for the Uintah GPU-Heterogeneous Asynchronous Many-Task Runtime

Journal Article · 2018 · International Journal of Parallel Programming · OSTI ID:1567537

Peterson, Brad; Humphrey, Alan; Sunderland, Dan; +4 more

Radiation modeling using the Uintah heterogeneous CPU/GPU runtime system. In: XSEDE '12 Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging from the eXtreme to the campus and beyond, Article No. 4

Conference · 2011 · OSTI ID:1567625

Humphrey, Alan; Meng, Qingyu; Berzins, Martin; +1 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Data dependencies
Asynchronous Many-Task
Programming Models
Runtime Systems
Scalability
GPU
Uintah
Coal Boiler
Radiative Heat Transfer

Addressing Global Data Dependencies in Heterogeneous Asynchronous Runtime Systems on GPUs

Citation Formats

References (13)

Cited By (1)

Similar Records

Related Subjects