Addressing Global Data Dependencies in Heterogeneous Asynchronous Runtime Systems on GPUs

Peterson, Brad; Humphrey, Alan; Schmidt, John; Berzins, Martin

doi:10.1145/3152041.3152082

Addressing Global Data Dependencies in Heterogeneous Asynchronous Runtime Systems on GPUs

Conference · Wed Nov 01 00:00:00 EDT 2017 · Proceedings of the 3rd International IEEE Workshop on Extreme Scale Programming Models and Middleware

DOI:https://doi.org/10.1145/3152041.3152082· OSTI ID:1582428

Peterson, Brad ^[1]; Humphrey, Alan ^[2]; Schmidt, John ^[2]; Berzins, Martin ^[2]

Univ. of Utah, Salt Lake City, UT (United States); University of Utah
Univ. of Utah, Salt Lake City, UT (United States)

Large-scale parallel applications with complex global data dependencies beyond those of reductions pose significant scalability challenges in an asynchronous runtime system. Internodal challenges include identifying the all-to-all communication of data dependencies among the nodes. Intranodal challenges include gathering together these data dependencies into usable data objects while avoiding data duplication. This paper addresses these challenges within the context of a large-scale, industrial coal boiler simulation using the Uintah asynchronous many-task runtime system on GPU architectures. We show significant reduction in time spent analyzing data dependencies through refinements in our dependency search algorithm. Multiple task graphs are used to eliminate subsequent analysis when task graphs change in predictable and repeatable ways. Using a combined data store and task scheduler redesign reduces data dependency duplication ensuring that problems fit within host and GPU memory. Furthermore, these modifications did not require any changes to application code or sweeping changes to the Uintah runtime system. We report results running on the DOE Titan system on 119K CPU cores and 7.5K GPUs simultaneously. Our solutions can be generalized to other task dependency problems with global dependencies among thousands of nodes which must be processed efficiently at large scale.

View Conference

Research Organization:: Univ. of Utah, Salt Lake City, UT (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC)

DOE Contract Number:: NA0002375; AC05-00OR22725

OSTI ID:: 1582428

Journal Information:: Proceedings of the 3rd International IEEE Workshop on Extreme Scale Programming Models and Middleware, Journal Name: Proceedings of the 3rd International IEEE Workshop on Extreme Scale Programming Models and Middleware

Country of Publication:: United States

Language:: English

References (13)

Dynamic task scheduling for the Uintah framework Meng, Qingyu; Luitjens, Justin; Berzins, Martin 2010 3rd Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) https://doi.org/10.1109/MTAGS.2010.5699431	conference	November 2010
Radiative Heat Transfer Calculation on 16384 GPUs Using a Reverse Monte Carlo Ray Tracing Approach with Adaptive Mesh Refinement Humphrey, Alan; Sunderland, Daniel; Harman, Todd 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) https://doi.org/10.1109/IPDPSW.2016.93	conference	May 2016
The uintah framework: a unified heterogeneous task scheduling and runtime system Meng, Qingyu; Humphrey, Alan; Berzins, Martin 2012 SC Companion: High Performance Computing, Networking Storage and Analysis https://doi.org/10.1109/SCC.2012.6674233	conference	November 2012
The cosmological simulation code gadget-2 Springel, Volker Monthly Notices of the Royal Astronomical Society, Vol. 364, Issue 4 https://doi.org/10.1111/j.1365-2966.2005.09655.x	journal	December 2005
Parallelization of the P-1 Radiation Model Krishnamoorthy, Gautham; Rawat, Rajesh; Smith, Philip J. Numerical Heat Transfer, Part B: Fundamentals, Vol. 49, Issue 1 https://doi.org/10.1080/10407790500344068	journal	January 2006
A Scalable Algorithm for Radiative Heat Transfer Using Reverse Monte Carlo Ray Tracing Humphrey, Alan; Harman, Todd; Berzins, Martin Lecture Notes in Computer Science https://doi.org/10.1007/978-3-319-20119-1_16	book	January 2015
The Design and Implementation of hypre, a Library of Parallel High Performance Preconditioners Falgout, Robert D.; Jones, Jim E.; Yang, Ulrike Meier Lecture Notes in Computational Science and Engineering https://doi.org/10.1007/3-540-31619-1_8	book	January 2006
Investigating applications portability with the Uintah DAG-based runtime system on PetaScale supercomputers Meng, Qingyu; Humphrey, Alan; Schmidt, John Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13 https://doi.org/10.1145/2503210.2503250	conference	January 2013
Reducing overhead in the Uintah framework to support short-lived tasks on GPU-heterogeneous architectures Peterson, Brad; Dasari, Harish; Humphrey, Alan Proceedings of the 5th International Workshop on Domain-Specific Languages and High-Level Frameworks for High Performance Computing - WOLFHPC '15 https://doi.org/10.1145/2830018.2830023	conference	January 2015
CHARM++: a portable concurrent object oriented system based on C++ Kale, Laxmikant V.; Krishnan, Sanjeev ACM SIGPLAN Notices, Vol. 28, Issue 10 https://doi.org/10.1145/167962.165874	journal	October 1993
Efficient Methods for Handling Long-Range Forces in Particle–Particle Simulations Fangohr, Hans; Price, Andrew R.; Cox, Simon J. Journal of Computational Physics, Vol. 162, Issue 2 https://doi.org/10.1006/jcph.2000.6541	journal	August 2000
Spatial Domain-Based Parallelism in Large-Scale, Participating-Media, Radiative Transport Applications Burns, Shawn P.; Christen, Mark A. Numerical Heat Transfer, Part B: Fundamentals, Vol. 31, Issue 4 https://doi.org/10.1080/10407799708915117	journal	June 1997
Large Scale Parallel Solution of Incompressible Flow Problems Using Uintah and Hypre Schmidt, J.; Berzins, M.; Thornock, J. 2013 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing https://doi.org/10.1109/CCGrid.2013.10	conference	May 2013

Similar Records

Automatic Halo Management for the Uintah GPU-Heterogeneous Asynchronous Many-Task Runtime

Journal Article · Thu Dec 06 23:00:00 EST 2018 · International Journal of Parallel Programming · OSTI ID:1567537

The uintah framework: a unified heterogeneous task scheduling and runtime system

Conference · Thu Nov 01 00:00:00 EDT 2012 · 2012 SC Companion: High Performance Computing, Networking Storage and Analysis; 10-16 Nov. 2012; Salt Lake City, UT, USA · OSTI ID:1567606

Radiation modeling using the Uintah heterogeneous CPU/GPU runtime system. In: XSEDE '12 Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging from the eXtreme to the campus and beyond, Article No. 4

Conference · Sat Dec 31 23:00:00 EST 2011 · OSTI ID:1567625

Related Subjects

97 MATHEMATICS AND COMPUTING
Asynchronous Many-Task
Coal Boiler
Data dependencies
GPU
Programming Models
Radiative Heat Transfer
Runtime Systems
Scalability
Uintah

Addressing Global Data Dependencies in Heterogeneous Asynchronous Runtime Systems on GPUs

Citation Formats

References (13)

Similar Records

Related Subjects