Improving Uintah's Scalability Through the Use of Portable Kokkos-Based Data Parallel Tasks

Holmen, John K.; Humphrey, Alan; Sunderland, Daniel; Berzins, Martin

doi:10.1145/3093338.3093388

Improving Uintah's Scalability Through the Use of Portable Kokkos-Based Data Parallel Tasks

Conference · Sat Jul 01 04:00:00 EDT 2017

DOI:https://doi.org/10.1145/3093338.3093388· OSTI ID:1582430

Holmen, John K. ^[1]; Humphrey, Alan ^[2]; Sunderland, Daniel ^[3]; Berzins, Martin ^[2]

Univ. of Utah, Salt Lake City, UT (United States); University of Utah
Univ. of Utah, Salt Lake City, UT (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

The University of Utah’s Carbon Capture Multidisciplinary Simulation Center (CCMSC) is using the Uintah Computational Framework to predict performance of a 1000 MWe ultra-supercritical clean coal boiler. The center aims to utilize the Intel Xeon Phi-based DOE systems, Theta and Aurora, through the Aurora Early Science Program by using the Kokkos C++ library to enable node-level performance portability. This paper describes infrastructure advancements and portability improvements made possible by the integration of Kokkos within Uintah. This integration marks a step towards consolidating Uintah’s MPI+PThreads and MPI+CUDA hybrid parallelism approaches into a single MPI+Kokkos approach. Scalability results are presented that compare serial and data parallel task execution models for a challenging radiative heat transfer calculation, central to the center’s predictive boiler simulations. Here, these results demonstrate both good strong-scaling characteristics to 256 Knights Landing (KNL) processors on the NSF Stampede system, and show the KNL-based calculation to compete with prior GPU-based results for the same calculation.

View Conference

Research Organization:: Univ. of Utah, Salt Lake City, UT (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

DOE Contract Number:: NA0002375

OSTI ID:: 1582430

Country of Publication:: United States

Language:: English

References (25)

Dynamic task scheduling for the Uintah framework Meng, Qingyu; Luitjens, Justin; Berzins, Martin 2010 3rd Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) https://doi.org/10.1109/MTAGS.2010.5699431	conference	November 2010
Radiative Heat Transfer Calculation on 16384 GPUs Using a Reverse Monte Carlo Ray Tracing Approach with Adaptive Mesh Refinement Humphrey, Alan; Sunderland, Daniel; Harman, Todd 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) https://doi.org/10.1109/IPDPSW.2016.93	conference	May 2016
Linux on the Coprocessor Jeffers, Jim; Reinders, James Intel Xeon Phi Coprocessor High Performance Programming https://doi.org/10.1016/B978-0-12-410414-3.00010-4	book	January 2013
Large Eddy Simulations of Accidental Fires Using Massively Parallel Computers Smith, Philip; Rawat, Rajesh; Spinti, Jennifer 16th AIAA Computational Fluid Dynamics Conference https://doi.org/10.2514/6.2003-3697	conference	June 2012
Knights Landing overview Jeffers, Jim; Reinders, James; Sodani, Avinash Intel Xeon Phi Processor High Performance Programming https://doi.org/10.1016/B978-0-12-809194-4.00002-8	book	January 2016
Efficient Parallelization of RMCRT for Large Scale LES Combustion Simulations Hunsaker, Isaac 20th AIAA Computational Fluid Dynamics Conference https://doi.org/10.2514/6.2011-3770	conference	June 2012
An Overview of Performance Portability in the Uintah Runtime System through the Use of Kokkos Sunderland, Daniel; Peterson, Brad; Schmidt, John 2016 Second International Workshop on Extreme Scale Programming Models and Middleware (ESPM2), 2016 Second International Workshop on Extreme Scale Programming Models and Middlewar (ESPM2) https://doi.org/10.1109/ESPM2.2016.012	conference	November 2016
Using hybrid parallelism to improve memory use in the Uintah framework Meng, Qingyu; Berzins, Martin; Schmidt, John Proceedings of the 2011 TeraGrid Conference on Extreme Digital Discovery - TG '11 https://doi.org/10.1145/2016741.2016767	conference	January 2011
A survey of high level frameworks in block-structured adaptive mesh refinement packages Dubey, Anshu; Almgren, Ann; Bell, John Journal of Parallel and Distributed Computing, Vol. 74, Issue 12 https://doi.org/10.1016/j.jpdc.2014.07.001	journal	December 2014
Parallelization of the P-1 Radiation Model Krishnamoorthy, Gautham; Rawat, Rajesh; Smith, Philip J. Numerical Heat Transfer, Part B: Fundamentals, Vol. 49, Issue 1 https://doi.org/10.1080/10407790500344068	journal	January 2006
Kokkos: Enabling manycore performance portability through polymorphic memory access patterns Carter Edwards, H.; Trott, Christian R.; Sunderland, Daniel Journal of Parallel and Distributed Computing, Vol. 74, Issue 12 https://doi.org/10.1016/j.jpdc.2014.07.003	journal	December 2014
A Scalable Algorithm for Radiative Heat Transfer Using Reverse Monte Carlo Ray Tracing Humphrey, Alan; Harman, Todd; Berzins, Martin Lecture Notes in Computer Science https://doi.org/10.1007/978-3-319-20119-1_16	book	January 2015
An Evaluation of Emerging Many-Core Parallel Programming Models Martineau, Matt; McIntosh-Smith, Simon; Boulton, Mike Proceedings of the 7th International Workshop on Programming Models and Applications for Multicores and Manycores - PMAM'16 https://doi.org/10.1145/2883404.2883420	conference	January 2016
Fast Parallel Algorithms for Short-Range Molecular Dynamics Plimpton, Steve Journal of Computational Physics, Vol. 117, Issue 1 https://doi.org/10.1006/jcph.1995.1039	journal	March 1995
Exploring Use of the Reserved Core Holmen, John; Humphrey, Alan; Berzins, Martin High Performance Parallelism Pearls https://doi.org/10.1016/B978-0-12-803819-2.00010-0	book	January 2015
Legion: Expressing locality and independence with logical regions Bauer, Michael; Treichler, Sean; Slaughter, Elliott 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2012.71	conference	November 2012
Investigating applications portability with the Uintah DAG-based runtime system on PetaScale supercomputers Meng, Qingyu; Humphrey, Alan; Schmidt, John Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13 https://doi.org/10.1145/2503210.2503250	conference	January 2013
Preliminary experiences with the uintah framework on Intel Xeon Phi and stampede Meng, Qingyu; Humphrey, Alan; Schmidt, John Proceedings of the Conference on Extreme Science and Engineering Discovery Environment Gateway to Discovery - XSEDE '13 https://doi.org/10.1145/2484762.2484779	conference	January 2013
CHARM++: a portable concurrent object oriented system based on C++ Kale, Laxmikant V.; Krishnan, Sanjeev Proceedings of the eighth annual conference on Object-oriented programming systems, languages, and applications - OOPSLA '93 https://doi.org/10.1145/165854.165874	conference	January 1993
Parallel Computations of Radiative heat Transfer Using the Discrete Ordinates Method Krishnamoorthy, Gautham; Rawat, Rajesh; Smith, Philip J. Numerical Heat Transfer, Part B: Fundamentals, Vol. 47, Issue 1 https://doi.org/10.1080/10407790490487451	journal	December 2004
Uintah: a scalable framework for hazard analysis Berzins, Martin; Luitjens, Justin; Meng, Qingyu Proceedings of the 2010 TeraGrid Conference on - TG '10 https://doi.org/10.1145/1838574.1838577	conference	January 2010
Spatial Domain-Based Parallelism in Large-Scale, Participating-Media, Radiative Transport Applications Burns, Shawn P.; Christen, Mark A. Numerical Heat Transfer, Part B: Fundamentals, Vol. 31, Issue 4 https://doi.org/10.1080/10407799708915117	journal	June 1997
Extending the Uintah Framework through the Petascale Modeling of Detonation in Arrays of High Explosive Devices Berzins, Martin; Beckvermit, Jacqueline; Harman, Todd SIAM Journal on Scientific Computing, Vol. 38, Issue 5 https://doi.org/10.1137/15M1023270	journal	January 2016
Radiation modeling using the Uintah heterogeneous CPU/GPU runtime system Humphrey, Alan; Meng, Qingyu; Berzins, Martin Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment on Bridging from the eXtreme to the campus and beyond - XSEDE '12 https://doi.org/10.1145/2335755.2335791	conference	January 2012
An overview of the Trilinos project Heroux, Michael A.; Phipps, Eric T.; Salinger, Andrew G. ACM Transactions on Mathematical Software, Vol. 31, Issue 3 https://doi.org/10.1145/1089014.1089021	journal	September 2005

Similar Records

An Overview of Performance Portability in the Uintah Runtime System through the Use of Kokkos

Conference · Tue Nov 01 00:00:00 EDT 2016 · 2016 Second International Workshop on Extreme Scale Programming Models and Middlewar (ESPM2) · OSTI ID:1756094

Making Uintah Performance Portable for Department of Energy Exascale Testbeds

Conference · Mon Apr 01 00:00:00 EDT 2024 · OSTI ID:2345338

Lessons Learned and Scalability Achieved When Porting Uintah to DOE Exascale Systems

Conference · Sun Jun 01 00:00:00 EDT 2025 · OSTI ID:2573585

Related Subjects

97 MATHEMATICS AND COMPUTING
Hybrid Parallelism
Knights Landing
Kokkos
MIC Parallel
Many-Core
Portability
Radiation Modeling
Reverse Monte-Carlo Ray Tracing Scalability
Stampede
Uintah
Xeon Phi

Improving Uintah's Scalability Through the Use of Portable Kokkos-Based Data Parallel Tasks

Citation Formats

References (25)

Similar Records

Related Subjects