Dynamic load balancing for a mesh‐based scientific application

Zhai, Keke; Banerjee, Tania; Zwick, David; Hackl, Jason; Koneru, Rahul; Ranka, Sanjay

doi:10.1002/cpe.5626

Title: Dynamic load balancing for a mesh‐based scientific application

Journal Article · 05 January 2020 · Concurrency and Computation. Practice and Experience

DOI: https://doi.org/10.1002/cpe.5626 · OSTI ID:1581451

^[1]; Banerjee, Tania ^[1]; Zwick, David ^[2]; Hackl, Jason ^[2]; Koneru, Rahul ^[2]; Ranka, Sanjay ^[1]

Department of Computer &, Information Science &, Engineering University of Florida Gainesville Florida
Department of Mechanical &, Aerospace Engineering University of Florida Gainesville Florida

Summary CMT‐nek is a new scientific application for performing high fidelity predictive simulations of particle‐laden, explosively dispersed turbulent flows. CMT‐nek is compute‐intensive and targeted for deployment on exascale platforms. The moving particles are the primary source of load imbalance when the application is executed on parallel processors. In a demonstration problem, all the particles are initially in a closed container until a detonation occurs and the particles move apart. If all processors get an equal share of the fluid domain, then only some of the processors get sections of the domain that are initially laden with particles, leading to disparate loads on the processors. To eliminate load imbalance in different processors and to speed up the makespan, we present different load‐balancing algorithms for CMT‐nek on large‐scale multicore platforms. The load on a processor is determined using different techniques. The performance of the different load‐balancing algorithms is compared, and the associated overheads are analyzed. Evaluations of the application with and without load‐balancing are conducted, and these show that with load‐balancing, simulation time becomes faster by a factor of up to 9.97. The performance was further improved by a factor of up to 1.42 using machine‐learning–based algorithms.

View Accepted Manuscript (Publisher)

Cite

Export

Save

Sponsoring Organization:: USDOE

OSTI ID:: 1581451

Journal Information:: Concurrency and Computation. Practice and Experience, Journal Name: Concurrency and Computation. Practice and Experience Journal Issue: 9 Vol. 32; ISSN 1532-0626

Publisher:: Wiley Blackwell (John Wiley & Sons)Copyright Statement

Country of Publication:: United Kingdom

Language:: English

References (36)

Dynamic Load Balancing for a 2D Concurrent Plasma PIC Code Ferraro, Robert D.; Liewer, Paulett C.; Decyk, Viktor K. Journal of Computational Physics, Vol. 109, Issue 2 https://doi.org/10.1006/jcph.1993.1221	journal	December 1993
Parallel Remapping of Adaptive Problems Ou, Chao-Wei; Ranka, Sanjay Journal of Parallel and Distributed Computing, Vol. 42, Issue 2 https://doi.org/10.1006/jpdc.1997.1314	journal	May 1997
Fast Parallel Direct Solvers for Coarse Grid Problems Tufo, H. M.; Fischer, P. F. Journal of Parallel and Distributed Computing, Vol. 61, Issue 2 https://doi.org/10.1006/jpdc.2000.1676	journal	February 2001
Dynamic Load Balancing Based on Rectilinear Partitioning in Particle-in-Cell Plasma Simulation Surmin, Igor; Bashinov, Alexei; Bastrakov, Sergey Lecture Notes in Computer Science https://doi.org/10.1007/978-3-319-21909-7_12	book	January 2015
Spectral Methods: Evolution to Complex Geometrics and Applications to Fluid Dynamics Canuto, Claudio; Quarteroni, Alfio; Hussaini, M. Yousuff Scientific Computation https://doi.org/10.1007/978-3-540-30728-0	book	January 2007
Fast and parallel mapping algorithms for irregular problems Ou, Chao-Wei; Ranka, Sanjay; Fox, Geoffrey The Journal of Supercomputing, Vol. 10, Issue 2 https://doi.org/10.1007/BF00130706	journal	January 1996
A Dynamic Load Balancing Framework for Real-time Applications in Message Passing Systems El Kabbany, Ghada F.; Wanas, Nayer M.; Hegazi, Nadia H. International Journal of Parallel Programming, Vol. 39, Issue 2 https://doi.org/10.1007/s10766-010-0134-5	journal	May 2010
A load index and load balancing algorithm for heterogeneous clusters Bosque, Jose Luis; Toharia, Pablo; Robles, Oscar D. The Journal of Supercomputing, Vol. 65, Issue 3 https://doi.org/10.1007/s11227-013-0881-3	journal	February 2013
Analysis of scalable data-privatization threading algorithms for hybrid MPI/OpenMP parallelization of molecular dynamics Kunaseth, Manaschai; Richards, David F.; Glosli, James N. The Journal of Supercomputing, Vol. 66, Issue 1 https://doi.org/10.1007/s11227-013-0915-x	journal	April 2013
Energy efficiency of load balancing for data-parallel applications in heterogeneous systems Pérez, Borja; Stafford, Esteban; Bosque, José Luis The Journal of Supercomputing, Vol. 73, Issue 1 https://doi.org/10.1007/s11227-016-1864-y	journal	September 2016
Software support for irregular and loosely synchronous problems Choudhary, A.; Fox, G.; Hiranandani, S. Computing Systems in Engineering, Vol. 3, Issue 1-4 https://doi.org/10.1016/0956-0521(92)90093-X	journal	January 1992
A load-balancing algorithm for a parallel electromagnetic particle-in-cell code Plimpton, Steven J.; Seidel, David B.; Pasik, Michael F. Computer Physics Communications, Vol. 152, Issue 3 https://doi.org/10.1016/S0010-4655(02)00795-6	journal	May 2003
Honey bee behavior inspired load balancing of tasks in cloud computing environments L. D., Dhinesh Babu; Venkata Krishna, P. Applied Soft Computing, Vol. 13, Issue 5 https://doi.org/10.1016/j.asoc.2013.01.025	journal	May 2013
Highly scalable SFC-based dynamic load balancing and its application to atmospheric modeling Lieber, Matthias; Nagel, Wolfgang E. Future Generation Computer Systems, Vol. 82 https://doi.org/10.1016/j.future.2017.04.042	journal	May 2018
The Plasma Simulation Code: A modern particle-in-cell code with patch-based load-balancing Germaschewski, Kai; Fox, William; Abbott, Stephen Journal of Computational Physics, Vol. 318 https://doi.org/10.1016/j.jcp.2016.05.013	journal	August 2016
High-Order Methods for Incompressible Fluid Flow Deville, M. O.; Fischer, P. F.; Mund, E. H. Cambridge University Press https://doi.org/10.1017/CBO9780511546792	book	January 2009
GROMACS 4: Algorithms for Highly Efficient, Load-Balanced, and Scalable Molecular Simulation Hess, Berk; Kutzner, Carsten; van der Spoel, David Journal of Chemical Theory and Computation, Vol. 4, Issue 3 https://doi.org/10.1021/ct700301q	journal	February 2008
Automatic data distribution and load balancing with space-filling curves: implementation in CONQUEST Brázdová, V.; Bowler, D. R. Journal of Physics: Condensed Matter, Vol. 20, Issue 27 https://doi.org/10.1088/0953-8984/20/27/275223	journal	June 2008
Total variation diminishing Runge-Kutta schemes Gottlieb, Sigal; Shu, Chi-Wang Mathematics of Computation of the American Mathematical Society, Vol. 67, Issue 221 https://doi.org/10.1090/S0025-5718-98-00913-2	journal	January 1998
Parallel construction of multidimensional binary search trees Al-Furajh, I.; Aluru, S.; Goil, S. IEEE Transactions on Parallel and Distributed Systems, Vol. 11, Issue 2 https://doi.org/10.1109/71.841750	journal	January 2000
Automated Load Balancing Invocation Based on Application Characteristics Menon, Harshitha; Jain, Nikhil; Zheng, Gengbin 2012 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2012.61	conference	September 2012
CMT-Bone — A Proxy Application for Compressible Multiphase Turbulent Flows Banerjee, Tania; Hackl, Jason; Shringarpure, Mrugesh 2016 IEEE 23rd International Conference on High Performance Computing (HiPC) https://doi.org/10.1109/HiPC.2016.029	conference	December 2016
Hierarchical Load Balancing for Charm++ Applications on Large Supercomputers Zheng, Gengbin; Meneses, Esteban; Bhatele, Abhinav 2010 International Conference on Parallel Processing Workshops (ICPPW), 2010 39th International Conference on Parallel Processing Workshops https://doi.org/10.1109/ICPPW.2010.65	conference	September 2010
A genetic algorithm based autotuning approach for performance and energy optimization Banerjee, Tania; Ranka, Sanjay 2015 Sixth International Green and Sustainable Computing Conference (IGSC) https://doi.org/10.1109/IGCC.2015.7393708	conference	December 2015
Dynamic Load Balancing for Unstructured Meshes on Space-Filling Curves Harlacher, Daniel F.; Klimach, Harald; Roller, Sabine 2012 26th IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum https://doi.org/10.1109/IPDPSW.2012.207	conference	May 2012
Barycentric Lagrange Interpolation Berrut, Jean-Paul; Trefethen, Lloyd N. SIAM Review, Vol. 46, Issue 3 https://doi.org/10.1137/S0036144502417715	journal	January 2004
OhHelp: a scalable domain-decomposing dynamic load balancing for particle-in-cell simulations Nakashima, Hiroshi; Miyake, Yohei; Usui, Hideyuki Proceedings of the 23rd international conference on Conference on Supercomputing - ICS '09 https://doi.org/10.1145/1542275.1542293	conference	January 2009
Dynamic topology aware load balancing algorithms for molecular dynamics applications Bhatelé, Abhinav; Kalé, Laxmikant V.; Kumar, Sameer Proceedings of the 23rd international conference on Conference on Supercomputing - ICS '09 https://doi.org/10.1145/1542275.1542295	conference	January 2009
Quantifying load imbalance on virtualized enterprise servers Arzuaga, Emmanuel; Kaeli, David R. Proceedings of the first joint WOSP/SIPEW international conference on Performance engineering - WOSP/SIPEW '10 https://doi.org/10.1145/1712605.1712641	conference	January 2010
Towards energy-aware scheduling in data centers using machine learning Berral, Josep Ll.; Goiri, Íñigo; Nou, Ramón Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking - e-Energy '10 https://doi.org/10.1145/1791314.1791349	conference	January 2010
Architecture-independent locality-improving transformations of computational graphs embedded in k -dimensions Ou, Chao-Wei; Gunwani, Manoj; Ranka, Sanjay Proceedings of the 9th international conference on Supercomputing - ICS '95 https://doi.org/10.1145/224538.224573	conference	January 1995
Quantifying the effectiveness of load balance algorithms Pearce, Olga; Gamblin, Todd; de Supinski, Bronis R. Proceedings of the 26th ACM international conference on Supercomputing - ICS '12 https://doi.org/10.1145/2304576.2304601	conference	January 2012
Load balancing n-body simulations with highly non-uniform density Pearce, Olga; Gamblin, Todd; de Supinski, Bronis R. Proceedings of the 28th ACM international conference on Supercomputing - ICS '14 https://doi.org/10.1145/2597652.2597659	conference	January 2014
POSTER: Automated Load Balancer Selection Based on Application Characteristics Menon, Harshitha; Chandrasekar, Kavitha; Kale, Laxmikant V. ACM SIGPLAN Notices, Vol. 52, Issue 8 https://doi.org/10.1145/3155284.3019033	journal	January 2017
Dynamic Load Balancing for Compressible Multiphase Turbulence Zhai, Keke; Banerjee, Tania; Zwick, David Proceedings of the 2018 International Conference on Supercomputing - ICS '18 https://doi.org/10.1145/3205289.3205304	conference	January 2018
Terascale spectral element algorithms and implementations Tufo, H. M.; Fischer, P. F. Proceedings of the 1999 ACM/IEEE conference on Supercomputing (CDROM) - Supercomputing '99 https://doi.org/10.1145/331532.331599	conference	January 1999

Similar Records

Dynamic load balance scheme for the DSMC algorithm

Journal Article · 2014 · AIP Conference Proceedings · OSTI ID:22390562

Li, Jin; Geng, Xiangren; Jiang, Dingwu; +1 more

Dynamic Load Balancing of Parallel Monte Carlo Transport Calculations

Conference · 2005 · OSTI ID:15015938

O'Brien, Matthew; Taylor, Janine; Procassini, Richard

Load Balancing of Parallel Monte Carlo Transport Calculations

Conference · 2005 · OSTI ID:877821

Procassini, R. J.; O'Brien, M. J.; Taylor, J. M.

Title: Dynamic load balancing for a mesh‐based scientific application

Citation Formats

References (36)

Similar Records

Related Subjects