Dynamic load balancing for a mesh‐based scientific application
Abstract
Summary CMT‐nek is a new scientific application for performing high fidelity predictive simulations of particle‐laden, explosively dispersed turbulent flows. CMT‐nek is compute‐intensive and targeted for deployment on exascale platforms. The moving particles are the primary source of load imbalance when the application is executed on parallel processors. In a demonstration problem, all the particles are initially in a closed container until a detonation occurs and the particles move apart. If all processors get an equal share of the fluid domain, then only some of the processors get sections of the domain that are initially laden with particles, leading to disparate loads on the processors. To eliminate load imbalance in different processors and to speed up the makespan, we present different load‐balancing algorithms for CMT‐nek on large‐scale multicore platforms. The load on a processor is determined using different techniques. The performance of the different load‐balancing algorithms is compared, and the associated overheads are analyzed. Evaluations of the application with and without load‐balancing are conducted, and these show that with load‐balancing, simulation time becomes faster by a factor of up to 9.97. The performance was further improved by a factor of up to 1.42 using machine‐learning–based algorithms.
- Authors:
-
- Department of Computer &, Information Science &, Engineering University of Florida Gainesville Florida
- Department of Mechanical &, Aerospace Engineering University of Florida Gainesville Florida
- Publication Date:
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1581451
- Resource Type:
- Publisher's Accepted Manuscript
- Journal Name:
- Concurrency and Computation. Practice and Experience
- Additional Journal Information:
- Journal Name: Concurrency and Computation. Practice and Experience Journal Volume: 32 Journal Issue: 9; Journal ID: ISSN 1532-0626
- Publisher:
- Wiley Blackwell (John Wiley & Sons)
- Country of Publication:
- United Kingdom
- Language:
- English
Citation Formats
Zhai, Keke, Banerjee, Tania, Zwick, David, Hackl, Jason, Koneru, Rahul, and Ranka, Sanjay. Dynamic load balancing for a mesh‐based scientific application. United Kingdom: N. p., 2020.
Web. doi:10.1002/cpe.5626.
Zhai, Keke, Banerjee, Tania, Zwick, David, Hackl, Jason, Koneru, Rahul, & Ranka, Sanjay. Dynamic load balancing for a mesh‐based scientific application. United Kingdom. https://doi.org/10.1002/cpe.5626
Zhai, Keke, Banerjee, Tania, Zwick, David, Hackl, Jason, Koneru, Rahul, and Ranka, Sanjay. Mon .
"Dynamic load balancing for a mesh‐based scientific application". United Kingdom. https://doi.org/10.1002/cpe.5626.
@article{osti_1581451,
title = {Dynamic load balancing for a mesh‐based scientific application},
author = {Zhai, Keke and Banerjee, Tania and Zwick, David and Hackl, Jason and Koneru, Rahul and Ranka, Sanjay},
abstractNote = {Summary CMT‐nek is a new scientific application for performing high fidelity predictive simulations of particle‐laden, explosively dispersed turbulent flows. CMT‐nek is compute‐intensive and targeted for deployment on exascale platforms. The moving particles are the primary source of load imbalance when the application is executed on parallel processors. In a demonstration problem, all the particles are initially in a closed container until a detonation occurs and the particles move apart. If all processors get an equal share of the fluid domain, then only some of the processors get sections of the domain that are initially laden with particles, leading to disparate loads on the processors. To eliminate load imbalance in different processors and to speed up the makespan, we present different load‐balancing algorithms for CMT‐nek on large‐scale multicore platforms. The load on a processor is determined using different techniques. The performance of the different load‐balancing algorithms is compared, and the associated overheads are analyzed. Evaluations of the application with and without load‐balancing are conducted, and these show that with load‐balancing, simulation time becomes faster by a factor of up to 9.97. The performance was further improved by a factor of up to 1.42 using machine‐learning–based algorithms.},
doi = {10.1002/cpe.5626},
journal = {Concurrency and Computation. Practice and Experience},
number = 9,
volume = 32,
place = {United Kingdom},
year = {Mon Jan 06 00:00:00 EST 2020},
month = {Mon Jan 06 00:00:00 EST 2020}
}
https://doi.org/10.1002/cpe.5626
Web of Science
Works referenced in this record:
A load-balancing algorithm for a parallel electromagnetic particle-in-cell code
journal, May 2003
- Plimpton, Steven J.; Seidel, David B.; Pasik, Michael F.
- Computer Physics Communications, Vol. 152, Issue 3
Automated Load Balancing Invocation Based on Application Characteristics
conference, September 2012
- Menon, Harshitha; Jain, Nikhil; Zheng, Gengbin
- 2012 IEEE International Conference on Cluster Computing (CLUSTER)
Hierarchical Load Balancing for Charm++ Applications on Large Supercomputers
conference, September 2010
- Zheng, Gengbin; Meneses, Esteban; Bhatele, Abhinav
- 2010 International Conference on Parallel Processing Workshops (ICPPW), 2010 39th International Conference on Parallel Processing Workshops
Architecture-independent locality-improving transformations of computational graphs embedded in k -dimensions
conference, January 1995
- Ou, Chao-Wei; Gunwani, Manoj; Ranka, Sanjay
- Proceedings of the 9th international conference on Supercomputing - ICS '95
Automatic data distribution and load balancing with space-filling curves: implementation in CONQUEST
journal, June 2008
- Brázdová, V.; Bowler, D. R.
- Journal of Physics: Condensed Matter, Vol. 20, Issue 27
Spectral Methods: Evolution to Complex Geometrics and Applications to Fluid Dynamics
book, January 2007
- Canuto, Claudio; Quarteroni, Alfio; Hussaini, M. Yousuff
- Scientific Computation
Load balancing n-body simulations with highly non-uniform density
conference, January 2014
- Pearce, Olga; Gamblin, Todd; de Supinski, Bronis R.
- Proceedings of the 28th ACM international conference on Supercomputing - ICS '14
GROMACS 4: Algorithms for Highly Efficient, Load-Balanced, and Scalable Molecular Simulation
journal, February 2008
- Hess, Berk; Kutzner, Carsten; van der Spoel, David
- Journal of Chemical Theory and Computation, Vol. 4, Issue 3
Terascale spectral element algorithms and implementations
conference, January 1999
- Tufo, H. M.; Fischer, P. F.
- Proceedings of the 1999 ACM/IEEE conference on Supercomputing (CDROM) - Supercomputing '99
POSTER: Automated Load Balancer Selection Based on Application Characteristics
journal, January 2017
- Menon, Harshitha; Chandrasekar, Kavitha; Kale, Laxmikant V.
- ACM SIGPLAN Notices, Vol. 52, Issue 8
Dynamic Load Balancing for Unstructured Meshes on Space-Filling Curves
conference, May 2012
- Harlacher, Daniel F.; Klimach, Harald; Roller, Sabine
- 2012 26th IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum
Honey bee behavior inspired load balancing of tasks in cloud computing environments
journal, May 2013
- L. D., Dhinesh Babu; Venkata Krishna, P.
- Applied Soft Computing, Vol. 13, Issue 5
CMT-Bone — A Proxy Application for Compressible Multiphase Turbulent Flows
conference, December 2016
- Banerjee, Tania; Hackl, Jason; Shringarpure, Mrugesh
- 2016 IEEE 23rd International Conference on High Performance Computing (HiPC)
Dynamic topology aware load balancing algorithms for molecular dynamics applications
conference, January 2009
- Bhatelé, Abhinav; Kalé, Laxmikant V.; Kumar, Sameer
- Proceedings of the 23rd international conference on Conference on Supercomputing - ICS '09
Dynamic Load Balancing for Compressible Multiphase Turbulence
conference, January 2018
- Zhai, Keke; Banerjee, Tania; Zwick, David
- Proceedings of the 2018 International Conference on Supercomputing - ICS '18
Quantifying load imbalance on virtualized enterprise servers
conference, January 2010
- Arzuaga, Emmanuel; Kaeli, David R.
- Proceedings of the first joint WOSP/SIPEW international conference on Performance engineering - WOSP/SIPEW '10
A load index and load balancing algorithm for heterogeneous clusters
journal, February 2013
- Bosque, Jose Luis; Toharia, Pablo; Robles, Oscar D.
- The Journal of Supercomputing, Vol. 65, Issue 3
Analysis of scalable data-privatization threading algorithms for hybrid MPI/OpenMP parallelization of molecular dynamics
journal, April 2013
- Kunaseth, Manaschai; Richards, David F.; Glosli, James N.
- The Journal of Supercomputing, Vol. 66, Issue 1
A Dynamic Load Balancing Framework for Real-time Applications in Message Passing Systems
journal, May 2010
- El Kabbany, Ghada F.; Wanas, Nayer M.; Hegazi, Nadia H.
- International Journal of Parallel Programming, Vol. 39, Issue 2
Software support for irregular and loosely synchronous problems
journal, January 1992
- Choudhary, A.; Fox, G.; Hiranandani, S.
- Computing Systems in Engineering, Vol. 3, Issue 1-4
Barycentric Lagrange Interpolation
journal, January 2004
- Berrut, Jean-Paul; Trefethen, Lloyd N.
- SIAM Review, Vol. 46, Issue 3
High-Order Methods for Incompressible Fluid Flow
book, January 2009
- Deville, M. O.; Fischer, P. F.; Mund, E. H.
- Cambridge University Press
OhHelp: a scalable domain-decomposing dynamic load balancing for particle-in-cell simulations
conference, January 2009
- Nakashima, Hiroshi; Miyake, Yohei; Usui, Hideyuki
- Proceedings of the 23rd international conference on Conference on Supercomputing - ICS '09
Total variation diminishing Runge-Kutta schemes
journal, January 1998
- Gottlieb, Sigal; Shu, Chi-Wang
- Mathematics of Computation of the American Mathematical Society, Vol. 67, Issue 221
Fast Parallel Direct Solvers for Coarse Grid Problems
journal, February 2001
- Tufo, H. M.; Fischer, P. F.
- Journal of Parallel and Distributed Computing, Vol. 61, Issue 2
Dynamic Load Balancing for a 2D Concurrent Plasma PIC Code
journal, December 1993
- Ferraro, Robert D.; Liewer, Paulett C.; Decyk, Viktor K.
- Journal of Computational Physics, Vol. 109, Issue 2
Highly scalable SFC-based dynamic load balancing and its application to atmospheric modeling
journal, May 2018
- Lieber, Matthias; Nagel, Wolfgang E.
- Future Generation Computer Systems, Vol. 82
Quantifying the effectiveness of load balance algorithms
conference, January 2012
- Pearce, Olga; Gamblin, Todd; de Supinski, Bronis R.
- Proceedings of the 26th ACM international conference on Supercomputing - ICS '12
A genetic algorithm based autotuning approach for performance and energy optimization
conference, December 2015
- Banerjee, Tania; Ranka, Sanjay
- 2015 Sixth International Green and Sustainable Computing Conference (IGSC)
Towards energy-aware scheduling in data centers using machine learning
conference, January 2010
- Berral, Josep Ll.; Goiri, Íñigo; Nou, Ramón
- Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking - e-Energy '10
Parallel Remapping of Adaptive Problems
journal, May 1997
- Ou, Chao-Wei; Ranka, Sanjay
- Journal of Parallel and Distributed Computing, Vol. 42, Issue 2
Fast and parallel mapping algorithms for irregular problems
journal, January 1996
- Ou, Chao-Wei; Ranka, Sanjay; Fox, Geoffrey
- The Journal of Supercomputing, Vol. 10, Issue 2
The Plasma Simulation Code: A modern particle-in-cell code with patch-based load-balancing
journal, August 2016
- Germaschewski, Kai; Fox, William; Abbott, Stephen
- Journal of Computational Physics, Vol. 318
Parallel construction of multidimensional binary search trees
journal, January 2000
- Al-Furajh, I.; Aluru, S.; Goil, S.
- IEEE Transactions on Parallel and Distributed Systems, Vol. 11, Issue 2
Energy efficiency of load balancing for data-parallel applications in heterogeneous systems
journal, September 2016
- Pérez, Borja; Stafford, Esteban; Bosque, José Luis
- The Journal of Supercomputing, Vol. 73, Issue 1