Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies

Smith, Cameron W.; Rasquin, Michel; Ibanez, Dan; Jansen, Kenneth E.; Shephard, Mark S.

doi:10.1137/15M1027814

Title: Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies

Journal Article · 13 February 2018 · SIAM Journal on Scientific Computing

DOI: https://doi.org/10.1137/15M1027814 · OSTI ID:1438033

Smith, Cameron W. ^[1]; Rasquin, Michel ^[2]; Ibanez, Dan ^[1]; Jansen, Kenneth E. ^[3]; Shephard, Mark S. ^[1]

Rensselaer Polytechnic Inst., Troy, NY (United States). Scientific Computation Research Center (SCOREC)
Univ. of Colorado, Boulder, CO (United States); Cenaero, Gosselies (Belgium)
Univ. of Colorado, Boulder, CO (United States)

The scalability of unstructured mesh based applications depends on partitioning methods that quickly balance the computational work while reducing communication costs. Zhou et al. [SIAM J. Sci. Comput., 32 (2010), pp. 3201{3227; J. Supercomput., 59 (2012), pp. 1218{1228] demonstrated the combination of (hyper)graph methods with vertex and element partition improvement for PHASTA CFD scaling to hundreds of thousands of processes. Our work generalizes partition improvement to support balancing combinations of all the mesh entity dimensions (vertices, edges, faces, regions) in partitions with imbalances exceeding 70%. Improvement results are then presented for multiple entity dimensions on up to one million processes on meshes with over 12 billion tetrahedral elements.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF); Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States). Frameworks, Algorithms, and Scalable Technologies for Mathematics (FASTMath)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); Rensselaer Polytechnic Inst., Troy, NY (United States); Simmetrix Inc., Clifton Park, NY (United States)

Grant/Contract Number:: SC0013919; SC00066117

OSTI ID:: 1438033

Journal Information:: SIAM Journal on Scientific Computing, Vol. 40, Issue 1; ISSN 1064-8275

Publisher:: SIAMCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 8 works

Citation information provided by
Web of Science

References (43)

Multi-level direct K-way hypergraph partitioning with multiple constraints and fixed vertices Aykanat, Cevdet; Cambazoglu, B. Barla; Uçar, Bora Journal of Parallel and Distributed Computing, Vol. 68, Issue 5 https://doi.org/10.1016/j.jpdc.2007.09.006	journal	May 2008
A General Topology-Based mesh data Structure Beall, Mark W.; Shephard, Mark S. International Journal for Numerical Methods in Engineering, Vol. 40, Issue 9 https://doi.org/10.1002/(SICI)1097-0207(19970515)40:9<1573::AID-NME128>3.0.CO;2-9	journal	May 1997
A new analytical method for parallel, diffusion-type load balancing Berenbrink, Petra; Friedetzky, Tom; Hu, Zengjian Journal of Parallel and Distributed Computing, Vol. 69, Issue 1 https://doi.org/10.1016/j.jpdc.2008.05.005	journal	January 2009
A Partitioning Strategy for Nonuniform Problems on Multiprocessors IEEE Transactions on Computers, Vol. C-36, Issue 5 https://doi.org/10.1109/TC.1987.1676942	journal	May 1987
Hypergraph-partitioning-based decomposition for parallel sparse-matrix vector multiplication Catalyurek, U. V.; Aykanat, C. IEEE Transactions on Parallel and Distributed Systems, Vol. 10, Issue 7 https://doi.org/10.1109/71.780863	journal	July 1999
A repartitioning hypergraph model for dynamic load balancing Catalyurek, Umit V.; Boman, Erik G.; Devine, Karen D. Journal of Parallel and Distributed Computing, Vol. 69, Issue 8 https://doi.org/10.1016/j.jpdc.2009.04.011	journal	August 2009
Dynamic load balancing for distributed memory multiprocessors Cybenko, George Journal of Parallel and Distributed Computing, Vol. 7, Issue 2 https://doi.org/10.1016/0743-7315(89)90021-X	journal	October 1989
Multi-Jagged: A Scalable Parallel Spatial Partitioning Algorithm Deveci, Mehmet; Rajamanickam, Sivasankaran; Devine, Karen D. IEEE Transactions on Parallel and Distributed Systems, Vol. 27, Issue 3 https://doi.org/10.1109/TPDS.2015.2412545	journal	March 2016
Zoltan data management services for parallel dynamic applications Devine, K.; Boman, E.; Heaphy, R. Computing in Science & Engineering, Vol. 4, Issue 2 https://doi.org/10.1109/5992.988653	journal	January 2002
New challenges in dynamic load balancing Devine, Karen D.; Boman, Erik G.; Heaphy, Robert T. Applied Numerical Mathematics, Vol. 52, Issue 2-3 https://doi.org/10.1016/j.apnum.2004.08.028	journal	February 2005
Shape-optimized mesh partitioning and load balancing for parallel adaptive FEM Diekmann, Ralf; Preis, Robert; Schlimbach, Frank Parallel Computing, Vol. 26, Issue 12 https://doi.org/10.1016/S0167-8191(00)00043-0	journal	November 2000
Generation of finite difference formulas on arbitrarily spaced grids Fornberg, Bengt Mathematics of Computation, Vol. 51, Issue 184 https://doi.org/10.1090/S0025-5718-1988-0935077-0	journal	January 1988
Software components for parallel multiscale simulation: an example with LAMMPS FrantzDale, Benjamin; Plimpton, Steven J.; Shephard, Mark S. Engineering with Computers, Vol. 26, Issue 2 https://doi.org/10.1007/s00366-009-0156-z	journal	December 2009
An Algorithm for Reducing the Bandwidth and Profile of a Sparse Matrix Gibbs, Norman E.; Poole, Jr., William G.; Stockmeyer, Paul K. SIAM Journal on Numerical Analysis, Vol. 13, Issue 2 https://doi.org/10.1137/0713023	journal	April 1976
Dynamic load balancing in computational mechanics Hendrickson, Bruce; Devine, Karen Computer Methods in Applied Mechanics and Engineering, Vol. 184, Issue 2-4 https://doi.org/10.1016/S0045-7825(99)00241-8	journal	April 2000
An improved diffusion algorithm for dynamic load balancing Hu, Y. F.; Blake, R. J. Parallel Computing, Vol. 25, Issue 4 https://doi.org/10.1016/S0167-8191(99)00002-2	journal	April 1999
An optimal migration algorithm for dynamic load balancing Hu, Y. F.; Blake, R. J.; Emerson, D. R. Concurrency: Practice and Experience, Vol. 10, Issue 6 https://doi.org/10.1002/(SICI)1096-9128(199805)10:6<467::AID-CPE325>3.0.CO;2-A	journal	May 1998
Hybrid MPI-thread parallelization of adaptive mesh operations Ibanez, Dan; Dunn, Ian; Shephard, Mark S. Parallel Computing, Vol. 52 https://doi.org/10.1016/j.parco.2016.01.003	journal	February 2016
PUMI: Parallel Unstructured Mesh Infrastructure Ibanez, Daniel A.; Seol, E. Seegyoung; Smith, Cameron W. ACM Transactions on Mathematical Software, Vol. 42, Issue 3 https://doi.org/10.1145/2814935	journal	May 2016
A generalized-α method for integrating the filtered Navier–Stokes equations with a stabilized finite element method Jansen, Kenneth E.; Whiting, Christian H.; Hulbert, Gregory M. Computer Methods in Applied Mechanics and Engineering, Vol. 190, Issue 3-4 https://doi.org/10.1016/S0045-7825(00)00203-6	journal	October 2000
Parallel Multilevel series k-Way Partitioning Scheme for Irregular Graphs Karypis, George; Kumar, Vipin SIAM Review, Vol. 41, Issue 2 https://doi.org/10.1137/S0036144598334138	journal	January 1999
An Efficient Heuristic Procedure for Partitioning Graphs Kernighan, B. W.; Lin, S. Bell System Technical Journal, Vol. 49, Issue 2 https://doi.org/10.1002/j.1538-7305.1970.tb01770.x	journal	February 1970
Graph partitioning and disturbed diffusion Meyerhenke, Henning; Monien, Burkhard; Schamberger, Stefan Parallel Computing, Vol. 35, Issue 10-11 https://doi.org/10.1016/j.parco.2009.09.006	journal	October 2009
Attribute Management System for Engineering Analysis O'Bara, Robert M.; Beall, Mark W.; Shephard, Mark S. Engineering with Computers, Vol. 18, Issue 4 https://doi.org/10.1007/s003660200030	journal	November 2002
Neighborhood communication paradigm to increase scalability in large-scale dynamic scientific applications Ovcharenko, Aleksandr; Ibanez, Daniel; Delalondre, Fabien Parallel Computing, Vol. 38, Issue 3 https://doi.org/10.1016/j.parco.2011.10.013	journal	March 2012
Scalable Implicit Flow Solver for Realistic Wing Simulations with Flow Control Rasquin, Michel; Smith, Cameron; Chitale, Kedar Computing in Science & Engineering, Vol. 16, Issue 6 https://doi.org/10.1109/MCSE.2014.75	journal	November 2014
Efficient anisotropic adaptive discretization of the cardiovascular system Sahni, O.; Müller, J.; Jansen, K. E. Computer Methods in Applied Mechanics and Engineering, Vol. 195, Issue 41-43 https://doi.org/10.1016/j.cma.2005.10.018	journal	August 2006
Load-balancing spatially located computations using rectangular partitions Saule, Erik; Baş, Erdeniz Ö.; Çatalyürek, Ümit V. Journal of Parallel and Distributed Computing, Vol. 72, Issue 10 https://doi.org/10.1016/j.jpdc.2012.05.013	journal	October 2012
Partitioning finite element meshes using space-filling curves Schamberger, Stefan; Wierum, Jens-Michael Future Generation Computer Systems, Vol. 21, Issue 5 https://doi.org/10.1016/j.future.2004.05.018	journal	May 2005
Multilevel Diffusion Schemes for Repartitioning of Adaptive Meshes Schloegel, Kirk; Karypis, George; Kumar, Vipin Journal of Parallel and Distributed Computing, Vol. 47, Issue 2 https://doi.org/10.1006/jpdc.1997.1410	journal	December 1997
Wavefront diffusion and LMSR: algorithms for dynamic repartitioning of adaptive meshes Schloegel, K.; Karypis, G.; Kumar, V. IEEE Transactions on Parallel and Distributed Systems, Vol. 12, Issue 5 https://doi.org/10.1109/71.926167	journal	May 2001
Parallel static and dynamic multi-constraint graph partitioning Schloegel, Kirk; Karypis, George; Kumar, Vipin Concurrency and Computation: Practice and Experience, Vol. 14, Issue 3 https://doi.org/10.1002/cpe.605	journal	January 2002
Toward simulation-based design Shephard, Mark S.; Beall, Mark W.; O'Bara, Robert M. Finite Elements in Analysis and Design, Vol. 40, Issue 12 https://doi.org/10.1016/j.finel.2003.11.004	journal	July 2004
Bringing HPC to Engineering Innovation Shephard, Mark S.; Smith, Cameron; Kolb, John E. Computing in Science & Engineering, Vol. 15, Issue 1 https://doi.org/10.1109/MCSE.2012.128	journal	January 2013
Partitioning of unstructured problems for parallel processing Simon, H. D. Computing Systems in Engineering, Vol. 2, Issue 2-3 https://doi.org/10.1016/0956-0521(91)90014-V	journal	January 1991
Programming the Hilbert curve Skilling, John BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: 23rd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, AIP Conference Proceedings https://doi.org/10.1063/1.1751381	conference	January 2004
Complex Network Partitioning Using Label Propagation Slota, George M.; Madduri, Kamesh; Rajamanickam, Sivasankaran SIAM Journal on Scientific Computing, Vol. 38, Issue 5 https://doi.org/10.1137/15M1026183	journal	January 2016
A study of the factorization fill-in for a parallel implementation of the finite element method Taylor, Valerie E.; Nour-Omid, Bahram International Journal for Numerical Methods in Engineering, Vol. 37, Issue 22 https://doi.org/10.1002/nme.1620372205	journal	November 1994
A stabilized finite element method for the incompressible Navier-Stokes equations using a hierarchical basis Whiting, Christian H.; Jansen, Kenneth E. International Journal for Numerical Methods in Fluids, Vol. 35, Issue 1 https://doi.org/10.1002/1097-0363(20010115)35:1<93::AID-FLD85>3.0.CO;2-G	journal	January 2001
Strategies for dynamic load balancing on highly parallel computers Willebeek-LeMair, M. H.; Reeves, A. P. IEEE Transactions on Parallel and Distributed Systems, Vol. 4, Issue 9 https://doi.org/10.1109/71.243526	journal	January 1993
Performance of dynamic load balancing algorithms for unstructured mesh calculations Williams, Roy D. Concurrency: Practice and Experience, Vol. 3, Issue 5 https://doi.org/10.1002/cpe.4330030502	journal	October 1991
Controlling Unstructured Mesh Partitions for Massively Parallel Simulations Zhou, Min; Sahni, Onkar; Devine, Karen D. SIAM Journal on Scientific Computing, Vol. 32, Issue 6 https://doi.org/10.1137/090777323	journal	January 2010
Unstructured mesh partition improvement for implicit finite element at extreme scale Zhou, Min; Sahni, Onkar; Xie, Ting The Journal of Supercomputing, Vol. 59, Issue 3 https://doi.org/10.1007/s11227-010-0521-0	journal	December 2010

Cited By (4)

In-memory integration of existing software components for parallel adaptive unstructured mesh workflows: In-memory integration of existing software components for parallel adaptive unstructured mesh workflows Smith, Cameron W.; Granzow, Brian; Diamond, Gerrett Concurrency and Computation: Practice and Experience, Vol. 30, Issue 18 https://doi.org/10.1002/cpe.4510	journal	April 2018
The PHASTA Science Gateway: Web-based Execution of Adaptive Computational Fluid Dynamics Simulations Smith, Cameron W.; Abeysinghe, Eroma PEARC17: Practice and Experience in Advanced Research Computing 2017, Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact https://doi.org/10.1145/3093338.3104151	conference	July 2017
An automated approach for parallel adjoint-based error estimation and mesh adaptation Granzow, Brian N.; Oberai, Assad A.; Shephard, Mark S. Engineering with Computers, Vol. 36, Issue 3 https://doi.org/10.1007/s00366-019-00890-2	journal	December 2019
Interface Tracking Investigation of Geometric Effects on the Bubbly Flow in PWR Subchannels Fang, Jun; Cambareri, Joseph J.; Rasquin, Michel Nuclear Science and Engineering, Vol. 193, Issue 1-2 https://doi.org/10.1080/00295639.2018.1499280	journal	August 2018