Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies

Smith, Cameron W.; Rasquin, Michel; Ibanez, Dan; Jansen, Kenneth E.; Shephard, Mark S.

doi:10.1137/15M1027814

Title: Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies

Journal Article · Mon Feb 12 23:00:00 EST 2018 · SIAM Journal on Scientific Computing

DOI: https://doi.org/10.1137/15M1027814 · OSTI ID:1438033

Smith, Cameron W. ^[1]; Rasquin, Michel ^[2]; Ibanez, Dan ^[3]; Jansen, Kenneth E. ^[4]; Shephard, Mark S. ^[3]

Rensselaer Polytechnic Inst., Troy, NY (United States). Scientific Computation Research Center (SCOREC); Rensselaer Polytechnic Institute
Univ. of Colorado, Boulder, CO (United States); Cenaero, Gosselies (Belgium)
Rensselaer Polytechnic Inst., Troy, NY (United States). Scientific Computation Research Center (SCOREC)
Univ. of Colorado, Boulder, CO (United States)

The scalability of unstructured mesh based applications depends on partitioning methods that quickly balance the computational work while reducing communication costs. Zhou et al. [SIAM J. Sci. Comput., 32 (2010), pp. 3201{3227; J. Supercomput., 59 (2012), pp. 1218{1228] demonstrated the combination of (hyper)graph methods with vertex and element partition improvement for PHASTA CFD scaling to hundreds of thousands of processes. Our work generalizes partition improvement to support balancing combinations of all the mesh entity dimensions (vertices, edges, faces, regions) in partitions with imbalances exceeding 70%. Improvement results are then presented for multiple entity dimensions on up to one million processes on meshes with over 12 billion tetrahedral elements.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF); Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States). Frameworks, Algorithms, and Scalable Technologies for Mathematics (FASTMath)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); Rensselaer Polytechnic Inst., Troy, NY (United States); Simmetrix Inc., Clifton Park, NY (United States)

Grant/Contract Number:: SC0013919

OSTI ID:: 1438033

Journal Information:: SIAM Journal on Scientific Computing, Journal Name: SIAM Journal on Scientific Computing Journal Issue: 1 Vol. 40; ISSN 1064-8275

Publisher:: SIAMCopyright Statement

Country of Publication:: United States

Language:: English

References (43)

An optimal migration algorithm for dynamic load balancing Hu, Y. F.; Blake, R. J.; Emerson, D. R. Concurrency: Practice and Experience, Vol. 10, Issue 6 https://doi.org/10.1002/(SICI)1096-9128(199805)10:6<467::AID-CPE325>3.0.CO;2-A	journal	May 1998
A General Topology-Based mesh data Structure Beall, Mark W.; Shephard, Mark S. International Journal for Numerical Methods in Engineering, Vol. 40, Issue 9 https://doi.org/10.1002/(SICI)1097-0207(19970515)40:9<1573::AID-NME128>3.0.CO;2-9	journal	May 1997
A stabilized finite element method for the incompressible Navier-Stokes equations using a hierarchical basis Whiting, Christian H.; Jansen, Kenneth E. International Journal for Numerical Methods in Fluids, Vol. 35, Issue 1 https://doi.org/10.1002/1097-0363(20010115)35:1<93::AID-FLD85>3.0.CO;2-G	journal	January 2001
Performance of dynamic load balancing algorithms for unstructured mesh calculations Williams, Roy D. Concurrency: Practice and Experience, Vol. 3, Issue 5 https://doi.org/10.1002/cpe.4330030502	journal	October 1991
Parallel static and dynamic multi-constraint graph partitioning Schloegel, Kirk; Karypis, George; Kumar, Vipin Concurrency and Computation: Practice and Experience, Vol. 14, Issue 3 https://doi.org/10.1002/cpe.605	journal	January 2002
An Efficient Heuristic Procedure for Partitioning Graphs Kernighan, B. W.; Lin, S. Bell System Technical Journal, Vol. 49, Issue 2 https://doi.org/10.1002/j.1538-7305.1970.tb01770.x	journal	February 1970
A study of the factorization fill-in for a parallel implementation of the finite element method Taylor, Valerie E.; Nour-Omid, Bahram International Journal for Numerical Methods in Engineering, Vol. 37, Issue 22 https://doi.org/10.1002/nme.1620372205	journal	November 1994
Multilevel Diffusion Schemes for Repartitioning of Adaptive Meshes Schloegel, Kirk; Karypis, George; Kumar, Vipin Journal of Parallel and Distributed Computing, Vol. 47, Issue 2 https://doi.org/10.1006/jpdc.1997.1410	journal	December 1997
Software components for parallel multiscale simulation: an example with LAMMPS FrantzDale, Benjamin; Plimpton, Steven J.; Shephard, Mark S. Engineering with Computers, Vol. 26, Issue 2 https://doi.org/10.1007/s00366-009-0156-z	journal	December 2009
Attribute Management System for Engineering Analysis O'Bara, Robert M.; Beall, Mark W.; Shephard, Mark S. Engineering with Computers, Vol. 18, Issue 4 https://doi.org/10.1007/s003660200030	journal	November 2002
Unstructured mesh partition improvement for implicit finite element at extreme scale Zhou, Min; Sahni, Onkar; Xie, Ting The Journal of Supercomputing, Vol. 59, Issue 3 https://doi.org/10.1007/s11227-010-0521-0	journal	December 2010
Dynamic load balancing for distributed memory multiprocessors Cybenko, George Journal of Parallel and Distributed Computing, Vol. 7, Issue 2 https://doi.org/10.1016/0743-7315(89)90021-X	journal	October 1989
Partitioning of unstructured problems for parallel processing Simon, H. D. Computing Systems in Engineering, Vol. 2, Issue 2-3 https://doi.org/10.1016/0956-0521(91)90014-V	journal	January 1991
A generalized-α method for integrating the filtered Navier–Stokes equations with a stabilized finite element method Jansen, Kenneth E.; Whiting, Christian H.; Hulbert, Gregory M. Computer Methods in Applied Mechanics and Engineering, Vol. 190, Issue 3-4 https://doi.org/10.1016/S0045-7825(00)00203-6	journal	October 2000
Dynamic load balancing in computational mechanics Hendrickson, Bruce; Devine, Karen Computer Methods in Applied Mechanics and Engineering, Vol. 184, Issue 2-4 https://doi.org/10.1016/S0045-7825(99)00241-8	journal	April 2000
Shape-optimized mesh partitioning and load balancing for parallel adaptive FEM Diekmann, Ralf; Preis, Robert; Schlimbach, Frank Parallel Computing, Vol. 26, Issue 12 https://doi.org/10.1016/S0167-8191(00)00043-0	journal	November 2000
An improved diffusion algorithm for dynamic load balancing Hu, Y. F.; Blake, R. J. Parallel Computing, Vol. 25, Issue 4 https://doi.org/10.1016/S0167-8191(99)00002-2	journal	April 1999
New challenges in dynamic load balancing Devine, Karen D.; Boman, Erik G.; Heaphy, Robert T. Applied Numerical Mathematics, Vol. 52, Issue 2-3 https://doi.org/10.1016/j.apnum.2004.08.028	journal	February 2005
Efficient anisotropic adaptive discretization of the cardiovascular system Sahni, O.; Müller, J.; Jansen, K. E. Computer Methods in Applied Mechanics and Engineering, Vol. 195, Issue 41-43 https://doi.org/10.1016/j.cma.2005.10.018	journal	August 2006
Toward simulation-based design Shephard, Mark S.; Beall, Mark W.; O'Bara, Robert M. Finite Elements in Analysis and Design, Vol. 40, Issue 12 https://doi.org/10.1016/j.finel.2003.11.004	journal	July 2004
Partitioning finite element meshes using space-filling curves Schamberger, Stefan; Wierum, Jens-Michael Future Generation Computer Systems, Vol. 21, Issue 5 https://doi.org/10.1016/j.future.2004.05.018	journal	May 2005
Multi-level direct K-way hypergraph partitioning with multiple constraints and fixed vertices Aykanat, Cevdet; Cambazoglu, B. Barla; Uçar, Bora Journal of Parallel and Distributed Computing, Vol. 68, Issue 5 https://doi.org/10.1016/j.jpdc.2007.09.006	journal	May 2008
A new analytical method for parallel, diffusion-type load balancing Berenbrink, Petra; Friedetzky, Tom; Hu, Zengjian Journal of Parallel and Distributed Computing, Vol. 69, Issue 1 https://doi.org/10.1016/j.jpdc.2008.05.005	journal	January 2009
A repartitioning hypergraph model for dynamic load balancing Catalyurek, Umit V.; Boman, Erik G.; Devine, Karen D. Journal of Parallel and Distributed Computing, Vol. 69, Issue 8 https://doi.org/10.1016/j.jpdc.2009.04.011	journal	August 2009
Load-balancing spatially located computations using rectangular partitions Saule, Erik; Baş, Erdeniz Ö.; Çatalyürek, Ümit V. Journal of Parallel and Distributed Computing, Vol. 72, Issue 10 https://doi.org/10.1016/j.jpdc.2012.05.013	journal	October 2012
Graph partitioning and disturbed diffusion Meyerhenke, Henning; Monien, Burkhard; Schamberger, Stefan Parallel Computing, Vol. 35, Issue 10-11 https://doi.org/10.1016/j.parco.2009.09.006	journal	October 2009
Neighborhood communication paradigm to increase scalability in large-scale dynamic scientific applications Ovcharenko, Aleksandr; Ibanez, Daniel; Delalondre, Fabien Parallel Computing, Vol. 38, Issue 3 https://doi.org/10.1016/j.parco.2011.10.013	journal	March 2012
Hybrid MPI-thread parallelization of adaptive mesh operations Ibanez, Dan; Dunn, Ian; Shephard, Mark S. Parallel Computing, Vol. 52 https://doi.org/10.1016/j.parco.2016.01.003	journal	February 2016
Programming the Hilbert curve Skilling, John BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: 23rd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, AIP Conference Proceedings https://doi.org/10.1063/1.1751381	conference	January 2004
Generation of finite difference formulas on arbitrarily spaced grids Fornberg, Bengt Mathematics of Computation, Vol. 51, Issue 184 https://doi.org/10.1090/S0025-5718-1988-0935077-0	journal	January 1988
Zoltan data management services for parallel dynamic applications Devine, K.; Boman, E.; Heaphy, R. Computing in Science & Engineering, Vol. 4, Issue 2 https://doi.org/10.1109/5992.988653	journal	January 2002
Strategies for dynamic load balancing on highly parallel computers Willebeek-LeMair, M. H.; Reeves, A. P. IEEE Transactions on Parallel and Distributed Systems, Vol. 4, Issue 9 https://doi.org/10.1109/71.243526	journal	January 1993
Hypergraph-partitioning-based decomposition for parallel sparse-matrix vector multiplication Catalyurek, U. V.; Aykanat, C. IEEE Transactions on Parallel and Distributed Systems, Vol. 10, Issue 7 https://doi.org/10.1109/71.780863	journal	July 1999
Wavefront diffusion and LMSR: algorithms for dynamic repartitioning of adaptive meshes Schloegel, K.; Karypis, G.; Kumar, V. IEEE Transactions on Parallel and Distributed Systems, Vol. 12, Issue 5 https://doi.org/10.1109/71.926167	journal	May 2001
Bringing HPC to Engineering Innovation Shephard, Mark S.; Smith, Cameron; Kolb, John E. Computing in Science & Engineering, Vol. 15, Issue 1 https://doi.org/10.1109/MCSE.2012.128	journal	January 2013
Scalable Implicit Flow Solver for Realistic Wing Simulations with Flow Control Rasquin, Michel; Smith, Cameron; Chitale, Kedar Computing in Science & Engineering, Vol. 16, Issue 6 https://doi.org/10.1109/MCSE.2014.75	journal	November 2014
A Partitioning Strategy for Nonuniform Problems on Multiprocessors IEEE Transactions on Computers, Vol. C-36, Issue 5 https://doi.org/10.1109/TC.1987.1676942	journal	May 1987
Multi-Jagged: A Scalable Parallel Spatial Partitioning Algorithm Deveci, Mehmet; Rajamanickam, Sivasankaran; Devine, Karen D. IEEE Transactions on Parallel and Distributed Systems, Vol. 27, Issue 3 https://doi.org/10.1109/TPDS.2015.2412545	journal	March 2016
An Algorithm for Reducing the Bandwidth and Profile of a Sparse Matrix Gibbs, Norman E.; Poole, Jr., William G.; Stockmeyer, Paul K. SIAM Journal on Numerical Analysis, Vol. 13, Issue 2 https://doi.org/10.1137/0713023	journal	April 1976
Controlling Unstructured Mesh Partitions for Massively Parallel Simulations Zhou, Min; Sahni, Onkar; Devine, Karen D. SIAM Journal on Scientific Computing, Vol. 32, Issue 6 https://doi.org/10.1137/090777323	journal	January 2010
Complex Network Partitioning Using Label Propagation Slota, George M.; Madduri, Kamesh; Rajamanickam, Sivasankaran SIAM Journal on Scientific Computing, Vol. 38, Issue 5 https://doi.org/10.1137/15M1026183	journal	January 2016
Parallel Multilevel series k-Way Partitioning Scheme for Irregular Graphs Karypis, George; Kumar, Vipin SIAM Review, Vol. 41, Issue 2 https://doi.org/10.1137/S0036144598334138	journal	January 1999
PUMI: Parallel Unstructured Mesh Infrastructure Ibanez, Daniel A.; Seol, E. Seegyoung; Smith, Cameron W. ACM Transactions on Mathematical Software, Vol. 42, Issue 3 https://doi.org/10.1145/2814935	journal	May 2016

Cited By (4)

In-memory integration of existing software components for parallel adaptive unstructured mesh workflows: In-memory integration of existing software components for parallel adaptive unstructured mesh workflows Smith, Cameron W.; Granzow, Brian; Diamond, Gerrett Concurrency and Computation: Practice and Experience, Vol. 30, Issue 18 https://doi.org/10.1002/cpe.4510	journal	April 2018
An automated approach for parallel adjoint-based error estimation and mesh adaptation Granzow, Brian N.; Oberai, Assad A.; Shephard, Mark S. Engineering with Computers, Vol. 36, Issue 3 https://doi.org/10.1007/s00366-019-00890-2	journal	December 2019
Interface Tracking Investigation of Geometric Effects on the Bubbly Flow in PWR Subchannels Fang, Jun; Cambareri, Joseph J.; Rasquin, Michel Nuclear Science and Engineering, Vol. 193, Issue 1-2 https://doi.org/10.1080/00295639.2018.1499280	journal	August 2018
The PHASTA Science Gateway: Web-based Execution of Adaptive Computational Fluid Dynamics Simulations Smith, Cameron W.; Abeysinghe, Eroma PEARC17: Practice and Experience in Advanced Research Computing 2017, Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact https://doi.org/10.1145/3093338.3104151	conference	July 2017

Similar Records

Robust and efficient overset grid assembly for partitioned unstructured meshes

Journal Article · Fri Feb 28 23:00:00 EST 2014 · Journal of Computational Physics · OSTI ID:22230873

Roget, Beatrice; Sitaraman, Jayanarayanan

Parallel performance optimizations on unstructured mesh-based simulations

Journal Article · Mon Jun 01 00:00:00 EDT 2015 · Procedia Computer Science · OSTI ID:1202396

Sarje, Abhinav; Song, Sukhyun; Jacobsen, Douglas; +5 more

Parallel Performance Optimizations on Unstructured Mesh-based Simulations

Conference · Wed Dec 31 23:00:00 EST 2014 · Procedia Computer Science · OSTI ID:1407282

Sarje, Abhinav; Song, Sukhyun; Jacobsen, Douglas; +5 more

Related Subjects

97 MATHEMATICS AND COMPUTING
dynamic load balancing
graph/hypergraph
partition improvement
unstructured mesh

Title: Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies

Citation Formats

References (43)

Cited By (4)

Similar Records

Related Subjects