DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies

Abstract

The scalability of unstructured mesh based applications depends on partitioning methods that quickly balance the computational work while reducing communication costs. Zhou et al. [SIAM J. Sci. Comput., 32 (2010), pp. 3201{3227; J. Supercomput., 59 (2012), pp. 1218{1228] demonstrated the combination of (hyper)graph methods with vertex and element partition improvement for PHASTA CFD scaling to hundreds of thousands of processes. Our work generalizes partition improvement to support balancing combinations of all the mesh entity dimensions (vertices, edges, faces, regions) in partitions with imbalances exceeding 70%. Improvement results are then presented for multiple entity dimensions on up to one million processes on meshes with over 12 billion tetrahedral elements.

Authors:
 [1];  [2];  [1];  [3];  [1]
  1. Rensselaer Polytechnic Inst., Troy, NY (United States). Scientific Computation Research Center (SCOREC)
  2. Univ. of Colorado, Boulder, CO (United States); Cenaero, Gosselies (Belgium)
  3. Univ. of Colorado, Boulder, CO (United States)
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF); Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States). Frameworks, Algorithms, and Scalable Technologies for Mathematics (FASTMath)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); Rensselaer Polytechnic Inst., Troy, NY (United States); Simmetrix Inc., Clifton Park, NY (United States)
OSTI Identifier:
1438033
Grant/Contract Number:  
SC0013919; SC00066117
Resource Type:
Accepted Manuscript
Journal Name:
SIAM Journal on Scientific Computing
Additional Journal Information:
Journal Volume: 40; Journal Issue: 1; Journal ID: ISSN 1064-8275
Publisher:
SIAM
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; partition improvement; graph/hypergraph; unstructured mesh; dynamic load balancing

Citation Formats

Smith, Cameron W., Rasquin, Michel, Ibanez, Dan, Jansen, Kenneth E., and Shephard, Mark S. Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies. United States: N. p., 2018. Web. doi:10.1137/15M1027814.
Smith, Cameron W., Rasquin, Michel, Ibanez, Dan, Jansen, Kenneth E., & Shephard, Mark S. Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies. United States. https://doi.org/10.1137/15M1027814
Smith, Cameron W., Rasquin, Michel, Ibanez, Dan, Jansen, Kenneth E., and Shephard, Mark S. Tue . "Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies". United States. https://doi.org/10.1137/15M1027814. https://www.osti.gov/servlets/purl/1438033.
@article{osti_1438033,
title = {Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies},
author = {Smith, Cameron W. and Rasquin, Michel and Ibanez, Dan and Jansen, Kenneth E. and Shephard, Mark S.},
abstractNote = {The scalability of unstructured mesh based applications depends on partitioning methods that quickly balance the computational work while reducing communication costs. Zhou et al. [SIAM J. Sci. Comput., 32 (2010), pp. 3201{3227; J. Supercomput., 59 (2012), pp. 1218{1228] demonstrated the combination of (hyper)graph methods with vertex and element partition improvement for PHASTA CFD scaling to hundreds of thousands of processes. Our work generalizes partition improvement to support balancing combinations of all the mesh entity dimensions (vertices, edges, faces, regions) in partitions with imbalances exceeding 70%. Improvement results are then presented for multiple entity dimensions on up to one million processes on meshes with over 12 billion tetrahedral elements.},
doi = {10.1137/15M1027814},
journal = {SIAM Journal on Scientific Computing},
number = 1,
volume = 40,
place = {United States},
year = {2018},
month = {2}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 7 works
Citation information provided by
Web of Science

Figures / Tables:

Fig. 1 Fig. 1: (left) Example of a mesh, (middle) its partition model, and (right) its ownership. Discs and bold segments denote entity ownership.

Save / Share:

Works referenced in this record:

Multi-level direct K-way hypergraph partitioning with multiple constraints and fixed vertices
journal, May 2008

  • Aykanat, Cevdet; Cambazoglu, B. Barla; Uçar, Bora
  • Journal of Parallel and Distributed Computing, Vol. 68, Issue 5
  • DOI: 10.1016/j.jpdc.2007.09.006

A General Topology-Based mesh data Structure
journal, May 1997


A new analytical method for parallel, diffusion-type load balancing
journal, January 2009

  • Berenbrink, Petra; Friedetzky, Tom; Hu, Zengjian
  • Journal of Parallel and Distributed Computing, Vol. 69, Issue 1
  • DOI: 10.1016/j.jpdc.2008.05.005

A Partitioning Strategy for Nonuniform Problems on Multiprocessors
journal, May 1987


Hypergraph-partitioning-based decomposition for parallel sparse-matrix vector multiplication
journal, July 1999

  • Catalyurek, U. V.; Aykanat, C.
  • IEEE Transactions on Parallel and Distributed Systems, Vol. 10, Issue 7
  • DOI: 10.1109/71.780863

A repartitioning hypergraph model for dynamic load balancing
journal, August 2009

  • Catalyurek, Umit V.; Boman, Erik G.; Devine, Karen D.
  • Journal of Parallel and Distributed Computing, Vol. 69, Issue 8
  • DOI: 10.1016/j.jpdc.2009.04.011

Dynamic load balancing for distributed memory multiprocessors
journal, October 1989


Multi-Jagged: A Scalable Parallel Spatial Partitioning Algorithm
journal, March 2016

  • Deveci, Mehmet; Rajamanickam, Sivasankaran; Devine, Karen D.
  • IEEE Transactions on Parallel and Distributed Systems, Vol. 27, Issue 3
  • DOI: 10.1109/TPDS.2015.2412545

Zoltan data management services for parallel dynamic applications
journal, January 2002

  • Devine, K.; Boman, E.; Heaphy, R.
  • Computing in Science & Engineering, Vol. 4, Issue 2
  • DOI: 10.1109/5992.988653

New challenges in dynamic load balancing
journal, February 2005

  • Devine, Karen D.; Boman, Erik G.; Heaphy, Robert T.
  • Applied Numerical Mathematics, Vol. 52, Issue 2-3
  • DOI: 10.1016/j.apnum.2004.08.028

Shape-optimized mesh partitioning and load balancing for parallel adaptive FEM
journal, November 2000


Software components for parallel multiscale simulation: an example with LAMMPS
journal, December 2009

  • FrantzDale, Benjamin; Plimpton, Steven J.; Shephard, Mark S.
  • Engineering with Computers, Vol. 26, Issue 2
  • DOI: 10.1007/s00366-009-0156-z

An Algorithm for Reducing the Bandwidth and Profile of a Sparse Matrix
journal, April 1976

  • Gibbs, Norman E.; Poole, Jr., William G.; Stockmeyer, Paul K.
  • SIAM Journal on Numerical Analysis, Vol. 13, Issue 2
  • DOI: 10.1137/0713023

Dynamic load balancing in computational mechanics
journal, April 2000

  • Hendrickson, Bruce; Devine, Karen
  • Computer Methods in Applied Mechanics and Engineering, Vol. 184, Issue 2-4
  • DOI: 10.1016/S0045-7825(99)00241-8

An improved diffusion algorithm for dynamic load balancing
journal, April 1999


An optimal migration algorithm for dynamic load balancing
journal, May 1998


Hybrid MPI-thread parallelization of adaptive mesh operations
journal, February 2016


PUMI: Parallel Unstructured Mesh Infrastructure
journal, May 2016

  • Ibanez, Daniel A.; Seol, E. Seegyoung; Smith, Cameron W.
  • ACM Transactions on Mathematical Software, Vol. 42, Issue 3
  • DOI: 10.1145/2814935

A generalized-α method for integrating the filtered Navier–Stokes equations with a stabilized finite element method
journal, October 2000

  • Jansen, Kenneth E.; Whiting, Christian H.; Hulbert, Gregory M.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 190, Issue 3-4
  • DOI: 10.1016/S0045-7825(00)00203-6

Parallel Multilevel series k-Way Partitioning Scheme for Irregular Graphs
journal, January 1999


An Efficient Heuristic Procedure for Partitioning Graphs
journal, February 1970


Graph partitioning and disturbed diffusion
journal, October 2009


Attribute Management System for Engineering Analysis
journal, November 2002

  • O'Bara, Robert M.; Beall, Mark W.; Shephard, Mark S.
  • Engineering with Computers, Vol. 18, Issue 4
  • DOI: 10.1007/s003660200030

Neighborhood communication paradigm to increase scalability in large-scale dynamic scientific applications
journal, March 2012


Scalable Implicit Flow Solver for Realistic Wing Simulations with Flow Control
journal, November 2014

  • Rasquin, Michel; Smith, Cameron; Chitale, Kedar
  • Computing in Science & Engineering, Vol. 16, Issue 6
  • DOI: 10.1109/MCSE.2014.75

Efficient anisotropic adaptive discretization of the cardiovascular system
journal, August 2006

  • Sahni, O.; Müller, J.; Jansen, K. E.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 195, Issue 41-43
  • DOI: 10.1016/j.cma.2005.10.018

Load-balancing spatially located computations using rectangular partitions
journal, October 2012

  • Saule, Erik; Baş, Erdeniz Ö.; Çatalyürek, Ümit V.
  • Journal of Parallel and Distributed Computing, Vol. 72, Issue 10
  • DOI: 10.1016/j.jpdc.2012.05.013

Partitioning finite element meshes using space-filling curves
journal, May 2005


Multilevel Diffusion Schemes for Repartitioning of Adaptive Meshes
journal, December 1997

  • Schloegel, Kirk; Karypis, George; Kumar, Vipin
  • Journal of Parallel and Distributed Computing, Vol. 47, Issue 2
  • DOI: 10.1006/jpdc.1997.1410

Wavefront diffusion and LMSR: algorithms for dynamic repartitioning of adaptive meshes
journal, May 2001

  • Schloegel, K.; Karypis, G.; Kumar, V.
  • IEEE Transactions on Parallel and Distributed Systems, Vol. 12, Issue 5
  • DOI: 10.1109/71.926167

Parallel static and dynamic multi-constraint graph partitioning
journal, January 2002

  • Schloegel, Kirk; Karypis, George; Kumar, Vipin
  • Concurrency and Computation: Practice and Experience, Vol. 14, Issue 3
  • DOI: 10.1002/cpe.605

Toward simulation-based design
journal, July 2004

  • Shephard, Mark S.; Beall, Mark W.; O'Bara, Robert M.
  • Finite Elements in Analysis and Design, Vol. 40, Issue 12
  • DOI: 10.1016/j.finel.2003.11.004

Bringing HPC to Engineering Innovation
journal, January 2013

  • Shephard, Mark S.; Smith, Cameron; Kolb, John E.
  • Computing in Science & Engineering, Vol. 15, Issue 1
  • DOI: 10.1109/MCSE.2012.128

Partitioning of unstructured problems for parallel processing
journal, January 1991


Programming the Hilbert curve
conference, January 2004

  • Skilling, John
  • BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: 23rd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, AIP Conference Proceedings
  • DOI: 10.1063/1.1751381

Complex Network Partitioning Using Label Propagation
journal, January 2016

  • Slota, George M.; Madduri, Kamesh; Rajamanickam, Sivasankaran
  • SIAM Journal on Scientific Computing, Vol. 38, Issue 5
  • DOI: 10.1137/15M1026183

A study of the factorization fill-in for a parallel implementation of the finite element method
journal, November 1994

  • Taylor, Valerie E.; Nour-Omid, Bahram
  • International Journal for Numerical Methods in Engineering, Vol. 37, Issue 22
  • DOI: 10.1002/nme.1620372205

A stabilized finite element method for the incompressible Navier-Stokes equations using a hierarchical basis
journal, January 2001


Strategies for dynamic load balancing on highly parallel computers
journal, January 1993

  • Willebeek-LeMair, M. H.; Reeves, A. P.
  • IEEE Transactions on Parallel and Distributed Systems, Vol. 4, Issue 9
  • DOI: 10.1109/71.243526

Performance of dynamic load balancing algorithms for unstructured mesh calculations
journal, October 1991


Controlling Unstructured Mesh Partitions for Massively Parallel Simulations
journal, January 2010

  • Zhou, Min; Sahni, Onkar; Devine, Karen D.
  • SIAM Journal on Scientific Computing, Vol. 32, Issue 6
  • DOI: 10.1137/090777323

Unstructured mesh partition improvement for implicit finite element at extreme scale
journal, December 2010


Works referencing / citing this record:

In-memory integration of existing software components for parallel adaptive unstructured mesh workflows: In-memory integration of existing software components for parallel adaptive unstructured mesh workflows
journal, April 2018

  • Smith, Cameron W.; Granzow, Brian; Diamond, Gerrett
  • Concurrency and Computation: Practice and Experience, Vol. 30, Issue 18
  • DOI: 10.1002/cpe.4510

The PHASTA Science Gateway: Web-based Execution of Adaptive Computational Fluid Dynamics Simulations
conference, July 2017

  • Smith, Cameron W.; Abeysinghe, Eroma
  • PEARC17: Practice and Experience in Advanced Research Computing 2017, Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact
  • DOI: 10.1145/3093338.3104151

An automated approach for parallel adjoint-based error estimation and mesh adaptation
journal, December 2019

  • Granzow, Brian N.; Oberai, Assad A.; Shephard, Mark S.
  • Engineering with Computers, Vol. 36, Issue 3
  • DOI: 10.1007/s00366-019-00890-2

Interface Tracking Investigation of Geometric Effects on the Bubbly Flow in PWR Subchannels
journal, August 2018