Parallel Performance Optimizations on Unstructured Mesh-based Simulations

Conference · Thu Jan 01 00:00:00 EST 2015 · Procedia Computer Science

DOI:https://doi.org/10.1016/j.procs.2015.05.466· OSTI ID:1407282

Sarje, Abhinav; Song, Sukhyun; Jacobsen, Douglas; Huck, Kevin; Hollingsworth, Jeffrey; Malony, Allen; Williams, Samuel; Oliker, Leonid

© The Authors. Published by Elsevier B.V. This paper addresses two key parallelization challenges the unstructured mesh-based ocean modeling code, MPAS-Ocean, which uses a mesh based on Voronoi tessellations: (1) load imbalance across processes, and (2) unstructured data access patterns, that inhibit intra- and inter-node performance. Our work analyzes the load imbalance due to naive partitioning of the mesh, and develops methods to generate mesh partitioning with better load balance and reduced communication. Furthermore, we present methods that minimize both inter- and intranode data movement and maximize data reuse. Our techniques include predictive ordering of data elements for higher cache efficiency, as well as communication reduction approaches. We present detailed performance data when running on thousands of cores using the Cray XC30 supercomputer and show that our optimization strategies can exceed the original performance by over 2×. Additionally, many of these solutions can be broadly applied to a wide variety of unstructured grid-based computations.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
• DOE Contract Number: AC02-05CH11231
• OSTI ID: 1407282
• Journal Information: Procedia Computer Science, Vol. 51, Issue C; Conference: International Conference On Computational Science (ICCS 2015), Reykjavík (Iceland) , 1-3 Jun 2015; ISSN 1877-0509
• Publisher: Elsevier
• Country of Publication: United States
• Language: English