A performance spectrum for parallel computational frameworks that solve PDEs

Chang, J.; Nakshatrala, K. B.; Knepley, M. G.; Johnsson, L.

doi:10.1002/cpe.4401

Title: A performance spectrum for parallel computational frameworks that solve PDEs

Journal Article · Wed Dec 20 00:00:00 EST 2017 · Concurrency and Computation. Practice and Experience

DOI:https://doi.org/10.1002/cpe.4401· OSTI ID:1414485

Chang, J. ^[1];

^[1]; Knepley, M. G. ^[2]; Johnsson, L. ^[3]

Department of Civil and Environmental Engineering University of Houston Houston TX USA
Department of Computational and Applied Mathematics Rice University Houston TX USA
Department of Computer Science University of Houston Houston TX USA

Summary Important computational physics problems are often large‐scale in nature, and it is highly desirable to have robust and high performing computational frameworks that can quickly address these problems. However, it is no trivial task to determine whether a computational framework is performing efficiently or is scalable. The aim of this paper is to present various strategies for better understanding the performance of any parallel computational frameworks for solving PDEs. Important performance issues that negatively impact time‐to‐solution are discussed, and we propose a performance spectrum analysis that can enhance one's understanding of critical aforementioned performance issues. As proof of concept, we examine commonly used finite element simulation packages and software and apply the performance spectrum to quickly analyze the performance and scalability across various hardware platforms, software implementations, and numerical discretizations. It is shown that the proposed performance spectrum is a versatile performance model that is not only extendable to more complex PDEs such as hydrostatic ice sheet flow equations but also useful for understanding hardware performance in a massively parallel computing environment. Potential applications and future extensions of this work are also discussed.

View Accepted Manuscript (Publisher)

Cite

Export

Save

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC02-06CH11357

OSTI ID:: 1414485

Journal Information:: Concurrency and Computation. Practice and Experience, Journal Name: Concurrency and Computation. Practice and Experience Vol. 30 Journal Issue: 11; ISSN 1532-0626

Publisher:: Wiley Blackwell (John Wiley & Sons)Copyright Statement

Country of Publication:: United Kingdom

Language:: English

Citation Metrics:

Cited by: 10 works

Citation information provided by
Web of Science

References (35)

A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs Karypis, George; Kumar, Vipin SIAM Journal on Scientific Computing, Vol. 20, Issue 1 https://doi.org/10.1137/S1064827595287997	journal	January 1998
The input/output complexity of sorting and related problems Aggarwal, Alok; Vitter, Jeffrey,S. Communications of the ACM, Vol. 31, Issue 9 https://doi.org/10.1145/48529.48535	journal	August 1988
Reflections on the memory wall McKee, Sally A. Proceedings of the first conference on computing frontiers on Computing frontiers - CF'04 https://doi.org/10.1145/977091.977115	conference	January 2004
Hitting the memory wall: implications of the obvious Wulf, Wm. A.; McKee, Sally A. ACM SIGARCH Computer Architecture News, Vol. 23, Issue 1 https://doi.org/10.1145/216585.216588	journal	March 1995
Composable Linear Solvers for Multiphysics Brown, Jed; Knepley, Matthew G.; May, David A. 2012 11th International Symposium on Parallel and Distributed Computing (ISPDC) https://doi.org/10.1109/ISPDC.2012.16	conference	June 2012
Variational inequality approach to enforcing the non-negative constraint for advection–diffusion equations Chang, J.; Nakshatrala, K. B. Computer Methods in Applied Mechanics and Engineering, Vol. 320 https://doi.org/10.1016/j.cma.2017.03.022	journal	June 2017
SuperLU_DIST: A scalable distributed-memory sparse direct solver for unsymmetric linear systems Li, Xiaoye S.; Demmel, James W. ACM Transactions on Mathematical Software, Vol. 29, Issue 2 https://doi.org/10.1145/779359.779361	journal	June 2003
OpenMP: an industry standard API for shared-memory programming Dagum, L.; Menon, R. IEEE Computational Science and Engineering, Vol. 5, Issue 1 https://doi.org/10.1109/99.660313	journal	January 1998
Thin-Film Flows with Wall Slip: An Asymptotic Analysis of Higher Order Glacier Flow Models Schoof, C.; Hindmarsh, R. C. A. The Quarterly Journal of Mechanics and Applied Mathematics, Vol. 63, Issue 1 https://doi.org/10.1093/qjmam/hbp025	journal	January 2010
Composing Scalable Nonlinear Algebraic Solvers Brune, Peter R.; Knepley, Matthew G.; Smith, Barry F. SIAM Review, Vol. 57, Issue 4 https://doi.org/10.1137/130936725	journal	January 2015
libMesh : a C++ library for parallel adaptive mesh refinement/coarsening simulations Kirk, Benjamin S.; Peterson, John W.; Stogner, Roy H. Engineering with Computers, Vol. 22, Issue 3-4 https://doi.org/10.1007/s00366-006-0049-3	journal	November 2006
Tibidabo: Making the case for an ARM-based HPC system Rajovic, Nikola; Rico, Alejandro; Puzovic, Nikola Future Generation Computer Systems, Vol. 36 https://doi.org/10.1016/j.future.2013.07.013	journal	July 2014
IBM POWER8 processor core microarchitecture Sinharoy, B.; Van Norstrand, J. A.; Eickemeyer, R. J. IBM Journal of Research and Development, Vol. 59, Issue 1 https://doi.org/10.1147/JRD.2014.2376112	journal	January 2015
Finite Element Integration on GPUs Knepley, Matthew G.; Terrel, Andy R. ACM Transactions on Mathematical Software, Vol. 39, Issue 2 https://doi.org/10.1145/2427023.2427027	journal	February 2013
Large-Scale Optimization-Based Non-negative Computational Framework for Diffusion Equations: Parallel Implementation and Performance Studies Chang, J.; Karra, S.; Nakshatrala, K. B. Journal of Scientific Computing, Vol. 70, Issue 1 https://doi.org/10.1007/s10915-016-0250-5	journal	July 2016
Achieving Textbook Multigrid Efficiency for Hydrostatic Ice Sheet Flow Brown, Jed; Smith, Barry; Ahmadia, Aron SIAM Journal on Scientific Computing, Vol. 35, Issue 2 https://doi.org/10.1137/110834512	journal	January 2013
deal.II---A general-purpose object-oriented finite element library Bangerth, W.; Hartmann, R.; Kanschat, G. ACM Transactions on Mathematical Software, Vol. 33, Issue 4 https://doi.org/10.1145/1268776.1268779	journal	August 2007
Non-determinism and overcount on modern hardware performance counter implementations Weaver, Vincent M.; Terpstra, Dan; Moore, Shirley 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) https://doi.org/10.1109/ISPASS.2013.6557172	conference	April 2013
Numerical simulation of water resources problems: Models, methods, and trends Miller, Cass T.; Dawson, Clint N.; Farthing, Matthew W. Advances in Water Resources, Vol. 51 https://doi.org/10.1016/j.advwatres.2012.05.008	journal	January 2013
The Tau Parallel Performance System Shende, Sameer S.; Malony, Allen D. The International Journal of High Performance Computing Applications, Vol. 20, Issue 2 https://doi.org/10.1177/1094342006064482	journal	May 2006
MUMPS: A General Purpose Distributed Memory Sparse Solver Amestoy, Patrick R.; Duff, Iain S.; L’Excellent, Jean-Yves Applied Parallel Computing. New Paradigms for HPC in Industry and Academia https://doi.org/10.1007/3-540-70734-4_16	book	April 2001
HPCTOOLKIT: tools for performance analysis of optimized parallel programs Adhianto, L.; Banerjee, S.; Fagan, M. Concurrency and Computation: Practice and Experience https://doi.org/10.1002/cpe.1553	journal	January 2009
An overview of the Trilinos project Heroux, Michael A.; Phipps, Eric T.; Salinger, Andrew G. ACM Transactions on Mathematical Software, Vol. 31, Issue 3 https://doi.org/10.1145/1089014.1089021	journal	September 2005
A variational approach to ice stream flow Schoof, Christian Journal of Fluid Mechanics, Vol. 556 https://doi.org/10.1017/S0022112006009591	journal	May 2006
Scalable algorithms for three-field mixed finite element coupled poromechanics Castelletto, Nicola; White, Joshua A.; Ferronato, Massimiliano Journal of Computational Physics, Vol. 327 https://doi.org/10.1016/j.jcp.2016.09.063	journal	December 2016
Multiphysics simulations: Challenges and opportunities Keyes, David E.; McInnes, Lois C.; Woodward, Carol The International Journal of High Performance Computing Applications, Vol. 27, Issue 1 https://doi.org/10.1177/1094342012468181	journal	February 2013
Firedrake: Automating the Finite Element Method by Composing Abstractions Rathgeber, Florian; Ham, David A.; Mitchell, Lawrence ACM Transactions on Mathematical Software, Vol. 43, Issue 3 https://doi.org/10.1145/2998441	journal	December 2016
On the Effects of Memory Latency and Bandwidth on Supercomputer Application Performance Murphy, Richard 2007 IEEE 10th International Symposium on Workload Characterization https://doi.org/10.1109/IISWC.2007.4362179	conference	September 2007
The Scalasca performance toolset architecture Geimer, Markus; Wolf, Felix; Wylie, Brian J. N. Concurrency and Computation: Practice and Experience https://doi.org/10.1002/cpe.1556	journal	January 2010
Roofline: an insightful visual performance model for multicore architectures Williams, Samuel; Waterman, Andrew; Patterson, David Communications of the ACM, Vol. 52, Issue 4 https://doi.org/10.1145/1498765.1498785	journal	April 2009
pTatin3D: High-Performance Methods for Long-Term Lithospheric Dynamics May, Dave A.; Brown, Jed; Pourhiet, Laetitia Le SC14: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2014.28	conference	November 2014
PT-Scotch: A tool for efficient parallel graph ordering Chevalier, C.; Pellegrini, F. Parallel Computing, Vol. 34, Issue 6-8 https://doi.org/10.1016/j.parco.2007.12.001	journal	July 2008
A set of level 3 basic linear algebra subprograms Dongarra, J. J.; Du Croz, Jeremy; Hammarling, Sven ACM Transactions on Mathematical Software, Vol. 16, Issue 1 https://doi.org/10.1145/77626.79170	journal	March 1990
Uniform Convergence of Multigrid V-Cycle Iterations for Indefinite and Nonsymmetric Problems Bramble, James H.; Kwak, Do Y.; Pasciak, Joseph E. SIAM Journal on Numerical Analysis, Vol. 31, Issue 6 https://doi.org/10.1137/0731089	journal	December 1994
MOOSE: A parallel computational framework for coupled systems of nonlinear equations Gaston, Derek; Newman, Chris; Hansen, Glen Nuclear Engineering and Design, Vol. 239, Issue 10 https://doi.org/10.1016/j.nucengdes.2009.05.021	journal	October 2009

Similar Records

Computer Science Research Needs for Parallel Discrete Event Simulation (PDES)

Technical Report · Wed May 11 00:00:00 EDT 2022 · OSTI ID:1414485

Perumalla, Kalyan; Barnes, Peter; Bremer, Maximilian; +11 more

A low-communication, parallel algorithm for solving PDEs based on range decomposition

Journal Article · Mon Mar 28 00:00:00 EDT 2016 · Numerical Linear Algebra with Applications · OSTI ID:1414485

Appelhans, David J.; Manteuffel, Tom; McCormick, Steve; +1 more

Supercomputers for solving PDE problems. Final report, 1 October 1985-31 March 1988

Technical Report · Tue Nov 01 00:00:00 EST 1988 · OSTI ID:1414485

Hwang, K

Title: A performance spectrum for parallel computational frameworks that solve PDEs

Citation Formats

References (35)

Similar Records

Related Subjects