Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD

Lin, P. T.; Shadid, J. N.; Hu, J. J.; Pawlowski, R. P.; Cyr, E. C.

doi:10.1016/j.cam.2017.09.028

Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD

Journal Article · 05 November 2017 · Journal of Computational and Applied Mathematics

DOI:https://doi.org/10.1016/j.cam.2017.09.028· OSTI ID:1411594

Lin, P. T. ^[1]; Shadid, J. N. ^[2]; Hu, J. J. ^[1]; Pawlowski, R. P. ^[1]; Cyr, E. C. ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States). Center for Computing (CCR)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States). Center for Computing (CCR); Univ. of New Mexico, Albuquerque, NM (United States). Dept. of Mathematics and Statistics

Here, we explore the current performance and scaling of a fully-implicit stabilized unstructured finite element (FE) variational multiscale (VMS) capability for large-scale simulations of 3D incompressible resistive magnetohydrodynamics (MHD). The large-scale linear systems that are generated by a Newton nonlinear solver approach are iteratively solved by preconditioned Krylov subspace methods. The efficiency of this approach is critically dependent on the scalability and performance of the algebraic multigrid preconditioner. Our study considers the performance of the numerical methods as recently implemented in the second-generation Trilinos implementation that is 64-bit compliant and is not limited by the 32-bit global identifiers of the original Epetra-based Trilinos. The study presents representative results for a Poisson problem on 1.6 million cores of an IBM Blue Gene/Q platform to demonstrate very large-scale parallel execution. Additionally, results for a more challenging steady-state MHD generator and a transient solution of a benchmark MHD turbulence calculation for the full resistive MHD system are also presented. These results are obtained on up to 131,000 cores of a Cray XC40 and one million cores of a BG/Q system.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States); Sandia National Laboratories, Livermore, CA

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1411594

Report Number(s):: SAND2017-0881J; 650766

Journal Information:: Journal of Computational and Applied Mathematics, Journal Name: Journal of Computational and Applied Mathematics Vol. 344; ISSN 0377-0427

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (21)

A parallel fully coupled algebraic multilevel preconditioner applied to multiphysics PDE applications: Drift-diffusion, flow/transport/reaction, resistive MHD Lin, Paul T.; Shadid, John N.; Tuminaro, Raymond S. International Journal for Numerical Methods in Fluids, Vol. 64, Issue 10-12 https://doi.org/10.1002/fld.2402	journal	September 2010
Performance of fully coupled algebraic multilevel domain decomposition preconditioners for incompressible flow and transport Lin, Paul T.; Sala, Marzio; Shadid, John N. International Journal for Numerical Methods in Engineering, Vol. 67, Issue 2 https://doi.org/10.1002/nme.1624	journal	January 2006
Improving multigrid performance for unstructured mesh drift-diffusion simulations on 147,000 cores: IMPROVING MULTIGRID PERFORMANCE FOR UNSTRUCTURED MESH DRIFT-DIFFUSION Lin, Paul T. International Journal for Numerical Methods in Engineering, Vol. 91, Issue 9 https://doi.org/10.1002/nme.4315	journal	May 2012
Preconditioning Techniques for Large Linear Systems: A Survey Benzi, Michele Journal of Computational Physics, Vol. 182, Issue 2 https://doi.org/10.1006/jcph.2002.7176	journal	November 2002
Multiscale phenomena: Green's functions, the Dirichlet-to-Neumann formulation, subgrid scale models, bubbles and the origins of stabilized methods Hughes, Thomas J. R. Computer Methods in Applied Mechanics and Engineering, Vol. 127, Issue 1-4 https://doi.org/10.1016/0045-7825(95)00844-9	journal	November 1995
The variational multiscale method—a paradigm for computational mechanics Hughes, Thomas J. R.; Feijóo, Gonzalo R.; Mazzei, Luca Computer Methods in Applied Mechanics and Engineering, Vol. 166, Issue 1-2 https://doi.org/10.1016/S0045-7825(98)00079-6	journal	November 1998
Scalable implicit incompressible resistive MHD with stabilized FE and fully-coupled Newton–Krylov-AMG Shadid, J. N.; Pawlowski, R. P.; Cyr, E. C. Computer Methods in Applied Mechanics and Engineering, Vol. 304 https://doi.org/10.1016/j.cma.2016.01.019	journal	June 2016
Jacobian-free Newton–Krylov methods: a survey of approaches and applications Knoll, D. A.; Keyes, D. E. Journal of Computational Physics, Vol. 193, Issue 2 https://doi.org/10.1016/j.jcp.2003.08.010	journal	January 2004
Performance of fully coupled domain decomposition preconditioners for finite element transport/reaction simulations Shadid, J. N.; Tuminaro, R. S.; Devine, K. D. Journal of Computational Physics, Vol. 205, Issue 1 https://doi.org/10.1016/j.jcp.2004.10.038	journal	May 2005
Performance of a parallel algebraic multilevel preconditioner for stabilized finite element semiconductor device modeling Lin, Paul T.; Shadid, John N.; Sala, Marzio Journal of Computational Physics, Vol. 228, Issue 17 https://doi.org/10.1016/j.jcp.2009.05.024	journal	September 2009
Towards a scalable fully-implicit fully-coupled resistive MHD formulation with stabilized FE methods Shadid, J. N.; Pawlowski, R. P.; Banks, J. W. Journal of Computational Physics, Vol. 229, Issue 20 https://doi.org/10.1016/j.jcp.2010.06.018	journal	October 2010
A new class of finite element variational multiscale turbulence models for incompressible magnetohydrodynamics Sondak, D.; Shadid, J. N.; Oberai, A. A. Journal of Computational Physics, Vol. 295 https://doi.org/10.1016/j.jcp.2015.04.035	journal	August 2015
An optimal, parallel, fully implicit Newton–Krylov solver for three-dimensional viscoresistive magnetohydrodynamics Chacón, L. Physics of Plasmas, Vol. 15, Issue 5 https://doi.org/10.1063/1.2838244	journal	May 2008
The dynamics of unforced turbulence at high Reynolds number for Taylor–Green vortices generalized to MHD Pouquet, A.; Lee, E.; Brachet, M. E. Geophysical & Astrophysical Fluid Dynamics, Vol. 104, Issue 2-3 https://doi.org/10.1080/03091920903304080	journal	April 2010
Lack of universality in decaying magnetohydrodynamic turbulence Lee, E.; Brachet, M. E.; Pouquet, A. Physical Review E, Vol. 81, Issue 1 https://doi.org/10.1103/PhysRevE.81.016318	journal	January 2010
Multi-Jagged: A Scalable Parallel Spatial Partitioning Algorithm Deveci, Mehmet; Rajamanickam, Sivasankaran; Devine, Karen D. IEEE Transactions on Parallel and Distributed Systems, Vol. 27, Issue 3 https://doi.org/10.1109/TPDS.2015.2412545	journal	March 2016
A New Petrov–Galerkin Smoothed Aggregation Preconditioner for Nonsymmetric Linear Systems Sala, Marzio; Tuminaro, Raymond S. SIAM Journal on Scientific Computing, Vol. 31, Issue 1 https://doi.org/10.1137/060659545	journal	January 2008
Hybrid Krylov Methods for Nonlinear Systems of Equations Brown, Peter N.; Saad, Youcef SIAM Journal on Scientific and Statistical Computing, Vol. 11, Issue 3 https://doi.org/10.1137/0911026	journal	May 1990
Multilevel Balancing Domain Decomposition at Extreme Scales Badia, Santiago; Martín, Alberto F.; Principe, Javier SIAM Journal on Scientific Computing, Vol. 38, Issue 1 https://doi.org/10.1137/15M1013511	journal	January 2016
Reducing Communication Costs for Sparse Matrix Multiplication within Algebraic Multigrid Ballard, Grey; Siefert, Christopher; Hu, Jonathan SIAM Journal on Scientific Computing, Vol. 38, Issue 3 https://doi.org/10.1137/15M1028807	journal	January 2016
Towards Extreme-Scale Simulations for Low Mach Fluids with Second-Generation Trilinos Lin, Paul; Bettencourt, Matthew; Domino, Stefan Parallel Processing Letters, Vol. 24, Issue 04 https://doi.org/10.1142/S0129626414420055	journal	December 2014

Cited By (2)

On the performance of Krylov smoothing for fully coupled AMG preconditioners for VMS resistive MHD Lin, Paul T.; Shadid, John N.; Tsuji, Paul H. International Journal for Numerical Methods in Engineering, Vol. 120, Issue 12 https://doi.org/10.1002/nme.6178	journal	August 2019
Performance of preconditioned iterative solvers in MFiX–Trilinos for fluidized beds Kotteda, V. M. Krushnarao; Kumar, Vinod; Spotz, William The Journal of Supercomputing, Vol. 74, Issue 8 https://doi.org/10.1007/s11227-018-2415-5	journal	May 2018

Similar Records

On performance of Krylov smoothing for fully-coupled AMG preconditioners for VMS resistive MHD

Program Document · 2017 · OSTI ID:1429689

Multigrid in energy preconditioner for Krylov solvers

Journal Article · 2013 · Journal of Computational Physics · OSTI ID:22233578

A multigrid preconditioned Newton-Krylov method

Journal Article · 1999 · SIAM Journal on Scientific Computing · OSTI ID:20015658

Related Subjects

97 MATHEMATICS AND COMPUTING

Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD

Citation Formats

References (21)

Cited By (2)

Similar Records

Related Subjects