Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD

Abstract

Here, we explore the current performance and scaling of a fully-implicit stabilized unstructured finite element (FE) variational multiscale (VMS) capability for large-scale simulations of 3D incompressible resistive magnetohydrodynamics (MHD). The large-scale linear systems that are generated by a Newton nonlinear solver approach are iteratively solved by preconditioned Krylov subspace methods. The efficiency of this approach is critically dependent on the scalability and performance of the algebraic multigrid preconditioner. Our study considers the performance of the numerical methods as recently implemented in the second-generation Trilinos implementation that is 64-bit compliant and is not limited by the 32-bit global identifiers of the original Epetra-based Trilinos. The study presents representative results for a Poisson problem on 1.6 million cores of an IBM Blue Gene/Q platform to demonstrate very large-scale parallel execution. Additionally, results for a more challenging steady-state MHD generator and a transient solution of a benchmark MHD turbulence calculation for the full resistive MHD system are also presented. These results are obtained on up to 131,000 cores of a Cray XC40 and one million cores of a BG/Q system.

Authors:
 [1];  [2];  [1];  [1];  [1]
+ Show Author Affiliations
  1. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States). Center for Computing (CCR)
  2. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States). Center for Computing (CCR); Univ. of New Mexico, Albuquerque, NM (United States). Dept. of Mathematics and Statistics
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1411594
Report Number(s):
SAND2017-0881J
Journal ID: ISSN 0377-0427; 650766
Grant/Contract Number:  
AC04-94AL85000
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Computational and Applied Mathematics
Additional Journal Information:
Journal Volume: 344; Journal ID: ISSN 0377-0427
Publisher:
Elsevier
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Lin, P. T., Shadid, J. N., Hu, J. J., Pawlowski, R. P., and Cyr, E. C. Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD. United States: N. p., 2017. Web. doi:10.1016/j.cam.2017.09.028.
Copy to clipboard
Lin, P. T., Shadid, J. N., Hu, J. J., Pawlowski, R. P., & Cyr, E. C. Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD. United States. https://doi.org/10.1016/j.cam.2017.09.028
Copy to clipboard
Lin, P. T., Shadid, J. N., Hu, J. J., Pawlowski, R. P., and Cyr, E. C. Mon . "Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD". United States. https://doi.org/10.1016/j.cam.2017.09.028. https://www.osti.gov/servlets/purl/1411594.
Copy to clipboard
@article{osti_1411594,
title = {Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD},
author = {Lin, P. T. and Shadid, J. N. and Hu, J. J. and Pawlowski, R. P. and Cyr, E. C.},
abstractNote = {Here, we explore the current performance and scaling of a fully-implicit stabilized unstructured finite element (FE) variational multiscale (VMS) capability for large-scale simulations of 3D incompressible resistive magnetohydrodynamics (MHD). The large-scale linear systems that are generated by a Newton nonlinear solver approach are iteratively solved by preconditioned Krylov subspace methods. The efficiency of this approach is critically dependent on the scalability and performance of the algebraic multigrid preconditioner. Our study considers the performance of the numerical methods as recently implemented in the second-generation Trilinos implementation that is 64-bit compliant and is not limited by the 32-bit global identifiers of the original Epetra-based Trilinos. The study presents representative results for a Poisson problem on 1.6 million cores of an IBM Blue Gene/Q platform to demonstrate very large-scale parallel execution. Additionally, results for a more challenging steady-state MHD generator and a transient solution of a benchmark MHD turbulence calculation for the full resistive MHD system are also presented. These results are obtained on up to 131,000 cores of a Cray XC40 and one million cores of a BG/Q system.},
doi = {10.1016/j.cam.2017.09.028},
journal = {Journal of Computational and Applied Mathematics},
number = ,
volume = 344,
place = {United States},
year = {Mon Nov 06 00:00:00 EST 2017},
month = {Mon Nov 06 00:00:00 EST 2017}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1016/j.cam.2017.09.028
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 11 works
Citation information provided by
Web of Science
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Towards a scalable fully-implicit fully-coupled resistive MHD formulation with stabilized FE methods
journal, October 2010

  • Shadid, J. N.; Pawlowski, R. P.; Banks, J. W.
  • Journal of Computational Physics, Vol. 229, Issue 20
  • DOI: 10.1016/j.jcp.2010.06.018

A parallel fully coupled algebraic multilevel preconditioner applied to multiphysics PDE applications: Drift-diffusion, flow/transport/reaction, resistive MHD
journal, September 2010

  • Lin, Paul T.; Shadid, John N.; Tuminaro, Raymond S.
  • International Journal for Numerical Methods in Fluids, Vol. 64, Issue 10-12
  • DOI: 10.1002/fld.2402

Scalable implicit incompressible resistive MHD with stabilized FE and fully-coupled Newton–Krylov-AMG
journal, June 2016

  • Shadid, J. N.; Pawlowski, R. P.; Cyr, E. C.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 304
  • DOI: 10.1016/j.cma.2016.01.019

Multilevel Balancing Domain Decomposition at Extreme Scales
journal, January 2016

  • Badia, Santiago; Martín, Alberto F.; Principe, Javier
  • SIAM Journal on Scientific Computing, Vol. 38, Issue 1
  • DOI: 10.1137/15M1013511

Multiscale phenomena: Green's functions, the Dirichlet-to-Neumann formulation, subgrid scale models, bubbles and the origins of stabilized methods
journal, November 1995

The variational multiscale method—a paradigm for computational mechanics
journal, November 1998

  • Hughes, Thomas J. R.; Feijóo, Gonzalo R.; Mazzei, Luca
  • Computer Methods in Applied Mechanics and Engineering, Vol. 166, Issue 1-2
  • DOI: 10.1016/S0045-7825(98)00079-6

Performance of fully coupled domain decomposition preconditioners for finite element transport/reaction simulations
journal, May 2005

  • Shadid, J. N.; Tuminaro, R. S.; Devine, K. D.
  • Journal of Computational Physics, Vol. 205, Issue 1
  • DOI: 10.1016/j.jcp.2004.10.038

Hybrid Krylov Methods for Nonlinear Systems of Equations
journal, May 1990

  • Brown, Peter N.; Saad, Youcef
  • SIAM Journal on Scientific and Statistical Computing, Vol. 11, Issue 3
  • DOI: 10.1137/0911026

Jacobian-free Newton–Krylov methods: a survey of approaches and applications
journal, January 2004

Preconditioning Techniques for Large Linear Systems: A Survey
journal, November 2002

Performance of fully coupled algebraic multilevel domain decomposition preconditioners for incompressible flow and transport
journal, January 2006

  • Lin, Paul T.; Sala, Marzio; Shadid, John N.
  • International Journal for Numerical Methods in Engineering, Vol. 67, Issue 2
  • DOI: 10.1002/nme.1624

Performance of a parallel algebraic multilevel preconditioner for stabilized finite element semiconductor device modeling
journal, September 2009

  • Lin, Paul T.; Shadid, John N.; Sala, Marzio
  • Journal of Computational Physics, Vol. 228, Issue 17
  • DOI: 10.1016/j.jcp.2009.05.024

Multi-Jagged: A Scalable Parallel Spatial Partitioning Algorithm
journal, March 2016

  • Deveci, Mehmet; Rajamanickam, Sivasankaran; Devine, Karen D.
  • IEEE Transactions on Parallel and Distributed Systems, Vol. 27, Issue 3
  • DOI: 10.1109/TPDS.2015.2412545

Reducing Communication Costs for Sparse Matrix Multiplication within Algebraic Multigrid
journal, January 2016

  • Ballard, Grey; Siefert, Christopher; Hu, Jonathan
  • SIAM Journal on Scientific Computing, Vol. 38, Issue 3
  • DOI: 10.1137/15M1028807

Towards Extreme-Scale Simulations for Low Mach Fluids with Second-Generation Trilinos
journal, December 2014

  • Lin, Paul; Bettencourt, Matthew; Domino, Stefan
  • Parallel Processing Letters, Vol. 24, Issue 04
  • DOI: 10.1142/S0129626414420055

A New Petrov–Galerkin Smoothed Aggregation Preconditioner for Nonsymmetric Linear Systems
journal, January 2008

  • Sala, Marzio; Tuminaro, Raymond S.
  • SIAM Journal on Scientific Computing, Vol. 31, Issue 1
  • DOI: 10.1137/060659545

The dynamics of unforced turbulence at high Reynolds number for Taylor–Green vortices generalized to MHD
journal, April 2010

  • Pouquet, A.; Lee, E.; Brachet, M. E.
  • Geophysical & Astrophysical Fluid Dynamics, Vol. 104, Issue 2-3
  • DOI: 10.1080/03091920903304080

Lack of universality in decaying magnetohydrodynamic turbulence
journal, January 2010

A new class of finite element variational multiscale turbulence models for incompressible magnetohydrodynamics
journal, August 2015

    Sort by:
    [ × clear filter / sort ]

    Works referencing / citing this record:

    Performance of preconditioned iterative solvers in MFiX–Trilinos for fluidized beds
    journal, May 2018

    • Kotteda, V. M. Krushnarao; Kumar, Vinod; Spotz, William
    • The Journal of Supercomputing, Vol. 74, Issue 8
    • DOI: 10.1007/s11227-018-2415-5

    On the performance of Krylov smoothing for fully coupled AMG preconditioners for VMS resistive MHD
    journal, August 2019

    • Lin, Paul T.; Shadid, John N.; Tsuji, Paul H.
    • International Journal for Numerical Methods in Engineering, Vol. 120, Issue 12
    • DOI: 10.1002/nme.6178
      Sort by:
      [ × clear filter / sort ]

      Similar Records in DOE PAGES and OSTI.GOV collections: