Scaled ILU Smoothers for Navier-Stokes Pressure Projection

Thomas, Stephen; Carr, Arielle; Mullowney, Paul; Swirydowicz, Katarzyna; Day, Marcus

doi:10.1002/fld.5254

Scaled ILU Smoothers for Navier-Stokes Pressure Projection

Journal Article · 27 December 2023 · International Journal for Numerical Methods in Fluids

DOI:https://doi.org/10.1002/fld.5254· OSTI ID:2377647

Thomas, Stephen; Carr, Arielle; Mullowney, Paul; Swirydowicz, Katarzyna;

Incomplete LU (ILU) smoothers are effective in the algebraic multigrid (AMG) V-cycle for reducing high-frequency components of the error. However, the requisite direct triangular solves are comparatively slow on GPUs. Previous work has demonstrated the advantages of Jacobi iteration as an alternative to direct solution of these systems. Depending on the threshold and fill-level parameters chosen, the factors can be highly nonnormal and Jacobi is unlikely to converge in a low number of iterations. We demonstrate that row scaling can reduce the departure from normality, allowing us to replace the inherently sequential solve with a rapidly converging Richardson iteration. There are several advantages beyond the lower compute time. Scaling is performed locally for a diagonal block of the global matrix because it is applied directly to the factor. Further, an ILUT Schur complement smoother maintains a constant GMRES iteration count as the number of MPI ranks increases, and thus parallel strong-scaling is improved. Our algorithms have been incorporated into hypre, and we demonstrate improved time to solution for linear systems arising in the Nalu-Wind and PeleLM pressure solvers. For large problem sizes, GMRES+AMG executes at least five times faster when using iterative triangular solves compared with direct solves on massively parallel GPUs.

Research Organization:: National Renewable Energy Laboratory (NREL), Golden, CO (United States)

Sponsoring Organization:: USDOE Office of Science (SC); USDOE National Nuclear Security Administration (NNSA); Exascale Computing Project (ECP)

DOE Contract Number:: AC36-08GO28308

OSTI ID:: 2377647

Report Number(s):: NREL/JA-2C00-90371; MainId:92149; UUID:18eff499-f215-4475-8c69-0932fdeb3f4c; MainAdminId:72948

Journal Information:: International Journal for Numerical Methods in Fluids, Journal Name: International Journal for Numerical Methods in Fluids Journal Issue: 4 Vol. 96

Country of Publication:: United States

Language:: English

References (30)

Iterative Sparse Triangular Solves for Preconditioning Anzt, Hartwig; Chow, Edmond; Dongarra, Jack Lecture Notes in Computer Science https://doi.org/10.1007/978-3-662-48096-0_50	book	January 2015
Iterative Methods for Sparse Linear Systems Saad, Yousef https://doi.org/10.1137/1.9780898718003	book	January 2003
Multigrid solvers for immersed finite element methods and immersed isogeometric analysis de Prenter, F.; Verhoosel, C. V.; van Brummelen, E. H. Computational Mechanics, Vol. 65, Issue 3 https://doi.org/10.1007/s00466-019-01796-y	journal	November 2019
Fields of values and iterative methods Eiermann, Michael Linear Algebra and its Applications, Vol. 180 https://doi.org/10.1016/0024-3795(93)90530-2	journal	February 1993
NekRS, a GPU-accelerated spectral element Navier–Stokes solver Fischer, Paul; Kerkemeier, Stefan; Min, Misun Parallel Computing, Vol. 114 https://doi.org/10.1016/j.parco.2022.102982	journal	December 2022
A conservative, thermodynamically consistent numerical approach for low Mach number combustion. Part I: Single-level integration Nonaka, Andrew; Day, Marcus S.; Bell, John B. Combustion Theory and Modelling, Vol. 22, Issue 1 https://doi.org/10.1080/13647830.2017.1390610	journal	August 2017
The Design and Implementation of hypre, a Library of Parallel High Performance Preconditioners Falgout, Robert D.; Jones, Jim E.; Yang, Ulrike Meier Lecture Notes in Computational Science and Engineering https://doi.org/10.1007/3-540-31619-1_8	book	January 2006
Fast and Accurate Multicomponent Transport Property Evaluation Ern, Alexandre; Giovangigli, Vincent Journal of Computational Physics, Vol. 120, Issue 1 https://doi.org/10.1006/jcph.1995.1151	journal	August 1995
Experimental study of ILU preconditioners for indefinite matrices Chow, Edmond; Saad, Yousef Journal of Computational and Applied Mathematics, Vol. 86, Issue 2 https://doi.org/10.1016/S0377-0427(97)00171-4	journal	December 1997
On measures of nonnormality of matrices Elsner, L.; Paardekooper, M. H. C. Linear Algebra and its Applications, Vol. 92 https://doi.org/10.1016/0024-3795(87)90253-9	journal	July 1987
CHEMKIN-III: A FORTRAN chemical kinetics package for the analysis of gas-phase chemical and plasma kinetics Kee, R. J.; Rupley, F. M.; Meeks, E. https://doi.org/10.2172/481621	report	May 1996
Preconditioning Parametrized Linear Systems Carr, Arielle; de Sturler, Eric; Gugercin, Serkan SIAM Journal on Scientific Computing, Vol. 43, Issue 3 https://doi.org/10.1137/20M1331123	journal	January 2021
Multilevel block factorizations in generalized hierarchical bases Chow, Edmond; Vassilevski, Panayot S. Numerical Linear Algebra with Applications, Vol. 10, Issue 1-2 https://doi.org/10.1002/nla.314	journal	January 2002
ExaDG: High-Order Discontinuous Galerkin for the Exa-Scale Arndt, Daniel; Fehn, Niklas; Kanschat, Guido Software for Exascale Computing - SPPEXA 2016-2019 https://doi.org/10.1007/978-3-030-47956-5_8	book	July 2020
Fine-Grained Parallel Incomplete LU Factorization Chow, Edmond; Patel, Aftab SIAM Journal on Scientific Computing, Vol. 37, Issue 2 https://doi.org/10.1137/140968896	journal	January 2015
hypre: A Library of High Performance Preconditioners Falgout, Robert D.; Yang, Ulrike Meier; Goos, Gerhard Computational Science — ICCS 2002: International Conference Amsterdam, The Netherlands, April 21–24, 2002 Proceedings, Part III https://doi.org/10.1007/3-540-47789-6_66	book	April 2002
Hierarchical multigrid approaches for the finite cell method on uniform and multi-level hp-refined grids Jomo, J.; Oztoprak, O.; de Prenter, F. Computer Methods in Applied Mechanics and Engineering, Vol. 386 https://doi.org/10.1016/j.cma.2021.114075	journal	December 2021
MFIX-Exa: A path toward exascale CFD-DEM simulations Musser, Jordan; Almgren, Ann S.; Fullmer, William D. The International Journal of High Performance Computing Applications https://doi.org/10.1177/10943420211009293	journal	April 2021
On long-range interpolation operators for aggressive coarsening: INTERPOLATION FOR AGGRESSIVE COARSENING Yang, Ulrike Meier Numerical Linear Algebra with Applications, Vol. 17, Issue 2-3 https://doi.org/10.1002/nla.689	journal	December 2009
Condition numbers and equilibration of matrices van der Sluis, A. Numerische Mathematik, Vol. 14, Issue 1 https://doi.org/10.1007/BF02165096	journal	December 1969
GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems Saad, Youcef; Schultz, Martin H. SIAM Journal on Scientific and Statistical Computing, Vol. 7, Issue 3 https://doi.org/10.1137/0907058	journal	July 1986
Algorithm 915, SuiteSparseQR Davis, Timothy A. ACM Transactions on Mathematical Software, Vol. 38, Issue 1 https://doi.org/10.1145/2049662.2049670	journal	November 2011
Using Jacobi iterations and blocking for solving sparse triangular systems in incomplete factorization preconditioning Chow, Edmond; Anzt, Hartwig; Scott, Jennifer Journal of Parallel and Distributed Computing, Vol. 119 https://doi.org/10.1016/j.jpdc.2018.04.017	journal	September 2018
Porting hypre to heterogeneous computer architectures: Strategies and experiences Falgout, Robert D.; Li, Ruipeng; Sjögreen, Björn Parallel Computing, Vol. 108 https://doi.org/10.1016/j.parco.2021.102840	journal	December 2021
Bounds for iterates, inverses, spectral variation and fields of values of non-normal matrices Henrici, Peter Numerische Mathematik, Vol. 4, Issue 1 https://doi.org/10.1007/BF01386294	journal	December 1962
A Comparison of Classical and Aggregation-Based Algebraic Multigrid Preconditioners for High-Fidelity Simulation of Wind Turbine Incompressible Flows Thomas, S. J.; Ananthan, S.; Yellapantula, S. SIAM Journal on Scientific Computing, Vol. 41, Issue 5 https://doi.org/10.1137/18M1179018	journal	January 2019
Multigrid Smoothers for Ultraparallel Computing Baker, Allison H.; Falgout, Robert D.; Kolev, Tzanio V. SIAM Journal on Scientific Computing, Vol. 33, Issue 5 https://doi.org/10.1137/100798806	journal	January 2011
AMReX: a framework for block-structured adaptive mesh refinement Zhang, Weiqun; Almgren, Ann; Beckner, Vince Journal of Open Source Software, Vol. 4, Issue 37 https://doi.org/10.21105/joss.01370	journal	May 2019
Optimally scaled matrices Bauer, F. L. Numerische Mathematik, Vol. 5, Issue 1 https://doi.org/10.1007/BF01385880	journal	December 1963
A New Class of AMG Interpolation Methods Based on Matrix-Matrix Multiplications Li, Ruipeng; Sjögreen, Björn; Yang, Ulrike Meier SIAM Journal on Scientific Computing https://doi.org/10.1137/20M134931X	journal	July 2021

Similar Records

Two-Stage Gauss-Seidel Preconditioners and Smoothers for Krylov Solvers on a GPU Cluster: Preprint

Conference · 2022 · OSTI ID:1845268

A two-level GPU-accelerated incomplete LU preconditioner for general sparse linear systems

Journal Article · 2025 · International Journal of High Performance Computing Applications · OSTI ID:2537970

On performance of Krylov smoothing for fully-coupled AMG preconditioners for VMS resistive MHD

Program Document · 2017 · OSTI ID:1429689

Related Subjects

GPU acceleration
ILU smoother
MATHEMATICS AND COMPUTING
algebraic multigrid
iterative triangular solvers

Scaled ILU Smoothers for Navier-Stokes Pressure Projection

Citation Formats

References (30)

Similar Records

Related Subjects