High-order matrix-free incompressible flow solvers with GPU acceleration and low-order refined preconditioners

Franco, Michael; Camier, Jean-Sylvain; Andrej, Julian; Pazner, Will

doi:10.1016/j.compfluid.2020.104541

Title: High-order matrix-free incompressible flow solvers with GPU acceleration and low-order refined preconditioners

Journal Article · Mon Apr 20 00:00:00 EDT 2020 · Computers and Fluids

DOI:https://doi.org/10.1016/j.compfluid.2020.104541· OSTI ID:1735799

Franco, Michael ^[1]; Camier, Jean-Sylvain ^[2]; Andrej, Julian ^[2]; Pazner, Will ^[2]

Univ. of California, Berkeley, CA (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Center for Applied Scientific Computing

In this work, we present a matrix-free flow solver for high-order finite element discretizations of the incompressible Navier-Stokes and Stokes equations with GPU acceleration. For high polynomial degrees, assembling the matrix for the linear systems resulting from the finite element discretization can be prohibitively expensive, both in terms of computational complexity and memory. For this reason, it is necessary to develop matrix-free operators and preconditioners, which can be used to efficiently solve these linear systems without access to the matrix entries themselves. The matrix-free operator evaluations utilize GPU-accelerated sum-factorization techniques to minimize memory movement and maximize throughput. The preconditioners developed in this work are based on a low-order refined methodology with parallel subspace corrections, as described for diffusion problems in [1]. The saddle-point Stokes system is solved using block-preconditioning techniques, which are robust in mesh size, polynomial degree, time step, and viscosity. For the incompressible Navier-Stokes equations, we make use of projection (fractional step) methods, which require Helmholtz and Poisson solves at each time step. The performance of our flow solvers is assessed on several benchmark problems in two and three spatial dimensions.

View Accepted Manuscript (DOE)

View Accepted Manuscript (Publisher)

Cite

Export

Save

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF)

Grant/Contract Number:: AC52-07NA27344; DGE 1752814

OSTI ID:: 1735799

Alternate ID(s):: OSTI ID: 1776273

Report Number(s):: LLNL-JRNL-791975; 991401

Journal Information:: Computers and Fluids, Vol. 203, Issue na; ISSN 0045-7930

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (46)

High-order CFD methods: current status and perspective: HIGH-ORDER CFD METHODS Wang, Z. J.; Fidkowski, Krzysztof; Abgrall, Rémi International Journal for Numerical Methods in Fluids, Vol. 72, Issue 8 https://doi.org/10.1002/fld.3767	journal	January 2013
A High-Order Accurate Discontinuous Finite Element Method for the Numerical Solution of the Compressible Navier–Stokes Equations Bassi, F.; Rebay, S. Journal of Computational Physics, Vol. 131, Issue 2 https://doi.org/10.1006/jcph.1996.5572	journal	March 1997
Nodal discontinuous Galerkin methods on graphics processors Klöckner, A.; Warburton, T.; Bridge, J. Journal of Computational Physics, Vol. 228, Issue 21 https://doi.org/10.1016/j.jcp.2009.06.041	journal	November 2009
On the utility of GPU accelerated high-order methods for unsteady flow simulations: A comparison with industry-standard tools Vermeire, B. C.; Witherden, F. D.; Vincent, P. E. Journal of Computational Physics, Vol. 334 https://doi.org/10.1016/j.jcp.2016.12.049	journal	April 2017
Multigrid for Matrix-Free High-Order Finite Element Computations on Graphics Processors Kronbichler, Martin; Ljungkvist, Karl ACM Transactions on Parallel Computing, Vol. 6, Issue 1 https://doi.org/10.1145/3322813	journal	May 2019
Spectral methods for problems in complex geometries Orszag, Steven A. Journal of Computational Physics, Vol. 37, Issue 1 https://doi.org/10.1016/0021-9991(80)90005-4	journal	August 1980
A spectral element method for fluid dynamics: Laminar flow in a channel expansion Patera, Anthony T. Journal of Computational Physics, Vol. 54, Issue 3 https://doi.org/10.1016/0021-9991(84)90128-1	journal	June 1984
Approximate tensor-product preconditioners for very high order discontinuous Galerkin methods Pazner, Will; Persson, Per-Olof Journal of Computational Physics, Vol. 354 https://doi.org/10.1016/j.jcp.2017.10.030	journal	February 2018
A generic interface for parallel cell-based finite element operator application Kronbichler, Martin; Kormann, Katharina Computers & Fluids, Vol. 63 https://doi.org/10.1016/j.compfluid.2012.04.012	journal	June 2012
Acceleration of tensor-product operations for high-order finite element methods Świrydowicz, Kasia; Chalmers, Noel; Karakus, Ali The International Journal of High Performance Computing Applications, Vol. 33, Issue 4 https://doi.org/10.1177/1094342018816368	journal	September 2018
Viscous Shock Capturing in a Time-Explicit Discontinuous Galerkin Method Klöckner, A.; Warburton, T.; Hesthaven, J. S. Mathematical Modelling of Natural Phenomena, Vol. 6, Issue 3 https://doi.org/10.1051/mmnp/20116303	journal	January 2011
Numerical solution of saddle point problems Benzi, Michele; Golub, Gene H.; Liesen, Jörg Acta Numerica, Vol. 14 https://doi.org/10.1017/S0962492904000212	journal	April 2005
An extreme-scale implicit solver for complex PDEs: highly heterogeneous flow in earth's mantle Rudi, Johann; Ghattas, Omar; Malossi, A. Cristiano I. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15 https://doi.org/10.1145/2807591.2807675	conference	January 2015
Matrix-free multigrid block-preconditioners for higher order discontinuous Galerkin discretisations Bastian, Peter; Müller, Eike Hermann; Müthing, Steffen Journal of Computational Physics, Vol. 394 https://doi.org/10.1016/j.jcp.2019.06.001	journal	October 2019
On the efficiency of a matrix-free linearly implicit time integration strategy for high-order Discontinuous Galerkin solutions of incompressible turbulent flows Franciolini, Matteo; Crivellini, Andrea; Nigro, Alessandra Computers & Fluids, Vol. 159 https://doi.org/10.1016/j.compfluid.2017.10.008	journal	December 2017
A matrix-free high-order discontinuous Galerkin compressible Navier-Stokes solver: A performance comparison of compressible and incompressible formulations for turbulent incompressible flows: High-order matrix-free DG: Compressible vs incompressible Fehn, Niklas; Wall, Wolfgang A.; Kronbichler, Martin International Journal for Numerical Methods in Fluids, Vol. 89, Issue 3 https://doi.org/10.1002/fld.4683	journal	October 2018
Finite-Element Preconditioning for Pseudospectral Solutions of Elliptic Problems Deville, M. O.; Mund, E. H. SIAM Journal on Scientific and Statistical Computing, Vol. 11, Issue 2 https://doi.org/10.1137/0911019	journal	March 1990
Preconditioned minimal residual methods for chebyshev spectral calculations Canuto, Claudio; Quarteroni, Alfio Journal of Computational Physics, Vol. 60, Issue 2 https://doi.org/10.1016/0021-9991(85)90010-5	journal	September 1985
Stability of Higher-Order Hood–Taylor Methods Brezzi, Franco; Falk, Richard S. SIAM Journal on Numerical Analysis, Vol. 28, Issue 3 https://doi.org/10.1137/0728032	journal	June 1991
An overview of projection methods for incompressible flows Guermond, J. L.; Minev, P.; Shen, Jie Computer Methods in Applied Mechanics and Engineering, Vol. 195, Issue 44-47 https://doi.org/10.1016/j.cma.2005.10.010	journal	September 2006
Diagonally Implicit Runge–Kutta Methods for Stiff O.D.E.’s Alexander, Roger SIAM Journal on Numerical Analysis, Vol. 14, Issue 6 https://doi.org/10.1137/0714068	journal	December 1977
A comparison of time-discretization/linearization approaches for the incompressible Navier–Stokes equations John, Volker; Matthies, Gunar; Rang, Joachim Computer Methods in Applied Mechanics and Engineering, Vol. 195, Issue 44-47 https://doi.org/10.1016/j.cma.2005.10.007	journal	September 2006
A numerical method for solving incompressible viscous flow problems Chorin, Alexandre Joel Journal of Computational Physics, Vol. 2, Issue 1 https://doi.org/10.1016/0021-9991(67)90037-X	journal	August 1967
Numerical solution of the Navier-Stokes equations Chorin, Alexandre Joel Mathematics of Computation, Vol. 22, Issue 104 https://doi.org/10.1090/S0025-5718-1968-0242392-2	journal	January 1968
A second-order projection method for the incompressible navier-stokes equations Bell, John B.; Colella, Phillip; Glaz, Harland M. Journal of Computational Physics, Vol. 85, Issue 2 https://doi.org/10.1016/0021-9991(89)90151-4	journal	December 1989
Accurate Projection Methods for the Incompressible Navier–Stokes Equations Brown, David L.; Cortez, Ricardo; Minion, Michael L. Journal of Computational Physics, Vol. 168, Issue 2 https://doi.org/10.1006/jcph.2001.6715	journal	April 2001
High-order splitting methods for the incompressible Navier-Stokes equations Karniadakis, George Em; Israeli, Moshe; Orszag, Steven A. Journal of Computational Physics, Vol. 97, Issue 2 https://doi.org/10.1016/0021-9991(91)90007-8	journal	December 1991
Boundary conditions for incompressible flows Orszag, Steven A.; Israeli, Moshe; Deville, Michel O. Journal of Scientific Computing, Vol. 1, Issue 1 https://doi.org/10.1007/BF01061454	journal	January 1986
Short note on the mass matrix for Gauss–Lobatto grid points Teukolsky, Saul A. Journal of Computational Physics, Vol. 283 https://doi.org/10.1016/j.jcp.2014.12.012	journal	February 2015
Stabilization of spectral methods by finite element bubble functions Canuto, Claudio Computer Methods in Applied Mechanics and Engineering, Vol. 116, Issue 1-4 https://doi.org/10.1016/S0045-7825(94)80004-9	journal	January 1994
Finite-Element Preconditioning of G-NI Spectral Methods Canuto, Claudio; Gervasio, Paola; Quarteroni, Alfio SIAM Journal on Scientific Computing, Vol. 31, Issue 6 https://doi.org/10.1137/090746367	journal	January 2010
An Overlapping Schwarz Method for Spectral Element Solution of the Incompressible Navier–Stokes Equations Fischer, Paul F. Journal of Computational Physics, Vol. 133, Issue 1 https://doi.org/10.1006/jcph.1997.5651	journal	May 1997
Scalable Low-Order Finite Element Preconditioners for High-Order Spectral Element Poisson Solvers Bello-Maldonado, Pedro D.; Fischer, Paul F. SIAM Journal on Scientific Computing, Vol. 41, Issue 5 https://doi.org/10.1137/18M1194997	journal	January 2019
Hybrid Multigrid/Schwarz Algorithms for the Spectral Element Method Lottes, James W.; Fischer, Paul F. Journal of Scientific Computing, Vol. 24, Issue 1 https://doi.org/10.1007/s10915-004-4787-3	journal	July 2005
Nested dyadic grids associated with Legendre–Gauss–Lobatto grids Brix, Kolja; Canuto, Claudio; Dahmen, Wolfgang Numerische Mathematik, Vol. 131, Issue 2 https://doi.org/10.1007/s00211-014-0691-4	journal	December 2014
A Flexible Inner-Outer Preconditioned GMRES Algorithm Saad, Youcef SIAM Journal on Scientific Computing, Vol. 14, Issue 2 https://doi.org/10.1137/0914028	journal	March 1993
Laminar flow behind a two-dimensional grid Kovasznay, L. I. G. Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 44, Issue 1 https://doi.org/10.1017/S0305004100023999	journal	January 1948
Small-scale structure of the Taylor–Green vortex Brachet, Marc E.; Meiron, Daniel I.; Orszag, Steven A. Journal of Fluid Mechanics, Vol. 130, Issue -1 https://doi.org/10.1017/S0022112083001159	journal	May 1983
The Taylor-Green vortex and fully developed turbulence Brachet, M. E.; Meiron, D.; Orszag, S. Journal of Statistical Physics, Vol. 34, Issue 5-6 https://doi.org/10.1007/BF01009458	journal	March 1984
Numerical Convergence Study of Nearly Incompressible, Inviscid Taylor–Green Vortex Flow Shu, Chi-Wang; Don, Wai-Sun; Gottlieb, David Journal of Scientific Computing, Vol. 24, Issue 1 https://doi.org/10.1007/s10915-004-5407-y	journal	July 2005
On coherent-vortex identification in turbulence Dubief †, Yves; Delcayre ‡, Franck Journal of Turbulence, Vol. 1 https://doi.org/10.1088/1468-5248/1/1/011	journal	January 2000
A comparison of vortex and pseudo-spectral methods for the simulation of periodic vortical flows at high Reynolds numbers van Rees, Wim M.; Leonard, Anthony; Pullin, D. I. Journal of Computational Physics, Vol. 230, Issue 8 https://doi.org/10.1016/j.jcp.2010.11.031	journal	April 2011
Interior penalty tensor-product preconditioners for high-order discontinuous Galerkin discretizations Pazner, Will; Persson, Per-Olof 2018 AIAA Aerospace Sciences Meeting https://doi.org/10.2514/6.2018-1093	conference	January 2018
Nodal Discontinuous Galerkin Methods on Graphics Processors Klöckner, Andreas; Warburton, Tim; Bridge, Jeffrey arXiv https://doi.org/10.48550/arxiv.0901.1024	text	January 2009
Viscous Shock Capturing in a Time-Explicit Discontinuous Galerkin Method Klöckner, Andreas; Warburton, Tim; Hesthaven, Jan S. arXiv https://doi.org/10.48550/arxiv.1102.3190	text	January 2011
Efficient low-order refined preconditioners for high-order matrix-free continuous and discontinuous Galerkin methods Pazner, Will arXiv https://doi.org/10.48550/arxiv.1908.07071	text	January 2019

Cited By (1)

Sparse invariant domain preserving discontinuous Galerkin methods with subcell convex limiting Pazner, Will arXiv https://doi.org/10.48550/arxiv.2004.08503	text	January 2020

Similar Records

Compare linear-system solver and preconditioner stacks with emphasis on GPU performance and propose phase-2 NGP solver development pathway

Technical Report · Thu May 14 00:00:00 EDT 2020 · OSTI ID:1735799

Hu, Jonathan J.; Berger-Vergiat, Luc; Thomas, Stephen; +7 more

A Comparison of Classical and Aggregation-Based Algebraic Multigrid Preconditioners for High-Fidelity Simulation of Wind Turbine Incompressible Flows

Journal Article · Tue Oct 29 00:00:00 EDT 2019 · SIAM Journal on Scientific Computing · OSTI ID:1735799

Thomas, Stephen; Ananthan, Shreyas; Yellapantula, Shashank; +3 more

A Comparison of Classical and Aggregation-Based Algebraic Multigrid Preconditioners for High-Fidelity Simulation of Wind Turbine Incompressible Flows

Journal Article · Tue Oct 29 00:00:00 EDT 2019 · SIAM Journal on Scientific Computing · OSTI ID:1735799

Thomas, Stephen J.; Ananthan, Shreyas; Yellapantula, Shashank; +3 more

Related Subjects

97 MATHEMATICS AND COMPUTING

Title: High-order matrix-free incompressible flow solvers with GPU acceleration and low-order refined preconditioners

Citation Formats

References (46)

Cited By (1)

Similar Records

Related Subjects