Manycore Parallel Computing for a Hybridizable Discontinuous Galerkin Nested Multigrid Method

Fabien, Maurice S.; Knepley, Matthew G.; Mills, Richard T.; Rivière, Béatrice M.

doi:10.1137/17M1128903

Manycore Parallel Computing for a Hybridizable Discontinuous Galerkin Nested Multigrid Method

Journal Article · Tue Mar 12 00:00:00 EDT 2019 · SIAM Journal on Scientific Computing

DOI:https://doi.org/10.1137/17M1128903· OSTI ID:1542582

Fabien, Maurice S. ^[1]; Knepley, Matthew G. ^[2]; Mills, Richard T. ^[3]; Rivière, Béatrice M. ^[1]

Rice Univ., Houston, TX (United States). Dept. of Computational and Applied Mathematics
Univ. of Buffalo, Buffalo, NY (United States). Dept. of Computer Science and Engineering
Argonne National Lab. (ANL), Portland, OR (United States). Computer Science and Mathematics Division

In this work, we present a parallel computing strategy for a hybridizable discontinuous Galerkin (HDG) nested geometric multigrid (GMG) solver. Parallel GMG solvers require a combination of coarse-grain and fine-grain parallelism to improve time-to-solution performance. In this work we focus on fine-grain parallelism. We use Intel's second generation Xeon Phi (Knights Landing) manycore processor. The GMG method achieves ideal convergence rates of 0.2 or less, for high polynomial orders. A matrix free (assembly free) technique is exploited to save considerable memory usage and increase arithmetic intensity. HDG enables static condensation, and due to the discontinuous nature of the discretization, we developed a matrix vector multiply routine that does not require any costly synchronizations or barriers. Our algorithm is able to attain 80% of peak bandwidth performance for higher order polynomials. This is possible due to the data locality inherent in the HDG method. Very high performance is realized for high order schemes, due to good arithmetic intensity, which declines as the order is reduced.

View Accepted Manuscript (DOE)

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: National Science Foundation (NSF)

Grant/Contract Number:: AC02-06CH11357

OSTI ID:: 1542582

Journal Information:: SIAM Journal on Scientific Computing, Journal Name: SIAM Journal on Scientific Computing Journal Issue: 2 Vol. 41; ISSN 1064-8275

Publisher:: SIAMCopyright Statement

Country of Publication:: United States

Language:: English

References (37)

Convergence of Multigrid Algorithms for Interior Penalty Methods Brenner, Susanne C.; Zhao, Jie Applied Numerical Analysis & Computational Mathematics, Vol. 2, Issue 1 https://doi.org/10.1002/anac.200410019	journal	May 2005
Fourier two-level analysis for discontinuous Galerkin discretization with linear elements Hemker, P. W.; Hoffmann, W.; van Raalte, M. H. Numerical Linear Algebra with Applications, Vol. 11, Issue 56 https://doi.org/10.1002/nla.356	journal	June 2004
Multigrid methods for the symmetric interior penalty method on graded meshes Brenner, S. C.; Cui, J.; Sung, L. -Y. Numerical Linear Algebra with Applications, Vol. 16, Issue 6 https://doi.org/10.1002/nla.630	journal	June 2009
A convergent multigrid cycle for the hybridized mixed method Gopalakrishnan, Jayadeep; Tan, Shuguang Numerical Linear Algebra with Applications, Vol. 16, Issue 9 https://doi.org/10.1002/nla.636	journal	September 2009
The hp -multigrid method applied to hp -adaptive refinement of triangular grids: THE hp -MULTIGRID METHOD APPLIED TO hp -ADAPTIVE REFINEMENT Mitchell, William F. Numerical Linear Algebra with Applications, Vol. 17, Issue 2-3 https://doi.org/10.1002/nla.700	journal	March 2010
Efficiency of high-order elements for continuous and discontinuous Galerkin methods: EFFICIENCY OF HIGH-ORDER ELEMENTS FOR CG AND DG Huerta, Antonio; Angeloski, Aleksandar; Roca, Xevi International Journal for Numerical Methods in Engineering, Vol. 96, Issue 9 https://doi.org/10.1002/nme.4547	journal	July 2013
Additive and multiplicative multi-grid — A comparison Bastian, P.; Wittum, G.; Hackbusch, W. Computing, Vol. 60, Issue 4 https://doi.org/10.1007/BF02684380	journal	December 1998
A multilevel discontinuous Galerkin method Gopalakrishnan, J.; Kanschat, G. Numerische Mathematik, Vol. 95, Issue 3 https://doi.org/10.1007/s002110200392	journal	September 2003
Fourier two-level analysis for higher dimensional discontinuous Galerkin discretisation Hemker, P. W.; van Raalte, M. H. Computing and Visualization in Science, Vol. 7, Issue 3-4 https://doi.org/10.1007/s00791-004-0136-1	journal	October 2004
Hybrid Multigrid/Schwarz Algorithms for the Spectral Element Method Lottes, James W.; Fischer, Paul F. Journal of Scientific Computing, Vol. 24, Issue 1 https://doi.org/10.1007/s10915-004-4787-3	journal	July 2005
On Hybrid Multigrid-Schwarz Algorithms Loisel, Sébastien; Nabben, Reinhard; Szyld, Daniel B. Journal of Scientific Computing, Vol. 36, Issue 2 https://doi.org/10.1007/s10915-007-9183-3	journal	December 2007
To CG or to HDG: A Comparative Study Kirby, Robert M.; Sherwin, Spencer J.; Cockburn, Bernardo Journal of Scientific Computing, Vol. 51, Issue 1 https://doi.org/10.1007/s10915-011-9501-7	journal	July 2011
Exploiting Batch Processing on Streaming Architectures to Solve 2D Elliptic Finite Element Problems: A Hybridized Discontinuous Galerkin (HDG) Case Study King, James; Yakovlev, Sergey; Fu, Zhisong Journal of Scientific Computing, Vol. 60, Issue 2 https://doi.org/10.1007/s10915-013-9805-x	journal	November 2013
To CG or to HDG: A Comparative Study in 3D Yakovlev, Sergey; Moxey, David; Kirby, Robert M. Journal of Scientific Computing, Vol. 67, Issue 1 https://doi.org/10.1007/s10915-015-0076-6	journal	July 2015
A Hybridized Discontinuous Galerkin Method for the Nonlinear Korteweg–de Vries Equation Samii, Ali; Panda, Nishant; Michoski, Craig Journal of Scientific Computing, Vol. 68, Issue 1 https://doi.org/10.1007/s10915-015-0133-1	journal	November 2015
The speed of convergence of one iterative process Fedorenko, R. P. USSR Computational Mathematics and Mathematical Physics, Vol. 4, Issue 3 https://doi.org/10.1016/0041-5553(64)90253-8	journal	January 1964
Parallel multigrid smoothing: polynomial versus Gauss–Seidel Adams, Mark; Brezina, Marian; Hu, Jonathan Journal of Computational Physics, Vol. 188, Issue 2 https://doi.org/10.1016/S0021-9991(03)00194-3	journal	July 2003
Sparse approximate inverse smoothers for geometric and algebraic multigrid Bröker, O. Applied Numerical Mathematics, Vol. 41, Issue 1 https://doi.org/10.1016/S0168-9274(01)00110-6	journal	April 2002
From h to p efficiently: Strategy selection for operator evaluation on hexahedral and tetrahedral elements Cantwell, C. D.; Sherwin, S. J.; Kirby, R. M. Computers & Fluids, Vol. 43, Issue 1 https://doi.org/10.1016/j.compfluid.2010.08.012	journal	April 2011
A generic interface for parallel cell-based finite element operator application Kronbichler, Martin; Kormann, Katharina Computers & Fluids, Vol. 63 https://doi.org/10.1016/j.compfluid.2012.04.012	journal	June 2012
Solving unsymmetric sparse systems of linear equations with PARDISO Schenk, Olaf; Gärtner, Klaus Future Generation Computer Systems, Vol. 20, Issue 3 https://doi.org/10.1016/j.future.2003.07.011	journal	April 2004
p-Multigrid solution of high-order discontinuous Galerkin discretizations of the compressible Navier–Stokes equations Fidkowski, Krzysztof J.; Oliver, Todd A.; Lu, James Journal of Computational Physics, Vol. 207, Issue 1 https://doi.org/10.1016/j.jcp.2005.01.005	journal	July 2005
From h to p efficiently: Implementing finite and spectral/hp element methods to achieve optimal performance for low- and high-order discretisations Vos, Peter E. J.; Sherwin, Spencer J.; Kirby, Robert M. Journal of Computational Physics, Vol. 229, Issue 13 https://doi.org/10.1016/j.jcp.2010.03.031	journal	July 2010
Block-asynchronous Multigrid Smoothers for GPU-accelerated Systems Anzt, Hartwig; Tomov, Stanimire; Gates, Mark Procedia Computer Science, Vol. 9 https://doi.org/10.1016/j.procs.2012.04.002	journal	January 2012
Frequency domain behavior of a set of parallel multigrid smoothing operators Benson, Maurice W. International Journal of Computer Mathematics, Vol. 36, Issue 1-2 https://doi.org/10.1080/00207169008803912	journal	January 1990
A superconvergent LDG-hybridizable Galerkin method for second-order elliptic problems Cockburn, Bernardo; Dong, Bo; Guzmán, Johnny Mathematics of Computation, Vol. 77, Issue 264 https://doi.org/10.1090/S0025-5718-08-02123-6	journal	May 2008
Multi-level adaptive solutions to boundary-value problems Brandt, Achi Mathematics of Computation, Vol. 31, Issue 138 https://doi.org/10.1090/S0025-5718-1977-0431719-X	journal	May 1977
Explicit barycentric weights for polynomial interpolation in the roots or extrema of classical orthogonal polynomials Wang, Haiyong; Huybrechs, Daan; Vandewalle, Stefan Mathematics of Computation, Vol. 83, Issue 290 https://doi.org/10.1090/S0025-5718-2014-02821-4	journal	March 2014
Multigrid for an HDG method Cockburn, B.; Dubois, O.; Gopalakrishnan, J. IMA Journal of Numerical Analysis, Vol. 34, Issue 4 https://doi.org/10.1093/imanum/drt024	journal	October 2013
Unified Hybridization of Discontinuous Galerkin, Mixed, and Continuous Galerkin Methods for Second Order Elliptic Problems Cockburn, Bernardo; Gopalakrishnan, Jayadeep; Lazarov, Raytcho SIAM Journal on Numerical Analysis, Vol. 47, Issue 2 https://doi.org/10.1137/070706616	journal	January 2009
A Hybridizable Discontinuous Galerkin Method for Steady-State Convection-Diffusion-Reaction Problems Cockburn, Bernardo; Dong, Bo; Guzmán, Johnny SIAM Journal on Scientific Computing, Vol. 31, Issue 5 https://doi.org/10.1137/080728810	journal	January 2009
Multigrid Algorithms for $hp$-Discontinuous Galerkin Discretizations of Elliptic Problems Antonietti, Paola F.; Sarti, Marco; Verani, Marco SIAM Journal on Numerical Analysis, Vol. 53, Issue 1 https://doi.org/10.1137/130947015	journal	January 2015
Unified Analysis of Discontinuous Galerkin Methods for Elliptic Problems Arnold, Douglas N.; Brezzi, Franco; Cockburn, Bernardo SIAM Journal on Numerical Analysis, Vol. 39, Issue 5 https://doi.org/10.1137/S0036142901384162	journal	January 2002
Barycentric Lagrange Interpolation Berrut, Jean-Paul; Trefethen, Lloyd N. SIAM Review, Vol. 46, Issue 3 https://doi.org/10.1137/S0036144502417715	journal	January 2004
Two-Level Fourier Analysis of a Multigrid Approach for Discontinuous Galerkin Discretization Hemker, P. W.; Hoffmann, W.; van Raalte, M. H. SIAM Journal on Scientific Computing, Vol. 25, Issue 3 https://doi.org/10.1137/S1064827502405100	journal	January 2003
Load Balancing for Adaptive Multigrid Methods Bastian, Peter SIAM Journal on Scientific Computing, Vol. 19, Issue 4 https://doi.org/10.1137/S1064827596297562	journal	July 1998
Roofline: an insightful visual performance model for multicore architectures Williams, Samuel; Waterman, Andrew; Patterson, David Communications of the ACM, Vol. 52, Issue 4 https://doi.org/10.1145/1498765.1498785	journal	April 2009

Cited By (3)

Discontinuous Galerkin approximations in computational mechanics: hybridization, exact geometry and degree adaptivity Giacomini, Matteo; Sevilla, Ruben SN Applied Sciences, Vol. 1, Issue 9 https://doi.org/10.1007/s42452-019-1065-4	journal	August 2019
hp-adaptive discontinuous Galerkin solver for elliptic equations in numerical relativity Vincent, Trevor; Pfeiffer, Harald P.; Fischer, Nils L. Physical Review D, Vol. 100, Issue 8 https://doi.org/10.1103/physrevd.100.084052	journal	October 2019
Discontinuous Galerkin approximations in computational mechanics: hybridization, exact geometry and degree adaptivity Giacomini, Matteo; Sevilla, Ruben arXiv https://doi.org/10.48550/arxiv.1908.06604	text	January 2019

Similar Records

Efficient discontinuous Galerkin implementations and preconditioners for implicit unsteady compressible flow simulations

Journal Article · Fri Apr 17 20:00:00 EDT 2020 · Computers and Fluids · OSTI ID:1802092

A hybridizable discontinuous Galerkin method for modeling fluid–structure interaction

Journal Article · Tue Aug 30 20:00:00 EDT 2016 · Journal of Computational Physics · OSTI ID:1326055

Overset meshing coupled with hybridizable discontinuous Galerkin finite elements

Journal Article · Tue Feb 28 19:00:00 EST 2017 · International Journal for Numerical Methods in Engineering · OSTI ID:1399894

Related Subjects

97 MATHEMATICS AND COMPUTING
accelerators
discontinuous Galerkin methods
high performance computing
multigrid

Manycore Parallel Computing for a Hybridizable Discontinuous Galerkin Nested Multigrid Method

Citation Formats

References (37)

Cited By (3)

Similar Records

Related Subjects