Asynchronous communication in spectral-element and discontinuous Galerkin methods for atmospheric dynamics – a case study using the High-Order Methods Modeling Environment (HOMME-homme_dg_branch)

Jamroz, Benjamin F.; Klofkorn, Robert

doi:10.5194/gmd-9-2881-2016

Title: Asynchronous communication in spectral-element and discontinuous Galerkin methods for atmospheric dynamics – a case study using the High-Order Methods Modeling Environment (HOMME-homme_dg_branch)

Full Record
Other Related Research

Abstract

The scalability of computational applications on current and next-generation supercomputers is increasingly limited by the cost of inter-process communication. We implement non-blocking asynchronous communication in the High-Order Methods Modeling Environment for the time integration of the hydrostatic fluid equations using both the spectral-element and discontinuous Galerkin methods. This allows the overlap of computation with communication, effectively hiding some of the costs of communication. A novel detail about our approach is that it provides some data movement to be performed during the asynchronous communication even in the absence of other computations. This method produces significant performance and scalability gains in large-scale simulations.

Authors:

Jamroz, Benjamin F. ^[1]; Klofkorn, Robert ^[2]

National Center for Atmospheric Research, Boulder, CO (United States). Computational Information Systems Lab.
National Center for Atmospheric Research, Boulder, CO (United States). Computational Information Systems Lab.; International Research Inst. of Stavanger (Norway)

Publication Date:: 2016-08-26

Research Org.:: National Center for Atmospheric Research (NCAR), Boulder, CO (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Biological and Environmental Research (BER); National Science Foundation (NSF)

OSTI Identifier:: 1437146

Grant/Contract Number:: SC0006959

Resource Type:: Accepted Manuscript

Journal Name:: Geoscientific Model Development (Online)

Additional Journal Information:: Journal Name: Geoscientific Model Development (Online); Journal Volume: 9; Journal Issue: 8; Journal ID: ISSN 1991-9603

Publisher:: European Geosciences Union

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; 58 GEOSCIENCES

Citation Formats


                    Jamroz, Benjamin F., and Klofkorn, Robert. Asynchronous communication in spectral-element and discontinuous Galerkin methods for atmospheric dynamics – a case study using the High-Order Methods Modeling Environment (HOMME-homme_dg_branch).  United States: N. p., 2016. 
Web.  doi:10.5194/gmd-9-2881-2016.

Copy to clipboard


                    Jamroz, Benjamin F., & Klofkorn, Robert. Asynchronous communication in spectral-element and discontinuous Galerkin methods for atmospheric dynamics – a case study using the High-Order Methods Modeling Environment (HOMME-homme_dg_branch).  United States.  https://doi.org/10.5194/gmd-9-2881-2016

Copy to clipboard


                    Jamroz, Benjamin F., and Klofkorn, Robert. Fri .  
"Asynchronous communication in spectral-element and discontinuous Galerkin methods for atmospheric dynamics – a case study using the High-Order Methods Modeling Environment (HOMME-homme_dg_branch)".  United States.  https://doi.org/10.5194/gmd-9-2881-2016.  https://www.osti.gov/servlets/purl/1437146.

Copy to clipboard


                    
@article{osti_1437146,

  title        = {Asynchronous communication in spectral-element and discontinuous Galerkin methods for atmospheric dynamics – a case study using the High-Order Methods Modeling Environment (HOMME-homme_dg_branch)},

  author       = {Jamroz, Benjamin F. and Klofkorn, Robert},

  abstractNote = {The scalability of computational applications on current and next-generation supercomputers is increasingly limited by the cost of inter-process communication. We implement non-blocking asynchronous communication in the High-Order Methods Modeling Environment for the time integration of the hydrostatic fluid equations using both the spectral-element and discontinuous Galerkin methods. This allows the overlap of computation with communication, effectively hiding some of the costs of communication. A novel detail about our approach is that it provides some data movement to be performed during the asynchronous communication even in the absence of other computations. This method produces significant performance and scalability gains in large-scale simulations.},

  doi          = {10.5194/gmd-9-2881-2016},

  journal      = {Geoscientific Model Development (Online)},

  number       = 8,

  volume       = 9,

  place        = {United States},

  year         = {2016},

  month        = {8}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.5194/gmd-9-2881-2016

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 2 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

A Petascale Non-Hydrostatic Atmospheric Dynamical Core in the HOMME Framework

Technical Report Tufo, Henry

The High-Order Method Modeling Environment (HOMME) is a framework for building scalable, conserva- tive atmospheric models for climate simulation and general atmospheric-modeling applications. Its spatial discretizations are based on Spectral-Element (SE) and Discontinuous Galerkin (DG) methods. These are local methods employing high-order accurate spectral basis-functions that have been shown to perform well on massively parallel supercomputers at any resolution and scale particularly well at high resolutions. HOMME provides the framework upon which the CAM-SE community atmosphere model dynamical-core is constructed. In its current incarnation, CAM-SE employs the hydrostatic primitive-equations (PE) of motion, which limits its resolution to simulations coarser thanmore »« less
https://doi.org/10.2172/1241514

Full Text Available
Scalable direct Vlasov solver with discontinuous Galerkin method on unstructured mesh.

Journal Article Xu, J ; Ostroumov, P N ; Mustapha, B ; ... - SIAM J. Sci. Comput.

This paper presents the development of parallel direct Vlasov solvers with discontinuous Galerkin (DG) method for beam and plasma simulations in four dimensions. Both physical and velocity spaces are in two dimesions (2P2V) with unstructured mesh. Contrary to the standard particle-in-cell (PIC) approach for kinetic space plasma simulations, i.e., solving Vlasov-Maxwell equations, direct method has been used in this paper. There are several benefits to solving a Vlasov equation directly, such as avoiding noise associated with a finite number of particles and the capability to capture fine structure in the plasma. The most challanging part of a direct Vlasov solvermore »« less
https://doi.org/10.1137/10078904X
Accelerating Flash-X Simulations with Asynchronous I/O

Conference Jain, Rajeev ; Tang, Houjun ; Dhruv, Akash ; ...

Most high-fidelity physics simulation codes, such as Flash-X, need to save intermediate results (checkpoint files) to restart or gain insights into the evolution of the simulation. These simulation codes save such intermediate files synchronously, where computation is stalled while the data is written to storage. Depending on the problem size and computational requirements, this file write time can be a substantial portion of the total simulation time. In order to hide the I/O latency of checkpointing, asynchronous I/O methods have been introduced. These methods use background threads for performing I/O while the main threads continue with the simulation. The usagemore »« less
https://doi.org/10.1109/PDSW56643.2022.00008

Full Text Available
An efficient parallel implementation of explicit multirate Runge–Kutta schemes for discontinuous Galerkin computations

Journal Article Seny, Bruno ; Lambrechts, Jonathan ; Toulorge, Thomas ; ... - Journal of Computational Physics

Although explicit time integration schemes require small computational efforts per time step, their efficiency is severely restricted by their stability limits. Indeed, the multi-scale nature of some physical processes combined with highly unstructured meshes can lead some elements to impose a severely small stable time step for a global problem. Multirate methods offer a way to increase the global efficiency by gathering grid cells in appropriate groups under local stability conditions. These methods are well suited to the discontinuous Galerkin framework. The parallelization of the multirate strategy is challenging because grid cells have different workloads. The computational cost is differentmore »« less
https://doi.org/10.1016/J.JCP.2013.07.041
A Parallel Reconstructed Discontinuous Galerkin Method for the Compressible Flows on Aritrary Grids

Conference Luo, Hong ; Ali, Amjad ; Nourgaliev, Robert ; ...

A reconstruction-based discontinuous Galerkin method is presented for the solution of the compressible Navier-Stokes equations on arbitrary grids. In this method, an in-cell reconstruction is used to obtain a higher-order polynomial representation of the underlying discontinuous Galerkin polynomial solution and an inter-cell reconstruction is used to obtain a continuous polynomial solution on the union of two neighboring, interface-sharing cells. The in-cell reconstruction is designed to enhance the accuracy of the discontinuous Galerkin method by increasing the order of the underlying polynomial solution. The inter-cell reconstruction is devised to remove an interface discontinuity of the solution and its derivatives and thusmore »« less
Full Text Available

Similar Records