High-order algorithmic developments and optimizations for large-scale GPU-accelerated simulations (Milestone CEED-MS36)

Kolev, Tzanio; Fischer, Paul; Austin, Anthony P.; Barker, Andrew T.; Beams, Natalie; Brown, Jed; Camier, Jean-Sylvain; Chalmers, Noel; Dobrev, Veselin; Dudouit, Yohann; Ghaffari, Leila; Kerkemeir, Stefan; Lan, Yu-Hsiang; Merzari, Elia; Min, Misun; Pazner, Will; Rathnayake, Thilina; Shephard, Mark S.; Siboni, Morteza H.; Smith, Cameron W.; Thompson, Jeremy L.; Tomov, Stanimire; Warburton, Tim

doi:10.2172/1845639

Title: High-order algorithmic developments and optimizations for large-scale GPU-accelerated simulations (Milestone CEED-MS36)

Full Record
Other Related Research

Abstract

The goal of this milestone was to improve the high-order software ecosystem for CEED-enabled ECP applications by making progress on efficient matrix-free kernels targeting forthcoming ECP architectures. These kernels included matrix-free preconditioning and the development of new set of CEED solver bake-off problems. As part of this milestone, we also released the next version of the CEED software stack, CEED-4.0, reported on results from several application collaborations, and documented the efforts of porting to AMD GPUs for Frontier and other modern architectures, such as Fugaku. The specific tasks addressed in this milestone were: (1) Port and run CEED benchmarks/miniapps on Frontier EA systems; (2) Demonstrate performant libCEED integration in MFEM, Nek and applications; (3) Matrix-free preconditioning of high-order operators; (4) Benchmark problems for fast high-order solvers on GPU platforms; and (5) Public release of CEED-4.0. The artifacts delivered include the next version of the CEED software stack, CEED-4.0, the next libCEED release, libCEED-0.8, and a number of developments integrated within applications to improve their GPU and CPU performance and capabilities. See the CEED website, https://ceed.exascaleproject.org and the CEED GitHub organization, https://github.com/ceed for more details.

Authors:

Kolev, Tzanio ^[1]; Fischer, Paul ^[1]; Austin, Anthony P. ^[1]; Barker, Andrew T. ^[1]; Beams, Natalie ^[1]; Brown, Jed ^[1]; Camier, Jean-Sylvain ^[1]; Chalmers, Noel ^[1]; Dobrev, Veselin ^[1]; Dudouit, Yohann ^[1]; Ghaffari, Leila ^[1]; Kerkemeir, Stefan ^[1]; Lan, Yu-Hsiang ^[1]; Merzari, Elia ^[1]; Min, Misun ^[1]; Pazner, Will ^[1]; Rathnayake, Thilina ^[1]; Shephard, Mark S. ^[1]; Siboni, Morteza H. ^[1]; Smith, Cameron W. ^[1] more »

Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Publication Date:: 2021-03-31

Research Org.:: Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Argonne National Lab. (ANL), Argonne, IL (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

OSTI Identifier:: 1845639

Report Number(s):: LLNL-TR-821011
1032720

DOE Contract Number:: AC52-07NA27344; AC02-06CH11357; AC05-00OR22725

Resource Type:: Technical Report

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Kolev, Tzanio, Fischer, Paul, Austin, Anthony P., Barker, Andrew T., Beams, Natalie, Brown, Jed, Camier, Jean-Sylvain, Chalmers, Noel, Dobrev, Veselin, Dudouit, Yohann, Ghaffari, Leila, Kerkemeir, Stefan, Lan, Yu-Hsiang, Merzari, Elia, Min, Misun, Pazner, Will, Rathnayake, Thilina, Shephard, Mark S., Siboni, Morteza H., Smith, Cameron W., Thompson, Jeremy L., Tomov, Stanimire, and Warburton, Tim. High-order algorithmic developments and optimizations for large-scale GPU-accelerated simulations (Milestone CEED-MS36).  United States: N. p., 2021. 
        Web.  doi:10.2172/1845639.

Copy to clipboard


                    Kolev, Tzanio, Fischer, Paul, Austin, Anthony P., Barker, Andrew T., Beams, Natalie, Brown, Jed, Camier, Jean-Sylvain, Chalmers, Noel, Dobrev, Veselin, Dudouit, Yohann, Ghaffari, Leila, Kerkemeir, Stefan, Lan, Yu-Hsiang, Merzari, Elia, Min, Misun, Pazner, Will, Rathnayake, Thilina, Shephard, Mark S., Siboni, Morteza H., Smith, Cameron W., Thompson, Jeremy L., Tomov, Stanimire, & Warburton, Tim. High-order algorithmic developments and optimizations for large-scale GPU-accelerated simulations (Milestone CEED-MS36).  United States.  https://doi.org/10.2172/1845639

Copy to clipboard


                    Kolev, Tzanio, Fischer, Paul, Austin, Anthony P., Barker, Andrew T., Beams, Natalie, Brown, Jed, Camier, Jean-Sylvain, Chalmers, Noel, Dobrev, Veselin, Dudouit, Yohann, Ghaffari, Leila, Kerkemeir, Stefan, Lan, Yu-Hsiang, Merzari, Elia, Min, Misun, Pazner, Will, Rathnayake, Thilina, Shephard, Mark S., Siboni, Morteza H., Smith, Cameron W., Thompson, Jeremy L., Tomov, Stanimire, and Warburton, Tim. 2021.  
        "High-order algorithmic developments and optimizations for large-scale GPU-accelerated simulations (Milestone CEED-MS36)".  United States.  https://doi.org/10.2172/1845639.  https://www.osti.gov/servlets/purl/1845639.

Copy to clipboard


                    
@article{osti_1845639,

  title        = {High-order algorithmic developments and optimizations for large-scale GPU-accelerated simulations (Milestone CEED-MS36)},

  author       = {Kolev, Tzanio and Fischer, Paul and Austin, Anthony P. and Barker, Andrew T. and Beams, Natalie and Brown, Jed and Camier, Jean-Sylvain and Chalmers, Noel and Dobrev, Veselin and Dudouit, Yohann and Ghaffari, Leila and Kerkemeir, Stefan and Lan, Yu-Hsiang and Merzari, Elia and Min, Misun and Pazner, Will and Rathnayake, Thilina and Shephard, Mark S. and Siboni, Morteza H. and Smith, Cameron W. and Thompson, Jeremy L. and Tomov, Stanimire and Warburton, Tim},

  abstractNote = {The goal of this milestone was to improve the high-order software ecosystem for CEED-enabled ECP applications by making progress on efficient matrix-free kernels targeting forthcoming ECP architectures. These kernels included matrix-free preconditioning and the development of new set of CEED solver bake-off problems. As part of this milestone, we also released the next version of the CEED software stack, CEED-4.0, reported on results from several application collaborations, and documented the efforts of porting to AMD GPUs for Frontier and other modern architectures, such as Fugaku. The specific tasks addressed in this milestone were: (1) Port and run CEED benchmarks/miniapps on Frontier EA systems; (2) Demonstrate performant libCEED integration in MFEM, Nek and applications; (3) Matrix-free preconditioning of high-order operators; (4) Benchmark problems for fast high-order solvers on GPU platforms; and (5) Public release of CEED-4.0. The artifacts delivered include the next version of the CEED software stack, CEED-4.0, the next libCEED release, libCEED-0.8, and a number of developments integrated within applications to improve their GPU and CPU performance and capabilities. See the CEED website, https://ceed.exascaleproject.org and the CEED GitHub organization, https://github.com/ceed for more details.},

  doi          = {10.2172/1845639},

  url          = {https://www.osti.gov/biblio/1845639},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2021},

  month        = {3}

}

Copy to clipboard

Technical Report:

View Technical Report (4.90 MB)

https://doi.org/10.2172/1845639

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Support CEED-enabled ECP applications in their preparation for Aurora/Frontier

Technical Report Kolev, Tzanio; Fischer, Paul; Abdelfattah, Ahmad; ...

The goal of this milestone was to help CEED-enabled ECP applications (particularly ExaSMR, MARBL, ExaAM, ExaWind and E3SM) in their preparations for the Aurora and Frontier architectures. This work included collaboration with ECP vendors and porting and optimization of CEED’s benchmarks and miniapps to early access hardware. As part of this milestone, we also made the best bake-off problems and bake-off kernel implementation from Nek, MFEM, libParanumal and the external community available in the latest libCEED release, libCEED-0.7. During the milestone period we also organized, in virtual form, the fourth CEED Annual meeting (CEED4AM) which included representatives from ECP applications,more »« less
https://doi.org/10.2172/1845651

Full Text Available
Initial CEED API (CEED-MS10)

Technical Report Brown, Jed; Camier, Jean-Sylvain; Dobrev, Veselin; ...

Key components of CEED software development involve fast finite element operator storage and evaluation, architecture optimizations, performant algorithms for all orders, global kernels for finite element operators, efficient use of the memory sub-system, optimal data locality and motion, enhanced scalability and parallelism, and fast tensor contractions. As ultimate goals of the project, CEED is exploring and identifying the best algorithms for the full range of discretizations and applying algorithmic and software development to support ECP applications’ needs. In this milestone, we initiated the development and implementation of a common CEED API consisting of two main components: low-level and high-level APIs.more »« less
https://doi.org/10.2172/1845655

Full Text Available
Port and optimize the CEED software stack to Aurora/Frontier EA (ECP Milestone Report)

Technical Report Kolev, Tzanio; Fischer, Paul; Beams, Natalie; ...

The goal of this milestone was to port the CEED software stack, including Nek, MFEM and libCEED to the Frontier and Aurora early access hardware, work on optimizing the performance on AMD and Intel GPUs, and demonstrate impact in CEED-enabled ECP applications. As part of this milestone, we also performed a number of other activities, including continued optimizations for CEED applications at scale on Summit, performance evaluation on non-ECP hardware, including the Fugaku’s A64FX chip and the NVIDIA A100 GPU architecture, exploring the use of just-in-time compilations in applications at scale and potential of mixed precision optimizations in the CEEDmore »« less
https://doi.org/10.2172/1822610

Full Text Available
Performance tuning of CEED software and 1st and 2nd wave apps

Technical Report Tomov, Stanimire; Abdelfattah, Ahmad; Barra, Valeria; ...

The goal of this milestone was the performance tuning of the CEED software, as well as the use and tuning of CEED to accelerate the first and second wave of targeted ECP applications. In this milestone, the CEED team developed optimization techniques and tuned for performance the CEED software to accelerate the first and second wave target ECP applications. Specifically, the focus was on the following: (1) Efficient use of the memory sub-system for managing data motion and interactions among different physics packages and libraries; this included integration of the new developments in a libCEED 0.5 software release; (2) Enhancingmore »« less
https://doi.org/10.2172/1845636

Full Text Available
CEED-MS34: Improve performance and capabilities of CEED-enabled ECP applications on Summit/Sierra. Exascale Computing Project Milestone Report

Technical Report Kolev, Tzanio; Fischer, Paul; Abdelfattah, Ahmad; ...

The main goal of this milestone was to help CEED-enabled ECP applications, including ExaSMR, MARBL, ExaWind and ExaAM, to improve their performance and capabilities on GPU systems like Summit and Lassen/Sierra. In addition, the CEED team also worked to: add and improve support for additional hardware and programming models in the CEED software components; release the next version of the CEED software stack, CEED-3.0; and demonstrate performance of libParanumal kernels in libCEED, Nek and MFEM. These additional tasks contributed directly to the main CEED-MS34 goal and will also play an important role in CEED’s future milestones.
https://doi.org/10.2172/1614965

Full Text Available

Similar Records