skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: SERGHEI (SERGHEI-SWE) v1.0: a performance-portable high-performance parallel-computing shallow-water solver for hydrology and environmental hydraulics

Journal Article · · Geoscientific Model Development (Online)

Abstract. The Simulation EnviRonment for Geomorphology, Hydrodynamics, and Ecohydrology in Integrated form (SERGHEI) is a multi-dimensional, multi-domain, and multi-physics model framework for environmental and landscape simulation, designed with an outlook towards Earth system modelling. At the core of SERGHEI's innovation is its performance-portable high-performance parallel-computing (HPC) implementation, built from scratch on the Kokkos portability layer, allowing SERGHEI to be deployed, in a performance-portable fashion, in graphics processing unit (GPU)-based heterogeneous systems. In this work, we explore combinations of MPI and Kokkos using OpenMP and CUDA backends. In this contribution, we introduce the SERGHEI model framework and present with detail its first operational module for solving shallow-water equations (SERGHEI-SWE) and its HPC implementation. This module is designed to be applicable to hydrological and environmental problems including flooding and runoff generation, with an outlook towards Earth system modelling. Its applicability is demonstrated by testing several well-known benchmarks and large-scale problems, for which SERGHEI-SWE achieves excellent results for the different types of shallow-water problems. Finally, SERGHEI-SWE scalability and performance portability is demonstrated and evaluated on several TOP500 HPC systems, with very good scaling in the range of over 20 000 CPUs and up to 256 state-of-the art GPUs.

Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
Grant/Contract Number:
AC02-05CH11231; AC05-00OR22725
OSTI ID:
1970338
Alternate ID(s):
OSTI ID: 2000308
Journal Information:
Geoscientific Model Development (Online), Vol. 16, Issue 3; ISSN 1991-9603
Publisher:
Copernicus Publications, EGUCopyright Statement
Country of Publication:
United States
Language:
English

References (160)

(Multi)wavelets increase both accuracy and efficiency of standard Godunov-type hydrodynamic models: Robust 2D approaches journal October 2020
Kokkos: Enabling manycore performance portability through polymorphic memory access patterns journal December 2014
A Spatially Distributed Model to Simulate Water, Energy, and Vegetation Dynamics Using Information from Regional Climate Models journal August 2013
Towards a generalised GPU/CPU shallow-flow modelling tool journal December 2013
Assessing climate-change-induced flood risk in the Conasauga River watershed: an application of ensemble hydrodynamic inundation modeling journal June 2021
An urban pluvial flood simulation model based on diffusive wave approximation of shallow water equations journal August 2017
2D Zero-Inertia Model for Solution of Overland Flow Problems in Flexible Meshes journal November 2016
Well-balanced RKDG2 solutions to the shallow water equations over irregular domains with wetting and drying journal December 2010
Benchmarking a multiresolution discontinuous Galerkin shallow water model: Implications for computational hydraulics journal December 2015
Overland flow computations in urban and industrial catchments from direct precipitation data using a two-dimensional shallow water model journal November 2010
The integrated hydrologic model intercomparison project, IH-MIP2: A second set of benchmark results to diagnose integrated hydrology and feedbacks: INTEGRATED HYDROLOGIC MODEL INTERCOMPARISON, IH-MIP2 journal January 2017
A full-scale fluvial flood modelling framework based on a high-performance integrated hydrodynamic modelling system (HiPIMS) journal October 2019
A robust well-balanced model on unstructured grids for shallow water flows with wetting and drying over complex topography journal April 2013
The validity of flow approximations when simulating catchment-integrated flash floods journal January 2018
An integrated GPU-accelerated modeling framework for high-resolution simulations of rural and urban flash floods journal October 2022
Translating novel HPC techniques into efficient geoscience solutions journal May 2021
Mesh type tradeoffs in 2D hydrodynamic modeling of flooding with a Godunov-based flow solver journal June 2014
The GeoClaw software for depth-averaged flows with adaptive refinement journal September 2011
The influence of grid shape and grid size on hydraulic river modelling performance journal February 2019
Measurement and modelling of high-resolution flow-velocity data under simulated rainfall on a low-slope sandy soil journal January 2008
B-flood 1.0: an open-source Saint-Venant model for flash-flood simulation using adaptive refinement journal November 2021
Numerical Tracking of Shallow Water Waves by the Unstructured Finite Volume WAF Approximation journal February 2007
Physically based modeling in catchment hydrology at 50: Survey and outlook: SURVEY OF PHYSICALLY BASED MODELING IN CATCHMENT HYDROLOGY journal September 2015
On the well-balanced numerical discretization of shallow water equations on unstructured meshes journal February 2013
An optimized GPU implementation of a 2D free surface simulation model on unstructured meshes journal December 2014
Discontinuous Galerkin Finite-Element Method for Transcritical Two-Dimensional Shallow Water Flows journal May 2004
Three‐dimensional simulation of tsunami run‐up around conical island journal August 2007
Nascent exascale supercomputers offer promise, present challenges journal September 2020
High-performance computing in water resources hydrodynamics journal March 2020
2D numerical simulation of unsteady flows for large scale floods prediction in real time journal December 2019
An efficient and stable hydrodynamic model with novel source term discretization schemes for overland flow and flood simulations: NEW HYDRODYNAMIC MODEL FOR OVERLAND FLOW journal May 2017
radaptation for Shallow Water flows: conservation, well balancedness, efficiency journal January 2018
A new efficient implicit scheme for discretising the stiff friction terms in the shallow water equations journal July 2018
Factors controlling seasonal groundwater and solute flux from snow-dominated basins journal June 2018
SWASHES: a compilation of shallow water analytic solutions for hydraulic and environmental studies: ANALYTIC SOLUTIONS FOR SHALLOW WATER EQUATIONS
  • Delestre, Olivier; Lucas, Carine; Ksinant, Pierre-Antoine
  • International Journal for Numerical Methods in Fluids, Vol. 72, Issue 3 https://doi.org/10.1002/fld.3741
journal October 2012
Multiwavelet-based mesh adaptivity with Discontinuous Galerkin schemes: Exploring 2D shallow water problems journal April 2020
2D GPU-Accelerated High Resolution Numerical Scheme for Solving Diffusive Wave Equations journal July 2019
An overview of current applications, challenges, and future trends in distributed process-based models in hydrology journal June 2016
Modelling flash flood propagation in urban areas using a two-dimensional numerical model journal October 2008
Friction term discretization and limitation to preserve stability and conservation in the 1D shallow-water model: Application to unsteady irrigation and river flow journal October 2008
A GPU-based numerical model coupling hydrodynamical and morphological processes journal August 2020
Rainfall/runoff simulation with 2D full shallow water equations: Sensitivity analysis and calibration of infiltration parameters journal May 2016
A 2D extension of a Large Time Step explicit scheme (CFL>1) for unsteady problems with wet/dry boundaries journal April 2014
Performance Evaluation of a Two-Dimensional Flood Model on Heterogeneous High-Performance Computing Architectures conference June 2020
Time step restrictions for well-balanced shallow water solutions in non-zero velocity steady states journal August 2009
Shallow water equations with depth-dependent anisotropic porosity for subgrid-scale topography journal September 2016
Numerical assessment of flood hazard risk to people and vehicles in flash floods journal August 2011
Toward performance portability of the Albany finite element analysis code using the Kokkos library journal October 2017
Dynamically adaptive grid based discontinuous Galerkin shallow water model journal March 2012
An investigation into the feasibility and benefits of GPU/multicore acceleration of the weather research and forecasting model conference July 2013
The Kokkos EcoSystem: Comprehensive Performance Portability For High Performance Computing journal January 2021
Modeling rainfall-runoff processes using smoothed particle hydrodynamics with mass-varied particles journal December 2016
A fast second-order shallow water scheme on two-dimensional structured grids over abrupt topography journal May 2019
Contradiction between the C-property and mass conservation in adaptive grid based shallow flow models: cause and solution: CONSERVATION PROPERTY AND MASS CONSERVATION ON ADAPTIVE GRID journal February 2015
PFLOTRAN User Manual: A Massively Parallel Reactive Flow and Transport Model for Describing Surface and Subsurface Processes report January 2015
Upscaling the shallow water model with a novel roughness formulation journal August 2015
Approximate Riemann solvers, parameter vectors, and difference schemes journal October 1981
A limiter‐based well‐balanced discontinuous Galerkin method for shallow‐water flows with wetting and drying: Triangular grids journal August 2019
An accurate time integration method for simplified overland flow models journal January 2008
Wavelet-based local mesh refinement for rainfall–runoff simulations journal April 2020
Some exact solutions to the nonlinear shallow-water wave equations journal June 1981
Central-upwind scheme for 2D turbulent shallow flows using high-resolution meshes with scalable wall functions journal January 2019
Programming revisited journal April 2015
A finite volume method for the simulation of the waves generated by landslides journal July 2009
Modeling wave runup with depth-integrated equations journal July 2002
Simplified versus Detailed Two-Dimensional Approaches to Transient Flow Modeling in Urban Areas journal April 2010
Parallel computing for high-resolution/large-scale flood simulation using the K supercomputer journal January 2015
The shallow water equations and their application to realistic cases journal January 2019
A General Design for a Scalable MPI-GPU Multi-Resolution 2D Numerical Solver journal May 2020
GPU-enhanced Finite Volume Shallow Water solver for fast flood simulations journal July 2014
Experiments of dam-break wave over a triangular bottom sill journal December 2007
Analysis of an open source quadtree grid shallow water flow solver for flood simulation journal October 2015
Making legacy Fortran code type safe through automated program transformation journal July 2021
Resistance Formulations in Shallow Overland Flow Along a Hillslope Covered With Patchy Vegetation journal May 2020
Simulation of Hurricane Harvey flood event through coupled hydrologic‐hydraulic models: Challenges and next steps journal March 2021
Comparison of roughness models to simulate overland flow and tracer transport experiments under simulated rainfall at plot scale journal May 2011
Efficient shallow water simulations on GPUs: Implementation, visualization, verification, and validation journal February 2012
Malpasset dam-break revisited with two-dimensional computations journal November 1999
A GIS based distributed rainfall–runoff model journal November 2004
The digital revolution of Earth-system science journal February 2021
Performance assessment of 2D Zero-Inertia and Shallow Water models for simulating rainfall-runoff processes journal May 2020
A Systematic Analysis of the Interaction between Rain-on-Grid-Simulations and Spatial Resolution in 2D Hydrodynamic Modeling journal August 2021
High performance shallow water kernels for parallel overland flow simulations based on FullSWOF2D journal July 2017
Towards transient experimental water surfaces: A new benchmark dataset for 2D shallow water solvers journal November 2018
A Study on the Performance Portability of the Finite Element Assembly Process Within the Albany Land Ice Solver book February 2020
Experimental validation of two-dimensional depth-averaged models for forecasting rainfall–runoff from precipitation data in urban areas journal March 2010
Case Study: Malpasset Dam-Break Simulation using a Two-Dimensional Finite Volume Method journal May 2002
Crossing the chasm: how to develop weather and climate models for next generation computers? journal January 2018
Finite-Volume Model for Shallow-Water Flooding of Arbitrary Topography journal March 2002
A hybrid method for flood simulation in small catchments combining hydrodynamic and hydrological techniques journal September 2016
Approximate Analytical Solutions for Overland Flows journal December 1990
A weighted surface-depth gradient method for the numerical integration of the 2D shallow water equations with topography journal July 2008
A simple raster-based model for flood inundation simulation journal September 2000
A model for overland flow and associated processes within the Hydroinformatics Modelling System journal March 2014
Dam-break flow through an idealised city journal September 2008
Laboratory experiments of tsunami runup on a circular island journal January 1995
From engineering hydrology to Earth system science: milestones in the transformation of hydrologic science journal January 2018
The evolution of process-based hydrologic models: historical challenges and the collective quest for physical realism journal January 2017
Evaluation of performance portability frameworks for the implementation of a particle‐in‐cell code journal December 2019
An efficient unstructured MUSCL scheme for solving the 2D shallow water equations journal April 2015
SW2D-GPU: A two-dimensional shallow water model accelerated by GPGPU journal November 2021
HydroGeoSphere: A Fully Integrated, Physically Based Hydrological Model journal December 2011
Kokkos implementation of an Ewald Coulomb solver and analysis of performance portability journal April 2020
An improved multislope MUSCL scheme for solving shallow water equations on unstructured grids journal January 2019
A GPU-Accelerated Shallow-Water Scheme for Surface Runoff Simulations journal February 2020
Review of Historical Dam-Break Events and Laboratory Tests on Real Topography for the Validation of Numerical Models journal July 2021
A mass-conservative predictor-corrector solution to the 1D Richards equation with adaptive time control journal January 2021
Weak solutions for partial differential equations with source terms: Application to the shallow water equations journal June 2010
2D well-balanced augmented ADER schemes for the Shallow Water Equations with bed elevation and extension to the rotating frame journal November 2018
HOMMEXX 1.0: a performance-portable atmospheric dynamical core for the Energy Exascale Earth System Model journal January 2019
Efficient surface water flow simulation on static Cartesian grid with local refinement according to key topographic features journal November 2018
Second-order discontinuous Galerkin flood model: Comparison with industry-standard finite volume models journal March 2021
Is HEC-RAS 2D accurate enough for storm-event hazard assessment? Lessons learnt from a benchmarking study based on rain-on-grid modelling journal December 2021
On-Tree Mango Fruit Size Estimation Using RGB-D Images journal November 2017
An Accelerated Tool for Flood Modelling Based on Iber journal October 2018
Adaptive finite volume methods with well-balanced Riemann solvers for modeling floods in rugged terrain: Application to the Malpasset dam-break flood (France, 1959) journal March 2010
New prospects for computational hydraulics by leveraging high-performance heterogeneous computing techniques journal December 2016
Hillslope Hydrology in Global Change Research and Earth System Modeling journal February 2019
Domain-specific acceleration and auto-parallelization of legacy scientific code in FORTRAN 77 using source-to-source compilation journal September 2018
Numerical Modeling of Rainfall-Generated Overland Flow Using Nonlinear Shallow-Water Equations journal August 2015
Zero mass error using unsteady wetting–drying conditions in shallow flows over dry irregular topography
  • Brufau, P.; García-Navarro, P.; Vázquez-Cendón, M. E.
  • International Journal for Numerical Methods in Fluids, Vol. 45, Issue 10 https://doi.org/10.1002/fld.729
journal July 2004
GPU implementation of the 2D shallow water equations for the simulation of rainfall/runoff events journal March 2015
TRITON: A Multi-GPU open source 2D hydrodynamic flood model journal July 2021
A local time stepping algorithm for GPU-accelerated 2D shallow water models journal January 2018
There’s plenty of room at the Top: What will drive computer performance after Moore’s law? journal June 2020
Augmented versions of the HLL and HLLC Riemann solvers including source terms in one and two dimensions for shallow flow applications journal August 2012
The East River, Colorado, Watershed: A Mountainous Community Testbed for Improving Predictive Understanding of Multiscale Hydrological–Biogeochemical Dynamics journal January 2018
Discretization of the divergence formulation of the bed slope term in the shallow-water equations and consequences in terms of energy balance journal September 2016
Analysis of the performance of a hybrid CPU/GPU 1D2D coupled model for real flood cases journal July 2020
A boundary-fitted numerical model for flood routing with shock-capturing capability journal January 2007
Runup of solitary waves on a circular Island journal November 1995
Cellular Automata and Finite Volume solvers converge for 2D shallow flow modelling for hydrological modelling journal August 2018
A 2D shallow flow model for practical dam-break simulations journal June 2011
A 2D-SWEs framework for efficient catchment-scale simulations: Hydrodynamic scaling properties of river networks and implications for non-uniform grids generation journal August 2021
A review on analysis of flood modelling using different numerical models journal August 2021
A simple and efficient unstructured finite volume scheme for solving the shallow water equations in overland flow applications journal July 2015
Simulation of Surface Runoff Using Hydrodynamic Model journal June 2017
K-Athena: A Performance Portable Structured Grid Finite Volume Magnetohydrodynamics Code journal January 2021
Influence of mesh structure on 2D full shallow water equations and SCS Curve Number simulation of rainfall/runoff events journal July 2012
An unstructured node-centered finite volume scheme for shallow water flows with wet/dry fronts over complex topography journal October 2009
Innovations towards the next generation of shallow flow models journal March 2021
An adaptive moving finite volume scheme for modeling flood inundation over dry and complex topography journal April 2013
RAJA: Portable Performance for Large-Scale Scientific Applications conference November 2019
Simulating coupled surface–subsurface flows with ParFlow v3.5.0: capabilities, applications, and ongoing development of an open-source, massively parallel, integrated hydrologic model journal January 2020
(Multi)wavelet-based Godunov-type simulators of flood inundation: Static versus dynamic adaptivity journal January 2023
LISFLOOD-FP 8.0: the new discontinuous Galerkin shallow-water solver for multi-core CPUs and GPUs journal June 2021
Exascale applications: skin in the game
  • Alexander, Francis; Almgren, Ann; Bell, John
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 378, Issue 2166 https://doi.org/10.1098/rsta.2019.0056
journal January 2020
Two-dimensional depth-averaged finite volume model for unsteady turbulent flow journal November 2012
A non-uniform efficient grid type for GPU-parallel Shallow Water Equations models journal February 2017
FullSWOF: Full Shallow-Water equations for Overland Flow journal December 2017
Application of a coastal modelling code in fluvial environments journal December 2011
A 2D finite volume simulation tool to enable the assessment of combined hydrological and morphodynamical processes in mountain catchments journal July 2020
A large time step 1D upwind explicit scheme (CFL>1): Application to shallow water equations journal August 2012
TELEMAC: An efficient hydrodynamics suite for massively parallel architectures journal December 2011
A stable 2D unstructured shallow flow model for simulations of wetting and drying over rough terrains journal August 2013
Net ecosystem productivity of boreal jack pine stands regenerating from clearcutting under current and future climates journal July 2007
(Multi)wavelets increase both accuracy and efficiency of standard Godunov-type hydrodynamic models journal July 2019
Numerical study on effects of building groups on dam-break flow in urban areas journal May 2012
Analysis of two sources of variability of basin outflow hydrographs computed with the 2D shallow water model Iber: Digital Terrain Model and unstructured mesh size journal September 2022
Efficient GPU-Implementation of Adaptive Mesh Refinement for the Shallow-Water Equations journal July 2014

Similar Records

Related Subjects