Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis

Johansen, Hans; Rodgers, Arthur; Petersson, N. Anders; McCallen, David; Sjogreen, Bjorn; Miah, Mamun

doi:10.1109/MCSE.2017.3421558

Title: Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis

Abstract

Modernizing SW4 for massively parallel time-domain simulations of earthquake ground motions in 3D earth models increases resolution and provides ground motion estimates for critical infrastructure risk evaluations. Simulations of ground motions from large (M ≥ 7.0) earthquakes require domains on the order of 100 to 500 km and spatial granularity on the order of 1 to 5 m resulting in hundreds of billions of grid points. Surface-focused structured mesh refinement (SMR) allows for more constant grid point per wavelength scaling in typical Earth models, where wavespeeds increase with depth. In fact, MR allows for simulations to double the frequency content relative to a fixed grid calculation on a given resource. The authors report improvements to the SW4 algorithm developed while porting the code to the Cori Phase 2 (Intel Xeon Phi) systems at the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory. Investigations of the performance of the innermost loop of the calculations found that reorganizing the order of operations can improve performance for massive problems.

Authors:

Johansen, Hans ^[1]; Rodgers, Arthur ^[2]; Petersson, N. Anders ^[2]; McCallen, David ^[1]; Sjogreen, Bjorn ^[2]; Miah, Mamun ^[1]

Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Publication Date:: Fri Sep 01 00:00:00 EDT 2017

Research Org.:: Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1399737

Alternate Identifier(s):: OSTI ID: 1474992

Report Number(s):: LLNL-JRNL-731165
Journal ID: ISSN 1521-9615; TRN: US1702849

Grant/Contract Number:: AC52-07NA27344; AC02-05CH11231

Resource Type:: Accepted Manuscript

Journal Name:: Computing in Science and Engineering

Additional Journal Information:: Journal Volume: 19; Journal Issue: 5; Journal ID: ISSN 1521-9615

Publisher:: IEEE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; 58 GEOSCIENCES; computational seismology; structural mechanics; earthquake simulations; exascale computing; scientific computing

Citation Formats


                    Johansen, Hans, Rodgers, Arthur, Petersson, N. Anders, McCallen, David, Sjogreen, Bjorn, and Miah, Mamun. Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis.  United States: N. p., 2017. 
Web.  doi:10.1109/MCSE.2017.3421558.

Copy to clipboard


                    Johansen, Hans, Rodgers, Arthur, Petersson, N. Anders, McCallen, David, Sjogreen, Bjorn, & Miah, Mamun. Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis.  United States.  https://doi.org/10.1109/MCSE.2017.3421558

Copy to clipboard


                    Johansen, Hans, Rodgers, Arthur, Petersson, N. Anders, McCallen, David, Sjogreen, Bjorn, and Miah, Mamun. Fri .  
"Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis".  United States.  https://doi.org/10.1109/MCSE.2017.3421558.  https://www.osti.gov/servlets/purl/1399737.

Copy to clipboard


                    
@article{osti_1399737,

  title        = {Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis},

  author       = {Johansen, Hans and Rodgers, Arthur and Petersson, N. Anders and McCallen, David and Sjogreen, Bjorn and Miah, Mamun},

  abstractNote = {Modernizing SW4 for massively parallel time-domain simulations of earthquake ground motions in 3D earth models increases resolution and provides ground motion estimates for critical infrastructure risk evaluations. Simulations of ground motions from large (M ≥ 7.0) earthquakes require domains on the order of 100 to 500 km and spatial granularity on the order of 1 to 5 m resulting in hundreds of billions of grid points. Surface-focused structured mesh refinement (SMR) allows for more constant grid point per wavelength scaling in typical Earth models, where wavespeeds increase with depth. In fact, MR allows for simulations to double the frequency content relative to a fixed grid calculation on a given resource. The authors report improvements to the SW4 algorithm developed while porting the code to the Cori Phase 2 (Intel Xeon Phi) systems at the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory. Investigations of the performance of the innermost loop of the calculations found that reorganizing the order of operations can improve performance for massive problems.},

  doi          = {10.1109/MCSE.2017.3421558},

  journal      = {Computing in Science and Engineering},

  number       = 5,

  volume       = 19,

  place        = {United States},

  year         = {Fri Sep 01 00:00:00 EDT 2017},

  month        = {Fri Sep 01 00:00:00 EDT 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/MCSE.2017.3421558

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 13 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referencing / citing this record:

Earthquake soil-structure interaction of nuclear power plants, differences in response to 3-D, 3 × 1-D, and 1-D excitations
journal, February 2018

Abell, José A.; Orbović, Nebojša; McCallen, David B.
Earthquake Engineering & Structural Dynamics, Vol. 47, Issue 6
DOI: 10.1002/eqe.3026

Similar Records in DOE PAGES and OSTI.GOV collections:

Regional-Scale 3D Ground-Motion Simulations of M_w 7 Earthquakes on the Hayward Fault, Northern California Resolving Frequencies 0–10 Hz and Including Site-Response Corrections

Journal Article Rodgers, Arthur J. ; Pitarka, Arben ; Pankajakshan, Ramesh ; ... - Bulletin of the Seismological Society of America

Large earthquake ground-motion simulations in 3D Earth models provide constraints on site-specific shaking intensities but have suffered from limited frequency resolution and ignored site response in soft soils. In this work we report new regional-scale 3D simulations for moment magnitude 7.0 scenario earthquakes on the Hayward Fault, northern California with SW4. Simulations resolved significantly broader band frequencies (0–10 Hz) than previous studies and represent the highest resolution simulations for any such earthquake to date. Seismic waves were excited by a kinematic rupture following Graves and Pitarka (2016) and obeyed wave propagation in a 3D Earth model with topography from the U.S.more »« less
https://doi.org/10.1785/0120200147

Full Text Available
Broadband (0–5 Hz) Fully Deterministic 3D Ground-Motion Simulations of a Magnitude 7.0 Hayward Fault Earthquake: Comparison with Empirical Ground-Motion Models and 3D Path and Site Effects from Source Normalized Intensities

Journal Article Rodgers, Arthur J. ; Anders Petersson, N. ; Pitarka, Arben ; ... - Seismological Research Letters

We report on high-performance computing (HPC) fully deterministic simulation of ground motions for a moment magnitude (Mw) 7.0 scenario earthquake on the Hayward fault resolved to 5 Hz using the SW4 finite-difference code. We computed motions obeying physics-based 3D wave propagation at a regional scale with an Mw 7.0 kinematic rupture model generated following Graves and Pitarka (2016). Both plane-layered (1D) and 3D Earth models were considered, with 3D subsurface material properties and topography interpolated from a model of the U.S. Geological Survey (USGS). The resulting ground-motion intensities cover a broader frequency range than typically considered in regional-scale simulations, includingmore »« less
Cited by 30
https://doi.org/10.1785/0220180261

Full Text Available
Broadband (0–5 Hz) Fully Deterministic 3D Ground‐Motion Simulations of a Magnitude 7.0 Hayward Fault Earthquake: Comparison with Empirical Ground‐Motion Models and 3D Path and Site Effects from Source Normalized Intensities

Journal Article Rodgers, Arthur J. ; Anders Petersson, N. ; Pitarka, Arben ; ... - Seismological Research Letters

We report on high‐performance computing (HPC) fully deterministic simulation of ground motions for a moment magnitude (⁠Mw⁠) 7.0 scenario earthquake on the Hayward fault resolved to 5 Hz using the SW4 finite‐difference code. We computed motions obeying physics‐based 3D wave propagation at a regional scale with an Mw 7.0 kinematic rupture model generated following Graves and Pitarka (2016). Both plane‐layered (1D) and 3D Earth models were considered, with 3D subsurface material properties and topography interpolated from a model of the U.S. Geological Survey (USGS). The resulting ground‐motion intensities cover a broader frequency range than typically considered in regional‐scale simulations, includingmore »« less
https://doi.org/10.1785/0220180261
Earthquake Ground Motion Simulations on the Sierra High-Performance Computing System

Technical Report Rodgers, Arthur J.

Physics-based three-dimensional (3D) numerical simulations of earthquake ground motions provide an alternative to reliance on limited and highly scattered empirical data and models for seismic hazard assessment. We performed fully 3D earthquake ground motion simulations on the Sierra and Lassen high-performance computing (HPC) platforms at Livermore Computing. These new HPC platforms feature graphics processing unit (GPU) acceleration for improved performance. Ground motions were computed with SW4, a summation-by-parts finite difference code for anelastic wave propagation. SW4 was recently enhanced to run efficiently on GPU-accelerated platforms. We found that SW4 runs almost fifty (50) times faster in terms of node-hours onmore »« less
https://doi.org/10.2172/1571735

Full Text Available
Running SW4 On New Commodity Technology Systems (CTS-1) Platform

Technical Report Rodgers, Arthur J. ; Petersson, N. Anders ; Pitarka, Arben ; ...

We have recently been running earthquake ground motion simulations with SW4 on the new capacity computing systems, called the Commodity Technology Systems - 1 (CTS-1) at Lawrence Livermore National Laboratory (LLNL). SW4 is a fourth order time domain finite difference code developed by LLNL and distributed by the Computational Infrastructure for Geodynamics (CIG). SW4 simulates seismic wave propagation in complex three-dimensional Earth models including anelasticity and surface topography. We are modeling near-fault earthquake strong ground motions for the purposes of evaluating the response of engineered structures, such as nuclear power plants and other critical infrastructure. Engineering analysis of structures requiresmore »« less
https://doi.org/10.2172/1321435

Full Text Available

Similar Records

Title: Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis

Abstract

Citation Formats

Earthquake soil-structure interaction of nuclear power plants, differences in response to 3-D, 3 × 1-D, and 1-D excitations journal, February 2018

Earthquake soil-structure interaction of nuclear power plants, differences in response to 3-D, 3 × 1-D, and 1-D excitations
journal, February 2018