DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Laboratory earthquake forecasting: A machine learning competition

Abstract

Earthquake prediction, the long-sought holy grail of earthquake science, continues to confound Earth scientists. Could we make advances by crowdsourcing, drawing from the vast knowledge and creativity of the machine learning (ML) community? We used Google’s ML competition platform, Kaggle, to engage the worldwide ML community with a competition to develop and improve data analysis approaches on a forecasting problem that uses laboratory earthquake data. The competitors were tasked with predicting the time remaining before the next earthquake of successive laboratory quake events, based on only a small portion of the laboratory seismic data. The more than 4,500 participating teams created and shared more than 400 computer programs in openly accessible notebooks. Complementing the now well-known features of seismic data that map to fault criticality in the laboratory, the winning teams employed unexpected strategies based on rescaling failure times as a fraction of the seismic cycle and comparing input distribution of training and testing data. In addition to yielding scientific insights into fault processes in the laboratory and their relation with the evolution of the statistical properties of the associated seismic data, the competition serves as a pedagogical tool for teaching ML in geophysics. The approach may provide a modelmore » for other competitions in geosciences or other domains of study to help engage the ML community on problems of significance.« less

Authors:
 [1];  [1]; ORCiD logo [2]; ORCiD logo [3];  [4];  [5];  [6];  [7]; ORCiD logo [7]; ORCiD logo [8]; ORCiD logo [9];  [10]; ORCiD logo [11];  [6]
  1. Geophysics Group, Los Alamos National Laboratory, Los Alamos, NM 87545,
  2. Department of Physics and Astronomy, Purdue University, West Lafayette, IN 47907,, Department of Earth, Atmospheric and Planetary Sciences, Purdue University, West Lafayette, IN 47907,, Lyles School of Civil Engineering, Purdue University, West Lafayette, IN 47907,
  3. Department of Geophysics, Stanford University, Stanford, CA 94305,
  4. Department of Earth Science, La Sapienza Università di Roma, 00413 Rome, Italy,, Department of Earth Science, Pennsylvania State University, University Park, PA 16802,
  5. Laboratoire de Géologie, Département de Géosciences, École Normale Supérieure, PSL University, CNRS UMR, 8538 Paris, France,
  6. Kaggle, Google, LLC, Denver, CO 80301,
  7. H2O.ai, 1010 Vienna, Austria,
  8. Private individual, Athens 11364, Greece,
  9. Private individual, Jacksonville, FL, 32207,
  10. Department of Electrical Engineering, Rheinisch-Westfälische Technische Hochschule Aachen University, 52056 Aachen, Germany,
  11. Private individual, Bethesda, MD 20817
Publication Date:
Research Org.:
Stanford Univ., CA (United States); Pennsylvania State Univ., University Park, PA (United States); Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES). Chemical Sciences, Geosciences & Biosciences Division; USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Basic Energy Sciences (BES)
OSTI Identifier:
1762534
Alternate Identifier(s):
OSTI ID: 1774648; OSTI ID: 1829644; OSTI ID: 1831406; OSTI ID: 1861296; OSTI ID: 2267616
Report Number(s):
LA-UR-20-28829; LA-UR-20-26035
Journal ID: ISSN 0027-8424; e2011362118
Grant/Contract Number:  
9233218CNA000001; FG02-09ER16022; SC0020445; SC0020512; EE0008763; 20200278ER; 89233218CNA000001
Resource Type:
Published Article
Journal Name:
Proceedings of the National Academy of Sciences of the United States of America
Additional Journal Information:
Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Volume: 118 Journal Issue: 5; Journal ID: ISSN 0027-8424
Publisher:
Proceedings of the National Academy of Sciences
Country of Publication:
United States
Language:
English
Subject:
58 GEOSCIENCES; machine learning competition; laboratory earthquakes; earthquake prediction; physics of faulting; machine learning; artificial intelligence; forecasting; Earth Sciences; Seismicity

Citation Formats

Johnson, Paul A., Rouet-Leduc, Bertrand, Pyrak-Nolte, Laura J., Beroza, Gregory C., Marone, Chris J., Hulbert, Claudia, Howard, Addison, Singer, Philipp, Gordeev, Dmitry, Karaflos, Dimosthenis, Levinson, Corey J., Pfeiffer, Pascal, Puk, Kin Ming, and Reade, Walter. Laboratory earthquake forecasting: A machine learning competition. United States: N. p., 2021. Web. doi:10.1073/pnas.2011362118.
Johnson, Paul A., Rouet-Leduc, Bertrand, Pyrak-Nolte, Laura J., Beroza, Gregory C., Marone, Chris J., Hulbert, Claudia, Howard, Addison, Singer, Philipp, Gordeev, Dmitry, Karaflos, Dimosthenis, Levinson, Corey J., Pfeiffer, Pascal, Puk, Kin Ming, & Reade, Walter. Laboratory earthquake forecasting: A machine learning competition. United States. https://doi.org/10.1073/pnas.2011362118
Johnson, Paul A., Rouet-Leduc, Bertrand, Pyrak-Nolte, Laura J., Beroza, Gregory C., Marone, Chris J., Hulbert, Claudia, Howard, Addison, Singer, Philipp, Gordeev, Dmitry, Karaflos, Dimosthenis, Levinson, Corey J., Pfeiffer, Pascal, Puk, Kin Ming, and Reade, Walter. Mon . "Laboratory earthquake forecasting: A machine learning competition". United States. https://doi.org/10.1073/pnas.2011362118.
@article{osti_1762534,
title = {Laboratory earthquake forecasting: A machine learning competition},
author = {Johnson, Paul A. and Rouet-Leduc, Bertrand and Pyrak-Nolte, Laura J. and Beroza, Gregory C. and Marone, Chris J. and Hulbert, Claudia and Howard, Addison and Singer, Philipp and Gordeev, Dmitry and Karaflos, Dimosthenis and Levinson, Corey J. and Pfeiffer, Pascal and Puk, Kin Ming and Reade, Walter},
abstractNote = {Earthquake prediction, the long-sought holy grail of earthquake science, continues to confound Earth scientists. Could we make advances by crowdsourcing, drawing from the vast knowledge and creativity of the machine learning (ML) community? We used Google’s ML competition platform, Kaggle, to engage the worldwide ML community with a competition to develop and improve data analysis approaches on a forecasting problem that uses laboratory earthquake data. The competitors were tasked with predicting the time remaining before the next earthquake of successive laboratory quake events, based on only a small portion of the laboratory seismic data. The more than 4,500 participating teams created and shared more than 400 computer programs in openly accessible notebooks. Complementing the now well-known features of seismic data that map to fault criticality in the laboratory, the winning teams employed unexpected strategies based on rescaling failure times as a fraction of the seismic cycle and comparing input distribution of training and testing data. In addition to yielding scientific insights into fault processes in the laboratory and their relation with the evolution of the statistical properties of the associated seismic data, the competition serves as a pedagogical tool for teaching ML in geophysics. The approach may provide a model for other competitions in geosciences or other domains of study to help engage the ML community on problems of significance.},
doi = {10.1073/pnas.2011362118},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 5,
volume = 118,
place = {United States},
year = {Mon Jan 25 00:00:00 EST 2021},
month = {Mon Jan 25 00:00:00 EST 2021}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1073/pnas.2011362118

Save / Share:

Works referenced in this record:

Acoustic emission and microslip precursors to stick-slip failure in sheared granular material: AE AND MICROSLIP PRECURSORS
journal, November 2013

  • Johnson, P. A.; Ferdowsi, B.; Kaproth, B. M.
  • Geophysical Research Letters, Vol. 40, Issue 21
  • DOI: 10.1002/2013GL057848

Classification of Seismic Windows Using Artificial Neural Networks
journal, January 2011


Detachment fronts and the onset of dynamic friction
journal, August 2004

  • Rubinstein, Shmuel M.; Cohen, Gil; Fineberg, Jay
  • Nature, Vol. 430, Issue 7003
  • DOI: 10.1038/nature02830

Machine Learning in Seismology: Turning Data into Insights
journal, November 2018

  • Kong, Qingkai; Trugman, Daniel T.; Ross, Zachary E.
  • Seismological Research Letters, Vol. 90, Issue 1
  • DOI: 10.1785/0220180259

A deep learning approach to detecting volcano deformation from satellite imagery using synthetic datasets
journal, September 2019


CRED: A Deep Residual Network of Convolutional and Recurrent Units for Earthquake Signal Detection
journal, July 2019


Slow Earthquakes, Preseismic Velocity Changes, and the Origin of Slow Frictional Stick-Slip
journal, August 2013


Travel Time Tomography With Adaptive Dictionaries
journal, December 2018

  • Bianco, Michael J.; Gerstoft, Peter
  • IEEE Transactions on Computational Imaging, Vol. 4, Issue 4
  • DOI: 10.1109/TCI.2018.2862644

Similarity of fast and slow earthquakes illuminated by machine learning
journal, December 2018

  • Hulbert, Claudia; Rouet-Leduc, Bertrand; Johnson, Paul A.
  • Nature Geoscience, Vol. 12, Issue 1
  • DOI: 10.1038/s41561-018-0272-8

Continuous chatter of the Cascadia subduction zone revealed by machine learning
journal, December 2018

  • Rouet-Leduc, Bertrand; Hulbert, Claudia; Johnson, Paul A.
  • Nature Geoscience, Vol. 12, Issue 1
  • DOI: 10.1038/s41561-018-0274-6

Deep-learning tomography
journal, January 2018

  • Araya-Polo, Mauricio; Jennings, Joseph; Adler, Amir
  • The Leading Edge, Vol. 37, Issue 1
  • DOI: 10.1190/tle37010058.1

A Silent Slip Event on the Deeper Cascadia Subduction Interface
journal, April 2001


Earthquake detection through computationally efficient similarity search
journal, December 2015

  • Yoon, Clara E.; O’Reilly, Ossian; Bergen, Karianne J.
  • Science Advances, Vol. 1, Issue 11
  • DOI: 10.1126/sciadv.1501057

Intense foreshocks and a slow slip event preceded the 2014 Iquique Mw 8.1 earthquake
journal, July 2014


The long precursory phase of most large interplate earthquakes
journal, March 2013

  • Bouchon, Michel; Durand, Virginie; Marsan, David
  • Nature Geoscience, Vol. 6, Issue 4
  • DOI: 10.1038/ngeo1770

Earthquake prediction: a critical review
journal, December 1997


PhaseLink: A Deep Learning Approach to Seismic Phase Association
journal, January 2019

  • Ross, Zachary E.; Yue, Yisong; Meier, Men‐Andrin
  • Journal of Geophysical Research: Solid Earth, Vol. 124, Issue 1
  • DOI: 10.1029/2018JB016674

Characterizing Acoustic Signals and Searching for Precursors during the Laboratory Seismic Cycle Using Unsupervised Machine Learning
journal, March 2019

  • Bolton, David C.; Shokouhi, Parisa; Rouet‐Leduc, Bertrand
  • Seismological Research Letters, Vol. 90, Issue 3
  • DOI: 10.1785/0220180367

Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)
journal, April 2000

  • Friedman, Jerome; Hastie, Trevor; Tibshirani, Robert
  • The Annals of Statistics, Vol. 28, Issue 2
  • DOI: 10.1214/aos/1016218223

Machine Learning Predicts Laboratory Earthquakes: MACHINE LEARNING PREDICTS LAB QUAKES
journal, September 2017

  • Rouet-Leduc, Bertrand; Hulbert, Claudia; Lubbers, Nicholas
  • Geophysical Research Letters, Vol. 44, Issue 18
  • DOI: 10.1002/2017GL074677

On the evolution of elastic properties during laboratory stick-slip experiments spanning the transition from slow slip to dynamic rupture: PRECURSORS TO THE SPECTRUM OF SLIP MODES
journal, December 2016

  • Tinti, E.; Scuderi, M. M.; Scognamiglio, L.
  • Journal of Geophysical Research: Solid Earth, Vol. 121, Issue 12
  • DOI: 10.1002/2016JB013545

Extended Nucleation of the 1999 Mw 7.6 Izmit Earthquake
journal, February 2011


Pervasive Foreshock Activity Across Southern California
journal, August 2019

  • Trugman, Daniel T.; Ross, Zachary E.
  • Geophysical Research Letters, Vol. 46, Issue 15
  • DOI: 10.1029/2019GL083725

Frictional strength and strain weakening in simulated fault gouge: Competition between geometrical weakening and chemical strengthening
journal, January 2010

  • Niemeijer, André; Marone, Chris; Elsworth, Derek
  • Journal of Geophysical Research, Vol. 115, Issue B10
  • DOI: 10.1029/2009JB000838

Probing Slow Earthquakes With Deep Learning
journal, February 2020

  • Rouet‐Leduc, Bertrand; Hulbert, Claudia; McBrearty, Ian W.
  • Geophysical Research Letters, Vol. 47, Issue 4
  • DOI: 10.1029/2019GL085870

Machine Learning Can Predict the Timing and Size of Analog Earthquakes
journal, February 2019

  • Corbi, F.; Sandri, L.; Bedford, J.
  • Geophysical Research Letters, Vol. 46, Issue 3
  • DOI: 10.1029/2018GL081251

Propagation of Slow Slip Leading Up to the 2011 Mw 9.0 Tohoku-Oki Earthquake
journal, January 2012


An exponential build-up in seismic energy suggests a months-long nucleation of slow slip in Cascadia
journal, August 2020

  • Hulbert, Claudia; Rouet-Leduc, Bertrand; Jolivet, Romain
  • Nature Communications, Vol. 11, Issue 1
  • DOI: 10.1038/s41467-020-17754-9

An Investigation of Rapid Earthquake Characterization Using Single‐Station Waveforms and a Convolutional Neural Network
journal, February 2019

  • Lomax, Anthony; Michelini, Alberto; Jozinović, Dario
  • Seismological Research Letters, Vol. 90, Issue 2A
  • DOI: 10.1785/0220180311

Emergent Wave Conversion as a Precursor to Shear Crack Initiation
journal, September 2018

  • Modiriasari, Anahita; Pyrak-Nolte, Laura J.; Bobet, Antonio
  • Geophysical Research Letters, Vol. 45, Issue 18
  • DOI: 10.1029/2018GL078622

Connecting slow earthquakes to huge earthquakes
journal, July 2016


Earthquake Catalog‐Based Machine Learning Identification of Laboratory Fault States and the Effects of Magnitude of Completeness
journal, December 2018

  • Lubbers, Nicholas; Bolton, David C.; Mohd‐Yusof, Jamaludin
  • Geophysical Research Letters, Vol. 45, Issue 24
  • DOI: 10.1029/2018GL079712

Foreshocks and Mainshock Nucleation of the 1999 M w 7.1 Hector Mine, California, Earthquake
journal, February 2019

  • Yoon, Clara E.; Yoshimitsu, Nana; Ellsworth, William L.
  • Journal of Geophysical Research: Solid Earth, Vol. 124, Issue 2
  • DOI: 10.1029/2018JB016383

Machine learning for graph-based representations of three-dimensional discrete fracture networks
journal, January 2018

  • Valera, Manuel; Guo, Zhengyang; Kelly, Priscilla
  • Computational Geosciences, Vol. 22, Issue 3
  • DOI: 10.1007/s10596-018-9720-1

Seismic velocity precursors to the 2016 Mw 6.5 Norcia (Italy) earthquake
journal, June 2020

  • Chiarabba, C.; De Gori, P.; Segou, M.
  • Geology, Vol. 48, Issue 9
  • DOI: 10.1130/G47048.1

Evolution of b-value during the seismic cycle: Insights from laboratory experiments on simulated faults
journal, January 2018


Precursor-Like Anomalies prior to the 2008 Wenchuan Earthquake: A Critical-but-Constructive Review
journal, January 2012

  • Ma, Tengfei; Wu, Zhongliang
  • International Journal of Geophysics, Vol. 2012
  • DOI: 10.1155/2012/583097

Nonvolcanic Deep Tremor Associated with Subduction in Southwest Japan
journal, May 2002


Machine learning for data-driven discovery in solid Earth geoscience
journal, March 2019

  • Bergen, Karianne J.; Johnson, Paul A.; de Hoop, Maarten V.
  • Science, Vol. 363, Issue 6433
  • DOI: 10.1126/science.aau0323

Numerical simulation of earthquake sequences
journal, October 1977

  • Rundle, John B.; Jackson, David D.
  • Bulletin of the Seismological Society of America, Vol. 67, Issue 5
  • DOI: 10.1785/BSSA0670051363

Towards Global Volcano Monitoring Using Multisensor Sentinel Missions and Artificial Intelligence: The MOUNTS Monitoring System
journal, June 2019

  • Valade, Sébastien; Ley, Andreas; Massimetti, Francesco
  • Remote Sensing, Vol. 11, Issue 13
  • DOI: 10.3390/rs11131528

Precursors to the shear failure of rock discontinuities: Precursors to Rock Frictional Sliding
journal, August 2014

  • Hedayat, Ahmadreza; Pyrak-Nolte, Laura J.; Bobet, Antonio
  • Geophysical Research Letters, Vol. 41, Issue 15
  • DOI: 10.1002/2014GL060848

Frictional Mechanics of Slow Earthquakes
journal, September 2018

  • Leeman, J. R.; Marone, C.; Saffer, D. M.
  • Journal of Geophysical Research: Solid Earth, Vol. 123, Issue 9
  • DOI: 10.1029/2018JB015768

Effect of humidity on time- and velocity-dependent friction in rocks
journal, June 1984

  • Dieterich, James H.; Conrad, Gerald
  • Journal of Geophysical Research: Solid Earth, Vol. 89, Issue B6
  • DOI: 10.1029/JB089iB06p04196

The Mechanics of Earthquakes and Faulting
book, December 2018


Micromechanics of asperity rupture during laboratory stick slip experiments: ASPERITY RUPTURE MICROMECHANICS
journal, June 2011

  • McLaskey, Gregory C.; Glaser, Steven D.
  • Geophysical Research Letters, Vol. 38, Issue 12
  • DOI: 10.1029/2011GL047507

Earthquake nucleation on (aging) rate and state faults: RATE AND STATE EARTHQUAKE NUCLEATION
journal, November 2005

  • Rubin, A. M.; Ampuero, J. -P.
  • Journal of Geophysical Research: Solid Earth, Vol. 110, Issue B11
  • DOI: 10.1029/2005JB003686

Studies on earthquake precursors in China: A review for recent 50 years
journal, January 2017


Multiple slow-slip events during a foreshock sequence of the 2014 Iquique, Chile M w 8.1 earthquake
journal, August 2014

  • Kato, Aitaro; Nakagawa, Shigeki
  • Geophysical Research Letters, Vol. 41, Issue 15
  • DOI: 10.1002/2014GL061138

Neural Network Applications in Earthquake Prediction (1994–2019): Meta-Analytic and Statistical Insights on Their Limitations
journal, May 2020

  • Mignan, Arnaud; Broccardo, Marco
  • Seismological Research Letters, Vol. 91, Issue 4
  • DOI: 10.1785/0220200021

Generalized Seismic Phase Detection with Deep Learning
journal, August 2018

  • Ross, Zachary E.; Meier, Men‐Andrin; Hauksson, Egill
  • Bulletin of the Seismological Society of America, Vol. 108, Issue 5A
  • DOI: 10.1785/0120180080

Triggering of the 2014 Mw7.3 Papanoa earthquake by a slow slip event in Guerrero, Mexico
journal, October 2016

  • Radiguet, M.; Perfettini, H.; Cotte, N.
  • Nature Geoscience, Vol. 9, Issue 11
  • DOI: 10.1038/ngeo2817

Seismic Evidence for an Earthquake Nucleation Phase
journal, May 1995


Episodic slow slip events and rate-and-state friction
journal, January 2008


Laboratory-Derived Friction laws and Their Application to Seismic Faulting
journal, May 1998


Do slow slip events trigger large and great megathrust earthquakes?
journal, October 2018


Preseismic Fault Creep and Elastic Wave Amplitude Precursors Scale With Lab Earthquake Magnitude for the Continuum of Tectonic Failure Modes
journal, April 2020

  • Shreedharan, Srisharan; Bolton, David Chas; Rivière, Jacques
  • Geophysical Research Letters, Vol. 47, Issue 8
  • DOI: 10.1029/2020GL086986

Random Forests
journal, January 2001


PhaseNet: A Deep-Neural-Network-Based Seismic Arrival Time Picking Method
journal, October 2018

  • Zhu, Weiqiang; Beroza, Gregory C.
  • Geophysical Journal International
  • DOI: 10.1093/gji/ggy423

Slow Earthquakes and Nonvolcanic Tremor
journal, May 2011


Modeling flow and transport in fracture networks using graphs
journal, March 2018


Precursory changes in seismic velocity for the spectrum of earthquake failure modes
journal, August 2016

  • Scuderi, M. M.; Marone, C.; Tinti, E.
  • Nature Geoscience, Vol. 9, Issue 9
  • DOI: 10.1038/ngeo2775