Laboratory earthquake forecasting: A machine learning competition
Abstract
Earthquake prediction, the long-sought holy grail of earthquake science, continues to confound Earth scientists. Could we make advances by crowdsourcing, drawing from the vast knowledge and creativity of the machine learning (ML) community? We used Google’s ML competition platform, Kaggle, to engage the worldwide ML community with a competition to develop and improve data analysis approaches on a forecasting problem that uses laboratory earthquake data. The competitors were tasked with predicting the time remaining before the next earthquake of successive laboratory quake events, based on only a small portion of the laboratory seismic data. The more than 4,500 participating teams created and shared more than 400 computer programs in openly accessible notebooks. Complementing the now well-known features of seismic data that map to fault criticality in the laboratory, the winning teams employed unexpected strategies based on rescaling failure times as a fraction of the seismic cycle and comparing input distribution of training and testing data. In addition to yielding scientific insights into fault processes in the laboratory and their relation with the evolution of the statistical properties of the associated seismic data, the competition serves as a pedagogical tool for teaching ML in geophysics. The approach may provide a modelmore »
- Authors:
-
- Geophysics Group, Los Alamos National Laboratory, Los Alamos, NM 87545,
- Department of Physics and Astronomy, Purdue University, West Lafayette, IN 47907,, Department of Earth, Atmospheric and Planetary Sciences, Purdue University, West Lafayette, IN 47907,, Lyles School of Civil Engineering, Purdue University, West Lafayette, IN 47907,
- Department of Geophysics, Stanford University, Stanford, CA 94305,
- Department of Earth Science, La Sapienza Università di Roma, 00413 Rome, Italy,, Department of Earth Science, Pennsylvania State University, University Park, PA 16802,
- Laboratoire de Géologie, Département de Géosciences, École Normale Supérieure, PSL University, CNRS UMR, 8538 Paris, France,
- Kaggle, Google, LLC, Denver, CO 80301,
- H2O.ai, 1010 Vienna, Austria,
- Private individual, Athens 11364, Greece,
- Private individual, Jacksonville, FL, 32207,
- Department of Electrical Engineering, Rheinisch-Westfälische Technische Hochschule Aachen University, 52056 Aachen, Germany,
- Private individual, Bethesda, MD 20817
- Publication Date:
- Research Org.:
- Stanford Univ., CA (United States); Pennsylvania State Univ., University Park, PA (United States); Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Basic Energy Sciences (BES). Chemical Sciences, Geosciences & Biosciences Division; USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Basic Energy Sciences (BES)
- OSTI Identifier:
- 1762534
- Alternate Identifier(s):
- OSTI ID: 1774648; OSTI ID: 1829644; OSTI ID: 1831406; OSTI ID: 1861296; OSTI ID: 2267616
- Report Number(s):
- LA-UR-20-28829; LA-UR-20-26035
Journal ID: ISSN 0027-8424; e2011362118
- Grant/Contract Number:
- 9233218CNA000001; FG02-09ER16022; SC0020445; SC0020512; EE0008763; 20200278ER; 89233218CNA000001
- Resource Type:
- Published Article
- Journal Name:
- Proceedings of the National Academy of Sciences of the United States of America
- Additional Journal Information:
- Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Volume: 118 Journal Issue: 5; Journal ID: ISSN 0027-8424
- Publisher:
- Proceedings of the National Academy of Sciences
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 58 GEOSCIENCES; machine learning competition; laboratory earthquakes; earthquake prediction; physics of faulting; machine learning; artificial intelligence; forecasting; Earth Sciences; Seismicity
Citation Formats
Johnson, Paul A., Rouet-Leduc, Bertrand, Pyrak-Nolte, Laura J., Beroza, Gregory C., Marone, Chris J., Hulbert, Claudia, Howard, Addison, Singer, Philipp, Gordeev, Dmitry, Karaflos, Dimosthenis, Levinson, Corey J., Pfeiffer, Pascal, Puk, Kin Ming, and Reade, Walter. Laboratory earthquake forecasting: A machine learning competition. United States: N. p., 2021.
Web. doi:10.1073/pnas.2011362118.
Johnson, Paul A., Rouet-Leduc, Bertrand, Pyrak-Nolte, Laura J., Beroza, Gregory C., Marone, Chris J., Hulbert, Claudia, Howard, Addison, Singer, Philipp, Gordeev, Dmitry, Karaflos, Dimosthenis, Levinson, Corey J., Pfeiffer, Pascal, Puk, Kin Ming, & Reade, Walter. Laboratory earthquake forecasting: A machine learning competition. United States. https://doi.org/10.1073/pnas.2011362118
Johnson, Paul A., Rouet-Leduc, Bertrand, Pyrak-Nolte, Laura J., Beroza, Gregory C., Marone, Chris J., Hulbert, Claudia, Howard, Addison, Singer, Philipp, Gordeev, Dmitry, Karaflos, Dimosthenis, Levinson, Corey J., Pfeiffer, Pascal, Puk, Kin Ming, and Reade, Walter. Mon .
"Laboratory earthquake forecasting: A machine learning competition". United States. https://doi.org/10.1073/pnas.2011362118.
@article{osti_1762534,
title = {Laboratory earthquake forecasting: A machine learning competition},
author = {Johnson, Paul A. and Rouet-Leduc, Bertrand and Pyrak-Nolte, Laura J. and Beroza, Gregory C. and Marone, Chris J. and Hulbert, Claudia and Howard, Addison and Singer, Philipp and Gordeev, Dmitry and Karaflos, Dimosthenis and Levinson, Corey J. and Pfeiffer, Pascal and Puk, Kin Ming and Reade, Walter},
abstractNote = {Earthquake prediction, the long-sought holy grail of earthquake science, continues to confound Earth scientists. Could we make advances by crowdsourcing, drawing from the vast knowledge and creativity of the machine learning (ML) community? We used Google’s ML competition platform, Kaggle, to engage the worldwide ML community with a competition to develop and improve data analysis approaches on a forecasting problem that uses laboratory earthquake data. The competitors were tasked with predicting the time remaining before the next earthquake of successive laboratory quake events, based on only a small portion of the laboratory seismic data. The more than 4,500 participating teams created and shared more than 400 computer programs in openly accessible notebooks. Complementing the now well-known features of seismic data that map to fault criticality in the laboratory, the winning teams employed unexpected strategies based on rescaling failure times as a fraction of the seismic cycle and comparing input distribution of training and testing data. In addition to yielding scientific insights into fault processes in the laboratory and their relation with the evolution of the statistical properties of the associated seismic data, the competition serves as a pedagogical tool for teaching ML in geophysics. The approach may provide a model for other competitions in geosciences or other domains of study to help engage the ML community on problems of significance.},
doi = {10.1073/pnas.2011362118},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 5,
volume = 118,
place = {United States},
year = {Mon Jan 25 00:00:00 EST 2021},
month = {Mon Jan 25 00:00:00 EST 2021}
}
https://doi.org/10.1073/pnas.2011362118
Works referenced in this record:
Acoustic emission and microslip precursors to stick-slip failure in sheared granular material: AE AND MICROSLIP PRECURSORS
journal, November 2013
- Johnson, P. A.; Ferdowsi, B.; Kaproth, B. M.
- Geophysical Research Letters, Vol. 40, Issue 21
Classification of Seismic Windows Using Artificial Neural Networks
journal, January 2011
- Diersen, Steve; Lee, En-Jui; Spears, Diana
- Procedia Computer Science, Vol. 4
Detachment fronts and the onset of dynamic friction
journal, August 2004
- Rubinstein, Shmuel M.; Cohen, Gil; Fineberg, Jay
- Nature, Vol. 430, Issue 7003
Machine Learning in Seismology: Turning Data into Insights
journal, November 2018
- Kong, Qingkai; Trugman, Daniel T.; Ross, Zachary E.
- Seismological Research Letters, Vol. 90, Issue 1
A deep learning approach to detecting volcano deformation from satellite imagery using synthetic datasets
journal, September 2019
- Anantrasirichai, N.; Biggs, J.; Albino, F.
- Remote Sensing of Environment, Vol. 230
CRED: A Deep Residual Network of Convolutional and Recurrent Units for Earthquake Signal Detection
journal, July 2019
- Mousavi, S. Mostafa; Zhu, Weiqiang; Sheng, Yixiao
- Scientific Reports, Vol. 9, Issue 1
Slow Earthquakes, Preseismic Velocity Changes, and the Origin of Slow Frictional Stick-Slip
journal, August 2013
- Kaproth, B. M.; Marone, C.
- Science, Vol. 341, Issue 6151
Travel Time Tomography With Adaptive Dictionaries
journal, December 2018
- Bianco, Michael J.; Gerstoft, Peter
- IEEE Transactions on Computational Imaging, Vol. 4, Issue 4
Similarity of fast and slow earthquakes illuminated by machine learning
journal, December 2018
- Hulbert, Claudia; Rouet-Leduc, Bertrand; Johnson, Paul A.
- Nature Geoscience, Vol. 12, Issue 1
Continuous chatter of the Cascadia subduction zone revealed by machine learning
journal, December 2018
- Rouet-Leduc, Bertrand; Hulbert, Claudia; Johnson, Paul A.
- Nature Geoscience, Vol. 12, Issue 1
Deep-learning tomography
journal, January 2018
- Araya-Polo, Mauricio; Jennings, Joseph; Adler, Amir
- The Leading Edge, Vol. 37, Issue 1
A Silent Slip Event on the Deeper Cascadia Subduction Interface
journal, April 2001
- Dragert, H.
- Science, Vol. 292, Issue 5521
Earthquake detection through computationally efficient similarity search
journal, December 2015
- Yoon, Clara E.; O’Reilly, Ossian; Bergen, Karianne J.
- Science Advances, Vol. 1, Issue 11
Intense foreshocks and a slow slip event preceded the 2014 Iquique Mw 8.1 earthquake
journal, July 2014
- Ruiz, S.; Metois, M.; Fuenzalida, A.
- Science, Vol. 345, Issue 6201
The long precursory phase of most large interplate earthquakes
journal, March 2013
- Bouchon, Michel; Durand, Virginie; Marsan, David
- Nature Geoscience, Vol. 6, Issue 4
Earthquake prediction: a critical review
journal, December 1997
- Geller, Robert J.
- Geophysical Journal International, Vol. 131, Issue 3
PhaseLink: A Deep Learning Approach to Seismic Phase Association
journal, January 2019
- Ross, Zachary E.; Yue, Yisong; Meier, Men‐Andrin
- Journal of Geophysical Research: Solid Earth, Vol. 124, Issue 1
Characterizing Acoustic Signals and Searching for Precursors during the Laboratory Seismic Cycle Using Unsupervised Machine Learning
journal, March 2019
- Bolton, David C.; Shokouhi, Parisa; Rouet‐Leduc, Bertrand
- Seismological Research Letters, Vol. 90, Issue 3
Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)
journal, April 2000
- Friedman, Jerome; Hastie, Trevor; Tibshirani, Robert
- The Annals of Statistics, Vol. 28, Issue 2
Machine Learning Predicts Laboratory Earthquakes: MACHINE LEARNING PREDICTS LAB QUAKES
journal, September 2017
- Rouet-Leduc, Bertrand; Hulbert, Claudia; Lubbers, Nicholas
- Geophysical Research Letters, Vol. 44, Issue 18
On the evolution of elastic properties during laboratory stick-slip experiments spanning the transition from slow slip to dynamic rupture: PRECURSORS TO THE SPECTRUM OF SLIP MODES
journal, December 2016
- Tinti, E.; Scuderi, M. M.; Scognamiglio, L.
- Journal of Geophysical Research: Solid Earth, Vol. 121, Issue 12
Extended Nucleation of the 1999 Mw 7.6 Izmit Earthquake
journal, February 2011
- Bouchon, M.; Karabulut, H.; Aktar, M.
- Science, Vol. 331, Issue 6019
Pervasive Foreshock Activity Across Southern California
journal, August 2019
- Trugman, Daniel T.; Ross, Zachary E.
- Geophysical Research Letters, Vol. 46, Issue 15
Frictional strength and strain weakening in simulated fault gouge: Competition between geometrical weakening and chemical strengthening
journal, January 2010
- Niemeijer, André; Marone, Chris; Elsworth, Derek
- Journal of Geophysical Research, Vol. 115, Issue B10
Probing Slow Earthquakes With Deep Learning
journal, February 2020
- Rouet‐Leduc, Bertrand; Hulbert, Claudia; McBrearty, Ian W.
- Geophysical Research Letters, Vol. 47, Issue 4
Machine Learning Can Predict the Timing and Size of Analog Earthquakes
journal, February 2019
- Corbi, F.; Sandri, L.; Bedford, J.
- Geophysical Research Letters, Vol. 46, Issue 3
Propagation of Slow Slip Leading Up to the 2011 Mw 9.0 Tohoku-Oki Earthquake
journal, January 2012
- Kato, A.; Obara, K.; Igarashi, T.
- Science, Vol. 335, Issue 6069
An exponential build-up in seismic energy suggests a months-long nucleation of slow slip in Cascadia
journal, August 2020
- Hulbert, Claudia; Rouet-Leduc, Bertrand; Jolivet, Romain
- Nature Communications, Vol. 11, Issue 1
An Investigation of Rapid Earthquake Characterization Using Single‐Station Waveforms and a Convolutional Neural Network
journal, February 2019
- Lomax, Anthony; Michelini, Alberto; Jozinović, Dario
- Seismological Research Letters, Vol. 90, Issue 2A
Emergent Wave Conversion as a Precursor to Shear Crack Initiation
journal, September 2018
- Modiriasari, Anahita; Pyrak-Nolte, Laura J.; Bobet, Antonio
- Geophysical Research Letters, Vol. 45, Issue 18
Connecting slow earthquakes to huge earthquakes
journal, July 2016
- Obara, Kazushige; Kato, Aitaro
- Science, Vol. 353, Issue 6296
Earthquake Catalog‐Based Machine Learning Identification of Laboratory Fault States and the Effects of Magnitude of Completeness
journal, December 2018
- Lubbers, Nicholas; Bolton, David C.; Mohd‐Yusof, Jamaludin
- Geophysical Research Letters, Vol. 45, Issue 24
Foreshocks and Mainshock Nucleation of the 1999 M w 7.1 Hector Mine, California, Earthquake
journal, February 2019
- Yoon, Clara E.; Yoshimitsu, Nana; Ellsworth, William L.
- Journal of Geophysical Research: Solid Earth, Vol. 124, Issue 2
Episodic Tremor and Slip on the Cascadia Subduction Zone: The Chatter of Silent Slip
journal, June 2003
- Rogers, G.
- Science, Vol. 300, Issue 5627
Machine learning for graph-based representations of three-dimensional discrete fracture networks
journal, January 2018
- Valera, Manuel; Guo, Zhengyang; Kelly, Priscilla
- Computational Geosciences, Vol. 22, Issue 3
Seismic velocity precursors to the 2016 Mw 6.5 Norcia (Italy) earthquake
journal, June 2020
- Chiarabba, C.; De Gori, P.; Segou, M.
- Geology, Vol. 48, Issue 9
Evolution of b-value during the seismic cycle: Insights from laboratory experiments on simulated faults
journal, January 2018
- Rivière, J.; Lv, Z.; Johnson, P. A.
- Earth and Planetary Science Letters, Vol. 482
Precursor-Like Anomalies prior to the 2008 Wenchuan Earthquake: A Critical-but-Constructive Review
journal, January 2012
- Ma, Tengfei; Wu, Zhongliang
- International Journal of Geophysics, Vol. 2012
Constraining self-organizing map facies analysis with stratigraphy: An approach to increase the credibility in automatic seismic facies classification
journal, May 2017
- Zhao, Tao; Li, Fangyu; Marfurt, Kurt J.
- Interpretation, Vol. 5, Issue 2
Nonvolcanic Deep Tremor Associated with Subduction in Southwest Japan
journal, May 2002
- Obara, K.
- Science, Vol. 296, Issue 5573
Machine learning for data-driven discovery in solid Earth geoscience
journal, March 2019
- Bergen, Karianne J.; Johnson, Paul A.; de Hoop, Maarten V.
- Science, Vol. 363, Issue 6433
Numerical simulation of earthquake sequences
journal, October 1977
- Rundle, John B.; Jackson, David D.
- Bulletin of the Seismological Society of America, Vol. 67, Issue 5
Towards Global Volcano Monitoring Using Multisensor Sentinel Missions and Artificial Intelligence: The MOUNTS Monitoring System
journal, June 2019
- Valade, Sébastien; Ley, Andreas; Massimetti, Francesco
- Remote Sensing, Vol. 11, Issue 13
Precursors to the shear failure of rock discontinuities: Precursors to Rock Frictional Sliding
journal, August 2014
- Hedayat, Ahmadreza; Pyrak-Nolte, Laura J.; Bobet, Antonio
- Geophysical Research Letters, Vol. 41, Issue 15
Frictional Mechanics of Slow Earthquakes
journal, September 2018
- Leeman, J. R.; Marone, C.; Saffer, D. M.
- Journal of Geophysical Research: Solid Earth, Vol. 123, Issue 9
Effect of humidity on time- and velocity-dependent friction in rocks
journal, June 1984
- Dieterich, James H.; Conrad, Gerald
- Journal of Geophysical Research: Solid Earth, Vol. 89, Issue B6
Review of the Physical Basis of Laboratory-derived Relations for Brittle Failure and their Implications for Earthquake Occurrence and Earthquake Nucleation
journal, August 2004
- Beeler, N. M.
- Pure and Applied Geophysics, Vol. 161, Issue 9-10
Micromechanics of asperity rupture during laboratory stick slip experiments: ASPERITY RUPTURE MICROMECHANICS
journal, June 2011
- McLaskey, Gregory C.; Glaser, Steven D.
- Geophysical Research Letters, Vol. 38, Issue 12
Earthquake nucleation on (aging) rate and state faults: RATE AND STATE EARTHQUAKE NUCLEATION
journal, November 2005
- Rubin, A. M.; Ampuero, J. -P.
- Journal of Geophysical Research: Solid Earth, Vol. 110, Issue B11
Studies on earthquake precursors in China: A review for recent 50 years
journal, January 2017
- Huang, Fuqiong; Li, Mei; Ma, Yuchuan
- Geodesy and Geodynamics, Vol. 8, Issue 1
Multiple slow-slip events during a foreshock sequence of the 2014 Iquique, Chile M w 8.1 earthquake
journal, August 2014
- Kato, Aitaro; Nakagawa, Shigeki
- Geophysical Research Letters, Vol. 41, Issue 15
Neural Network Applications in Earthquake Prediction (1994–2019): Meta-Analytic and Statistical Insights on Their Limitations
journal, May 2020
- Mignan, Arnaud; Broccardo, Marco
- Seismological Research Letters, Vol. 91, Issue 4
Generalized Seismic Phase Detection with Deep Learning
journal, August 2018
- Ross, Zachary E.; Meier, Men‐Andrin; Hauksson, Egill
- Bulletin of the Seismological Society of America, Vol. 108, Issue 5A
Triggering of the 2014 Mw7.3 Papanoa earthquake by a slow slip event in Guerrero, Mexico
journal, October 2016
- Radiguet, M.; Perfettini, H.; Cotte, N.
- Nature Geoscience, Vol. 9, Issue 11
Seismic Evidence for an Earthquake Nucleation Phase
journal, May 1995
- Ellsworth, W. L.; Beroza, G. C.
- Science, Vol. 268, Issue 5212
Reducing process delays for real-time earthquake parameter estimation – An application of KD tree to large databases for Earthquake Early Warning
journal, May 2018
- Yin, Lucy; Andrews, Jennifer; Heaton, Thomas
- Computers & Geosciences, Vol. 114
Episodic slow slip events and rate-and-state friction
journal, January 2008
- Rubin, Allan M.
- Journal of Geophysical Research, Vol. 113, Issue B11
Laboratory-Derived Friction laws and Their Application to Seismic Faulting
journal, May 1998
- Marone, Chris
- Annual Review of Earth and Planetary Sciences, Vol. 26, Issue 1
Do slow slip events trigger large and great megathrust earthquakes?
journal, October 2018
- Voss, N.; Dixon, T. H.; Liu, Z.
- Science Advances, Vol. 4, Issue 10
Preseismic Fault Creep and Elastic Wave Amplitude Precursors Scale With Lab Earthquake Magnitude for the Continuum of Tectonic Failure Modes
journal, April 2020
- Shreedharan, Srisharan; Bolton, David Chas; Rivière, Jacques
- Geophysical Research Letters, Vol. 47, Issue 8
PhaseNet: A Deep-Neural-Network-Based Seismic Arrival Time Picking Method
journal, October 2018
- Zhu, Weiqiang; Beroza, Gregory C.
- Geophysical Journal International
Geological mapping using remote sensing data: A comparison of five machine learning algorithms, their response to variations in the spatial distribution of training data and the use of explicit spatial information
journal, February 2014
- Cracknell, Matthew J.; Reading, Anya M.
- Computers & Geosciences, Vol. 63
Slow Earthquakes and Nonvolcanic Tremor
journal, May 2011
- Beroza, Gregory C.; Ide, Satoshi
- Annual Review of Earth and Planetary Sciences, Vol. 39, Issue 1
Modeling flow and transport in fracture networks using graphs
journal, March 2018
- Karra, S.; O'Malley, D.; Hyman, J. D.
- Physical Review E, Vol. 97, Issue 3
Precursory changes in seismic velocity for the spectrum of earthquake failure modes
journal, August 2016
- Scuderi, M. M.; Marone, C.; Tinti, E.
- Nature Geoscience, Vol. 9, Issue 9