A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Molina, Maria J.; Gagne, David John; Prein, Andreas F.

doi:10.1029/2020EA001490

Title: A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Journal Article · Mon Aug 30 00:00:00 EDT 2021 · Earth and Space Science

DOI:https://doi.org/10.1029/2020EA001490· OSTI ID:1817804

^[1];

^[1]

National Center for Atmospheric Research Boulder CO USA

Abstract This is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present‐day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high‐resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid‐levels for intense thunderstorm development when low‐level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human‐labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out‐of‐sample robustness with hyperparameter tuning in certain applications.

View Journal Article

Cite

Export

Save

Research Organization:: National Center for Atmospheric Research (NCAR), Boulder, CO (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: NSF IA 1947282; SC0022070

OSTI ID:: 1817804

Alternate ID(s):: OSTI ID: 1817805; OSTI ID: 1845079

Journal Information:: Earth and Space Science, Journal Name: Earth and Space Science Vol. 8 Journal Issue: 9; ISSN 2333-5084

Publisher:: American Geophysical Union (AGU)Copyright Statement

Country of Publication:: United States

Language:: English

References (58)

Surrogate climate-change scenarios for regional climate models Schär, Christoph; Frei, Christoph; Lüthi, Daniel Geophysical Research Letters, Vol. 23, Issue 6 https://doi.org/10.1029/96GL00265	journal	March 1996
Forecasting Different Types of Convective Weather: A Deep Learning Approach Zhou, Kanghui; Zheng, Yongguang; Li, Bo Journal of Meteorological Research, Vol. 33, Issue 5 https://doi.org/10.1007/s13351-019-8162-6	journal	October 2019
A New Vertical Diffusion Package with an Explicit Treatment of Entrainment Processes Hong, Song-You; Noh, Yign; Dudhia, Jimy Monthly Weather Review, Vol. 134, Issue 9 https://doi.org/10.1175/MWR3199.1	journal	September 2006
Moisture Attribution and Sensitivity Analysis of a Winter Tornado Outbreak Molina, Maria J.; Allen, John T.; Prein, Andreas F. Weather and Forecasting, Vol. 35, Issue 4 https://doi.org/10.1175/WAF-D-19-0240.1	journal	August 2020
Automated detection of weather fronts using a deep learning neural network Biard, James C.; Kunkel, Kenneth E. Advances in Statistical Climatology, Meteorology and Oceanography, Vol. 5, Issue 2 https://doi.org/10.5194/ascmo-5-147-2019	journal	January 2019
Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Changes in severe thunderstorm environment frequency during the 21st century caused by anthropogenically enhanced global radiative forcing Trapp, R. J.; Diffenbaugh, N. S.; Brooks, H. E. Proceedings of the National Academy of Sciences, Vol. 104, Issue 50 https://doi.org/10.1073/pnas.0705494104	journal	December 2007
The attributes diagram A geometrical framework for assessing the quality of probability forecasts Hsu, Wu-ron; Murphy, Allan H. International Journal of Forecasting, Vol. 2, Issue 3 https://doi.org/10.1016/0169-2070(86)90048-8	journal	January 1986
Regionally-stratified tornadoes: Moisture source physical reasoning and climate trends Molina, Maria J.; Allen, John T. Weather and Climate Extremes, Vol. 28 https://doi.org/10.1016/j.wace.2020.100244	journal	June 2020
Increased rainfall volume from future convective storms in the US Prein, Andreas F.; Liu, Changhai; Ikeda, Kyoko Nature Climate Change, Vol. 7, Issue 12 https://doi.org/10.1038/s41558-017-0007-7	journal	November 2017
Viewing Forced Climate Patterns Through an AI Lens Barnes, Elizabeth A.; Hurrell, James W.; Ebert‐Uphoff, Imme Geophysical Research Letters, Vol. 46, Issue 22 https://doi.org/10.1029/2019GL084944	journal	November 2019
A Study of Aerosol Impacts on Clouds and Precipitation Development in a Large Winter Cyclone Thompson, Gregory; Eidhammer, Trude Journal of the Atmospheric Sciences, Vol. 71, Issue 10 https://doi.org/10.1175/JAS-D-13-0305.1	journal	October 2014
Which Polarimetric Variables Are Important for Weather/No-Weather Discrimination? Lakshmanan, Valliappa; Karstens, Christopher; Krause, John Journal of Atmospheric and Oceanic Technology, Vol. 32, Issue 6 https://doi.org/10.1175/JTECH-D-13-00205.1	journal	June 2015
Effective Storm-Relative Helicity and Bulk Shear in Supercell Thunderstorm Environments Thompson, Richard L.; Mead, Corey M.; Edwards, Roger Weather and Forecasting, Vol. 22, Issue 1 https://doi.org/10.1175/WAF969.1	journal	February 2007
An Overview of CMIP5 and the Experiment Design Taylor, Karl E.; Stouffer, Ronald J.; Meehl, Gerald A. Bulletin of the American Meteorological Society, Vol. 93, Issue 4 https://doi.org/10.1175/BAMS-D-11-00094.1	journal	April 2012
Seasonal Variations in Severe Weather Forecast Skill in an Experimental Convection-Allowing Model Sobash, Ryan A.; Kain, John S. Weather and Forecasting, Vol. 32, Issue 5 https://doi.org/10.1175/WAF-D-17-0043.1	journal	September 2017
ImageNet Large Scale Visual Recognition Challenge Russakovsky, Olga; Deng, Jia; Su, Hao International Journal of Computer Vision, Vol. 115, Issue 3 https://doi.org/10.1007/s11263-015-0816-y	journal	April 2015
Climatological Estimates of Local Daily Tornado Probability for the United States Brooks, Harold E.; Doswell, Charles A.; Kay, Michael P. Weather and Forecasting, Vol. 18, Issue 4 https://doi.org/10.1175/1520-0434(2003)018<0626:ceoldt>2.0.co;2	journal	August 2003
A Spectral Nudging Technique for Dynamical Downscaling Purposes von Storch, Hans; Langenberg, Heike; Feser, Frauke Monthly Weather Review, Vol. 128, Issue 10 https://doi.org/10.1175/1520-0493(2000)128<3664:ASNTFD>2.0.CO;2	journal	October 2000
Making the Black Box More Transparent: Understanding the Physical Implications of Machine Learning McGovern, Amy; Lagerquist, Ryan; John Gagne, David Bulletin of the American Meteorological Society, Vol. 100, Issue 11 https://doi.org/10.1175/BAMS-D-18-0195.1	journal	November 2019
The next generation of scenarios for climate change research and assessment Moss, Richard H.; Edmonds, Jae A.; Hibbard, Kathy A. Nature, Vol. 463, Issue 7282 https://doi.org/10.1038/nature08823	journal	February 2010
Deep Learning on Three-Dimensional Multiscale Data for Next-Hour Tornado Prediction Lagerquist, Ryan; McGovern, Amy; Homeyer, Cameron R. Monthly Weather Review, Vol. 148, Issue 7 https://doi.org/10.1175/MWR-D-19-0372.1	journal	June 2020
Deep learning to represent subgrid processes in climate models Rasp, Stephan; Pritchard, Michael S.; Gentine, Pierre Proceedings of the National Academy of Sciences, Vol. 115, Issue 39 https://doi.org/10.1073/pnas.1810286115	journal	September 2018
Transient response of severe thunderstorm forcing to elevated greenhouse gas concentrations Trapp, Robert J.; Diffenbaugh, Noah S.; Gluhovsky, Alexander Geophysical Research Letters, Vol. 36, Issue 1 https://doi.org/10.1029/2008GL036203	journal	January 2009
Differing Trends in United States and European Severe Thunderstorm Environments in a Warming Climate Taszarek, Mateusz; Allen, John T.; Brooks, Harold E. Bulletin of the American Meteorological Society, Vol. 102, Issue 2 https://doi.org/10.1175/BAMS-D-20-0004.1	journal	February 2021
Short-Range Ensemble Forecasts of Precipitation Type Wandishin, Matthew S.; Baldwin, Michael E.; Mullen, Steven L. Weather and Forecasting, Vol. 20, Issue 4 https://doi.org/10.1175/WAF871.1	journal	August 2005
Interpretable Deep Learning for Spatial Analysis of Severe Hailstorms Gagne II, David John; Haupt, Sue Ellen; Nychka, Douglas W. Monthly Weather Review, Vol. 147, Issue 8 https://doi.org/10.1175/MWR-D-18-0316.1	journal	August 2019
Classifying Convective Storms Using Machine Learning Jergensen, G. Eli; McGovern, Amy; Lagerquist, Ryan Weather and Forecasting, Vol. 35, Issue 2 https://doi.org/10.1175/WAF-D-19-0170.1	journal	April 2019
The community Noah land surface model with multiparameterization options (Noah-MP): 1. Model description and evaluation with local-scale measurements Niu, Guo-Yue; Yang, Zong-Liang; Mitchell, Kenneth E. Journal of Geophysical Research, Vol. 116, Issue D12 https://doi.org/10.1029/2010JD015139	journal	January 2011
The ERA-Interim reanalysis: configuration and performance of the data assimilation system Dee, D. P.; Uppala, S. M.; Simmons, A. J. Quarterly Journal of the Royal Meteorological Society, Vol. 137, Issue 656 https://doi.org/10.1002/qj.828	journal	April 2011
Some Practical Considerations Regarding Horizontal Resolution in the First Generation of Operational Convection-Allowing NWP Kain, John S.; Weiss, Steven J.; Bright, David R. Weather and Forecasting, Vol. 23, Issue 5 https://doi.org/10.1175/WAF2007106.1	journal	October 2008
A Comparison of Neural-Network and Surrogate-Severe Probabilistic Convective Hazard Guidance Derived from a Convection-Allowing Model Sobash, Ryan A.; Romine, Glen S.; Schwartz, Craig S. Weather and Forecasting, Vol. 35, Issue 5 https://doi.org/10.1175/WAF-D-20-0036.1	journal	October 2020
The Realization of Extreme Tornadic Storm Events under Future Anthropogenic Climate Change Trapp, Robert J.; Hoogewind, Kimberly A. Journal of Climate, Vol. 29, Issue 14 https://doi.org/10.1175/JCLI-D-15-0623.1	journal	July 2016
Visualizing Multiple Measures of Forecast Quality Roebber, Paul J. Weather and Forecasting, Vol. 24, Issue 2 https://doi.org/10.1175/2008WAF2222159.1	journal	April 2009
Convective Modes for Significant Severe Thunderstorms in the Contiguous United States. Part II: Supercell and QLCS Tornado Environments Thompson, Richard L.; Smith, Bryan T.; Grams, Jeremy S. Weather and Forecasting, Vol. 27, Issue 5 https://doi.org/10.1175/WAF-D-11-00116.1	journal	April 2012
Radiative forcing by long-lived greenhouse gases: Calculations with the AER radiative transfer models Iacono, Michael J.; Delamere, Jennifer S.; Mlawer, Eli J. Journal of Geophysical Research, Vol. 113, Issue D13 https://doi.org/10.1029/2008JD009944	journal	January 2008
Severe Weather Prediction Using Storm Surrogates from an Ensemble Forecasting System Sobash, Ryan A.; Schwartz, Craig S.; Romine, Glen S. Weather and Forecasting, Vol. 31, Issue 1 https://doi.org/10.1175/WAF-D-15-0138.1	journal	February 2016
Severe thunderstorms and climate change Brooks, H. E. Atmospheric Research, Vol. 123 https://doi.org/10.1016/j.atmosres.2012.04.002	journal	April 2013
The Impact of Climate Change on Hazardous Convective Weather in the United States: Insight from High-Resolution Dynamical Downscaling Hoogewind, Kimberly A.; Baldwin, Michael E.; Trapp, Robert J. Journal of Climate, Vol. 30, Issue 24 https://doi.org/10.1175/JCLI-D-16-0885.1	journal	December 2017
Deep Learning for Spatially Explicit Prediction of Synoptic-Scale Fronts Lagerquist, Ryan; McGovern, Amy; Gagne II, David John Weather and Forecasting, Vol. 34, Issue 4 https://doi.org/10.1175/WAF-D-18-0183.1	journal	August 2019
Tornado Pathlength Forecasts from 2010 to 2011 Using Ensemble Updraft Helicity Clark, Adam J.; Gao, Jidong; Marsh, Patrick T. Weather and Forecasting, Vol. 28, Issue 2 https://doi.org/10.1175/WAF-D-12-00038.1	journal	April 2013
Continental-scale convection-permitting modeling of the current and future climate of North America Liu, Changhai; Ikeda, Kyoko; Rasmussen, Roy Climate Dynamics, Vol. 49, Issue 1-2 https://doi.org/10.1007/s00382-016-3327-9	journal	August 2016
An Observational Examination of Long-Lived Supercells. Part I: Characteristics, Evolution, and Demise Bunkers, Matthew J.; Hjelmfelt, Mark R.; Smith, Paul L. Weather and Forecasting, Vol. 21, Issue 5 https://doi.org/10.1175/WAF949.1	journal	October 2006
Probabilistic Forecast Guidance for Severe Thunderstorms Based on the Identification of Extreme Phenomena in Convection-Allowing Model Forecasts Sobash, Ryan A.; Kain, John S.; Bright, David R. Weather and Forecasting, Vol. 26, Issue 5 https://doi.org/10.1175/WAF-D-10-05046.1	journal	October 2011
SMOTE: Synthetic Minority Over-sampling Technique Chawla, N. V.; Bowyer, K. W.; Hall, L. O. Journal of Artificial Intelligence Research, Vol. 16 https://doi.org/10.1613/jair.953	journal	January 2002
Regional climate of hazardous convective weather through high-resolution dynamical downscaling Trapp, Robert J.; Robinson, Eric D.; Baldwin, Michael E. Climate Dynamics, Vol. 37, Issue 3-4 https://doi.org/10.1007/s00382-010-0826-y	journal	May 2010
Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition Fukushima, Kunihiko; Miyake, Sei Competition and Cooperation in Neural Nets https://doi.org/10.1007/978-3-642-46466-9_18	book	January 1982
A time-split nonhydrostatic atmospheric model for weather research and forecasting applications Skamarock, William C.; Klemp, Joseph B. Journal of Computational Physics, Vol. 227, Issue 7 https://doi.org/10.1016/j.jcp.2007.01.037	journal	March 2008
Deep learning for multi-year ENSO forecasts Ham, Yoo-Geun; Kim, Jeong-Hwan; Luo, Jing-Jia Nature, Vol. 573, Issue 7775 https://doi.org/10.1038/s41586-019-1559-7	journal	September 2019
Changes in the convective population and thermodynamic environments in convection-permitting regional climate simulations over the United States Rasmussen, K. L.; Prein, A. F.; Rasmussen, R. M. Climate Dynamics https://doi.org/10.1007/s00382-017-4000-7	journal	November 2017
Robust increases in severe thunderstorm environments in response to greenhouse forcing Diffenbaugh, N. S.; Scherer, M.; Trapp, R. J. Proceedings of the National Academy of Sciences, Vol. 110, Issue 41 https://doi.org/10.1073/pnas.1307758110	journal	September 2013
Downscaled estimates of late 21st century severe weather from CCSM3 Gensini, Vittorio A.; Mote, Thomas L. Climatic Change, Vol. 129, Issue 1-2 https://doi.org/10.1007/s10584-014-1320-z	journal	January 2015
Spatial trends in United States tornado frequency Gensini, Vittorio A.; Brooks, Harold E. npj Climate and Atmospheric Science, Vol. 1, Issue 1 https://doi.org/10.1038/s41612-018-0048-2	journal	October 2018
Gradient-based learning applied to document recognition Lecun, Y.; Bottou, L.; Bengio, Y. Proceedings of the IEEE, Vol. 86, Issue 11 https://doi.org/10.1109/5.726791	journal	January 1998
Recipe for Disaster: How the Dynamic Ingredients of Risk and Exposure Are Changing the Tornado Disaster Landscape Ashley, Walker S.; Strader, Stephen M. Bulletin of the American Meteorological Society, Vol. 97, Issue 5 https://doi.org/10.1175/BAMS-D-15-00150.1	journal	May 2016
Spring and Summer Midwestern Severe Weather Reports in Supercells Compared to Other Morphologies Duda, Jeffrey D.; Gallus, William A. Weather and Forecasting, Vol. 25, Issue 1 https://doi.org/10.1175/2009WAF2222338.1	journal	February 2010
The Impact of the Nocturnal Transition on the Lifetime and Evolution of Supercell Thunderstorms in the Great Plains Gropp, Matthew E.; Davenport, Casey E. Weather and Forecasting, Vol. 33, Issue 4 https://doi.org/10.1175/WAF-D-17-0150.1	journal	August 2018
An Efficient, General-Purpose Technique for Identifying Storm Cells in Geospatial Images Lakshmanan, Valliappa; Hondl, Kurt; Rabin, Robert Journal of Atmospheric and Oceanic Technology, Vol. 26, Issue 3 https://doi.org/10.1175/2008JTECHA1153.1	journal	March 2009

Similar Records

MABAL: a Novel Deep-Learning Architecture for Machine-Assisted Bone Age Labeling

Journal Article · Wed Aug 15 00:00:00 EDT 2018 · Journal of Digital Imaging (Online) · OSTI ID:1817804

Mutasa, Simukayi; Chang, Peter D.; Ruzal-Shapiro, Carrie; +1 more

An Analytic Formula for Entraining CAPE in Midlatitude Storm Environments

Journal Article · Fri Sep 01 00:00:00 EDT 2023 · Journal of the Atmospheric Sciences · OSTI ID:1817804

Peters, John M.; Chavas, Daniel R.; Su, Chun-Yian; +2 more

Deep Learning Image Segmentation for Atmospheric Rivers

Journal Article · Mon Jan 01 00:00:00 EST 2024 · Artificial Intelligence for the Earth Systems · OSTI ID:1817804

Galea, Daniel; Ma, Hsi-Yen; Wu, Wen-Ying; +1 more

Related Subjects

54 ENVIRONMENTAL SCIENCES

Title: A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Citation Formats

References (58)

Similar Records

Related Subjects