skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on February 26, 2021

Title: Technical Note: Deep Learning for Creating Surrogate Models of Precipitation in Earth System Models

Abstract

We investigate techniques for using deep neural networks to produce surrogate models for short term climate forecasts. A convolutional neural network is trained on 97 years of monthly precipitation output from the 1pctCO2 run (the CO2 concentration increases by 1?% per year) simulated by the CanESM2 Earth System Model. The neural network clearly outperforms a persistence forecast and does not show substantially degraded performance even when the forecast length is extended to 120 months. The model is prone to underpredicting precipitation in areas characterized by intense precipitation events. Scheduled sampling (forcing the model to gradually use its own past predictions rather than ground truth) is essential for avoiding amplification of early forecasting errors. However, the use of scheduled sampling also necessitates preforecasting (generating forecasts prior to the first forecast date) to obtain adequate performance for the first few prediction time steps. We document the training procedures and hyperparameter optimization process for researchers who wish to extend the use of neural networks in developing surrogate models.

Authors:
 [1];  [1];  [2]; ORCiD logo [3]; ORCiD logo [3]
  1. Western Washington University
  2. WESTERN WASHINGTON UNIVERSITY
  3. BATTELLE (PACIFIC NW LAB)
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1604800
Report Number(s):
[PNNL-SA-141681]
Grant/Contract Number:  
[AC05-76RL01830]
Resource Type:
Accepted Manuscript
Journal Name:
Atmospheric Chemistry and Physics
Additional Journal Information:
[ Journal Volume: 20; Journal Issue: 4]
Country of Publication:
United States
Language:
English

Citation Formats

Weber, Theodore, Corotan, Austin, Hutchinson, Brian J., Kravitz, Benjamin S., and Link, Robert P. Technical Note: Deep Learning for Creating Surrogate Models of Precipitation in Earth System Models. United States: N. p., 2020. Web. doi:10.5194/acp-20-2303-2020.
Weber, Theodore, Corotan, Austin, Hutchinson, Brian J., Kravitz, Benjamin S., & Link, Robert P. Technical Note: Deep Learning for Creating Surrogate Models of Precipitation in Earth System Models. United States. doi:10.5194/acp-20-2303-2020.
Weber, Theodore, Corotan, Austin, Hutchinson, Brian J., Kravitz, Benjamin S., and Link, Robert P. Wed . "Technical Note: Deep Learning for Creating Surrogate Models of Precipitation in Earth System Models". United States. doi:10.5194/acp-20-2303-2020.
@article{osti_1604800,
title = {Technical Note: Deep Learning for Creating Surrogate Models of Precipitation in Earth System Models},
author = {Weber, Theodore and Corotan, Austin and Hutchinson, Brian J. and Kravitz, Benjamin S. and Link, Robert P.},
abstractNote = {We investigate techniques for using deep neural networks to produce surrogate models for short term climate forecasts. A convolutional neural network is trained on 97 years of monthly precipitation output from the 1pctCO2 run (the CO2 concentration increases by 1?% per year) simulated by the CanESM2 Earth System Model. The neural network clearly outperforms a persistence forecast and does not show substantially degraded performance even when the forecast length is extended to 120 months. The model is prone to underpredicting precipitation in areas characterized by intense precipitation events. Scheduled sampling (forcing the model to gradually use its own past predictions rather than ground truth) is essential for avoiding amplification of early forecasting errors. However, the use of scheduled sampling also necessitates preforecasting (generating forecasts prior to the first forecast date) to obtain adequate performance for the first few prediction time steps. We document the training procedures and hyperparameter optimization process for researchers who wish to extend the use of neural networks in developing surrogate models.},
doi = {10.5194/acp-20-2303-2020},
journal = {Atmospheric Chemistry and Physics},
number = [4],
volume = [20],
place = {United States},
year = {2020},
month = {2}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on February 26, 2021
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Deep echo state networks with uncertainty quantification for spatio‐temporal forecasting
journal, November 2018

  • McDermott, Patrick L.; Wikle, Christopher K.
  • Environmetrics, Vol. 30, Issue 3
  • DOI: 10.1002/env.2553

Monthly Rainfall Forecasting Using Echo State Networks Coupled with Data Preprocessing Methods
journal, October 2017


Detection of transverse cirrus bands in satellite imagery using deep learning
journal, September 2018


ENSO Atmospheric Teleconnections and Their Response to Greenhouse Gas Forcing
journal, February 2018

  • Yeh, Sang-Wook; Cai, Wenju; Min, Seung-Ki
  • Reviews of Geophysics, Vol. 56, Issue 1
  • DOI: 10.1002/2017RG000568

Statistical Emulation of Climate Model Projections Based on Precomputed GCM Runs
journal, March 2014

  • Castruccio, Stefano; McInerney, David J.; Stein, Michael L.
  • Journal of Climate, Vol. 27, Issue 5
  • DOI: 10.1175/JCLI-D-13-00099.1

Listen, attend and spell: A neural network for large vocabulary conversational speech recognition
conference, March 2016

  • Chan, William; Jaitly, Navdeep; Le, Quoc
  • 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • DOI: 10.1109/ICASSP.2016.7472621

Dynamic climate emulators for solar geoengineering
journal, January 2016


An open-access CMIP5 pattern library for temperature and precipitation: description and methodology
journal, January 2017

  • Lynch, Cary; Hartin, Corinne; Bond-Lamberty, Ben
  • Earth System Science Data, Vol. 9, Issue 1
  • DOI: 10.5194/essd-9-281-2017

Predicting Near-Term Changes in the Earth System: A Large Ensemble of Initialized Decadal Prediction Simulations Using the Community Earth System Model
journal, September 2018

  • Yeager, S. G.; Danabasoglu, G.; Rosenbloom, N. A.
  • Bulletin of the American Meteorological Society, Vol. 99, Issue 9
  • DOI: 10.1175/BAMS-D-17-0098.1

Efficient surrogate modeling methods for large-scale Earth system models based on machine-learning techniques
journal, January 2019


Going deeper with convolutions
conference, June 2015


Differences in potential and actual skill in a decadal prediction experiment
journal, December 2018


Systematic Estimates of Initial-Value Decadal Predictability for Six AOGCMs
journal, March 2012


On Early Stopping in Gradient Descent Learning
journal, April 2007

  • Yao, Yuan; Rosasco, Lorenzo; Caponnetto, Andrea
  • Constructive Approximation, Vol. 26, Issue 2
  • DOI: 10.1007/s00365-006-0663-2

Advancements in decadal climate predictability: The role of nonoceanic drivers
journal, April 2015

  • Bellucci, A.; Haarsma, R.; Bellouin, N.
  • Reviews of Geophysics, Vol. 53, Issue 2
  • DOI: 10.1002/2014RG000473

Carbon emission limits required to satisfy future representative concentration pathways of greenhouse gases: ALLOWABLE FUTURE CARBON EMISSIONS
journal, March 2011

  • Arora, V. K.; Scinocca, J. F.; Boer, G. J.
  • Geophysical Research Letters, Vol. 38, Issue 5
  • DOI: 10.1029/2010GL046270

Cassini detection of Enceladus' cold water-group plume ionosphere
journal, January 2009

  • Tokar, R. L.; Johnson, R. E.; Thomsen, M. F.
  • Geophysical Research Letters, Vol. 36, Issue 13
  • DOI: 10.1029/2009GL038923

Retrospective prediction of the global warming slowdown in the past decade
journal, April 2013

  • Guemas, Virginie; Doblas-Reyes, Francisco J.; Andreu-Burillo, Isabel
  • Nature Climate Change, Vol. 3, Issue 7
  • DOI: 10.1038/nclimate1863

The representative concentration pathways: an overview
journal, August 2011


Deep Residual Learning for Image Recognition
conference, June 2016

  • He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2016.90

Improving and Promoting Subseasonal to Seasonal Prediction
journal, March 2015

  • Robertson, Andrew W.; Kumar, Arun; Peña, Malaquias
  • Bulletin of the American Meteorological Society, Vol. 96, Issue 3
  • DOI: 10.1175/BAMS-D-14-00139.1

Deep learning to represent subgrid processes in climate models
journal, September 2018

  • Rasp, Stephan; Pritchard, Michael S.; Gentine, Pierre
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 39
  • DOI: 10.1073/pnas.1810286115

A verification framework for interannual-to-decadal predictions experiments
journal, August 2012


Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
conference, December 2015

  • He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing
  • 2015 IEEE International Conference on Computer Vision (ICCV)
  • DOI: 10.1109/ICCV.2015.123

Gradient-based learning applied to document recognition
journal, January 1998

  • Lecun, Y.; Bottou, L.; Bengio, Y.
  • Proceedings of the IEEE, Vol. 86, Issue 11
  • DOI: 10.1109/5.726791

Tropical Cyclone Intensity Estimation Using a Deep Convolutional Neural Network
journal, February 2018

  • Pradhan, Ritesh; Aygun, Ramazan S.; Maskey, Manil
  • IEEE Transactions on Image Processing, Vol. 27, Issue 2
  • DOI: 10.1109/TIP.2017.2766358

A Deep Learning Algorithm of Neural Network for the Parameterization of Typhoon-Ocean Feedback in Typhoon Forecast Models
journal, April 2018

  • Jiang, Guo-Qing; Xu, Jing; Wei, Jun
  • Geophysical Research Letters, Vol. 45, Issue 8
  • DOI: 10.1002/2018GL077004

Large-Scale Video Classification with Convolutional Neural Networks
conference, June 2014

  • Karpathy, Andrej; Toderici, George; Shetty, Sanketh
  • 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2014.223

On climate prediction: how much can we expect from climate memory?
journal, March 2018


Recent Arctic amplification and extreme mid-latitude weather
journal, August 2014

  • Cohen, Judah; Screen, James A.; Furtado, Jason C.
  • Nature Geoscience, Vol. 7, Issue 9
  • DOI: 10.1038/ngeo2234

Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
journal, November 2012


Deep Learning in Drug Discovery
journal, December 2015

  • Gawehn, Erik; Hiss, Jan A.; Schneider, Gisbert
  • Molecular Informatics, Vol. 35, Issue 1
  • DOI: 10.1002/minf.201501008

Making sense of the early-2000s warming slowdown
journal, February 2016

  • Fyfe, John C.; Meehl, Gerald A.; England, Matthew H.
  • Nature Climate Change, Vol. 6, Issue 3
  • DOI: 10.1038/nclimate2938