Machine learning assisted hybrid models can improve streamflow simulation in diverse catchments across the conterminous US

Konapala, Goutam; Kao, Shih-Chieh; Painter, Scott; Lu, Dan

doi:10.1088/1748-9326/aba927

Machine learning assisted hybrid models can improve streamflow simulation in diverse catchments across the conterminous US

Journal Article · Fri Jul 24 00:00:00 EDT 2020 · Environmental Research Letters

DOI:https://doi.org/10.1088/1748-9326/aba927· OSTI ID:1651326

Konapala, Goutam ^[1]; ^[1]; ^[1]; ^[1]

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Climate Change Science Inst.

Incomplete representations of physical processes often lead to structural errors in process-based (PB) hydrologic models. Machine learning (ML) algorithms can reduce streamflow modeling errors but do not enforce physical consistency. As a result, ML algorithms may be unreliable if used to provide future hydroclimate projections where climates and land use patterns are outside the range of training data. Here we test hybrid models built by integrating PB model outputs with a ML algorithm known as Long Short-Term Memory (LSTM) network on their ability to simulate streamflow in 531 catchments representing diverse conditions across the Conterminous United States. Model performance of hybrid models as measured by Nash-Sutcliffe efficiency (NSE) improved relative to standalone PB and LSTM models. More importantly, hybrid models provide highest improvement in catchments where PB models fail completely (i.e., NSE < 0). However, all models performed poorly in catchments with extended low flow periods, suggesting need for additional research.

View Accepted Manuscript (DOE)

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 1651326

Journal Information:: Environmental Research Letters, Journal Name: Environmental Research Letters Journal Issue: 10 Vol. 15; ISSN 1748-9326

Publisher:: IOP PublishingCopyright Statement

Country of Publication:: United States

Language:: English

References (73)

Daymet: Daily Surface Weather Data on a 1-km Grid for North America, Version 3 Thornton, P. E.; Thornton, M. M.; Mayer, B. W. ORNL Distributed Active Archive Center https://doi.org/10.3334/ornldaac/1328	collection	January 2017
A large-sample watershed-scale hydrometeorological dataset for the contiguous USA Newman, Andrew; Sampson, Kevin; Clark, Martyn UCAR/NCAR https://doi.org/10.5065/d6mw2f4d	dataset	January 2014
Model-based analysis of the influence of catchment properties on hydrologic partitioning across five mountain headwater subcatchments: MODEL ANALYSIS FOR FIVE HEADWATER SUBCATCHMENTS Kelleher, Christa; Wagener, Thorsten; McGlynn, Brian Water Resources Research, Vol. 51, Issue 6 https://doi.org/10.1002/2014WR016147	journal	June 2015
Predicting the resilience and recovery of aquatic systems: A framework for model evolution within environmental observatories: PREDICTING THE RESILIENCE OF AQUATIC SYSTEMS Hipsey, Matthew R.; Hamilton, David P.; Hanson, Paul C. Water Resources Research, Vol. 51, Issue 9 https://doi.org/10.1002/2015WR017175	journal	September 2015
A unified approach for process‐based hydrologic modeling: 1. Modeling concept Clark, Martyn P.; Nijssen, Bart; Lundquist, Jessica D. Water Resources Research, Vol. 51, Issue 4 https://doi.org/10.1002/2015WR017198	journal	April 2015
Towards seamless large-domain parameter estimation for hydrologic models: LARGE-DOMAIN MODEL PARAMETERS Mizukami, Naoki; Clark, Martyn P.; Newman, Andrew J. Water Resources Research, Vol. 53, Issue 9 https://doi.org/10.1002/2017WR020401	journal	September 2017
Prediction in ungauged basins: a grand challenge for theoretical hydrology Sivapalan, Murugesu Hydrological Processes, Vol. 17, Issue 15 https://doi.org/10.1002/hyp.5155	journal	January 2003
Reconciling theory with observations: elements of a diagnostic approach to model evaluation Gupta, Hoshin V.; Wagener, Thorsten; Liu, Yuqiong Hydrological Processes, Vol. 22, Issue 18 https://doi.org/10.1002/hyp.6989	journal	August 2008
Model identification for hydrological forecasting under uncertainty Wagener, Thorsten; Gupta, Hoshin V. Stochastic Environmental Research and Risk Assessment, Vol. 19, Issue 6 https://doi.org/10.1007/s00477-005-0006-5	journal	September 2005
Inferring groundwater contributions and pathways to streamflow during snowmelt over multiple years in a discontinuous permafrost subarctic environment (Yukon, Canada) Carey, Sean K.; Boucher, Jessica L.; Duarte, Celina M. Hydrogeology Journal, Vol. 21, Issue 1 https://doi.org/10.1007/s10040-012-0920-9	journal	November 2012
Deep groundwater mediates streamflow response to climate warming in the Oregon Cascades Tague, Christina; Grant, Gordon; Farrell, Mike Climatic Change, Vol. 86, Issue 1-2 https://doi.org/10.1007/s10584-007-9294-8	journal	November 2007
River flow forecasting through conceptual models part I — A discussion of principles Nash, J. E.; Sutcliffe, J. V. Journal of Hydrology, Vol. 10, Issue 3 https://doi.org/10.1016/0022-1694(70)90255-6	journal	April 1970
The rise of deep learning in drug discovery Chen, Hongming; Engkvist, Ola; Wang, Yinhai Drug Discovery Today, Vol. 23, Issue 6 https://doi.org/10.1016/J.DRUDIS.2018.01.039	journal	June 2018
Rainfall-runoff modeling using conceptual, data driven, and wavelet based computing approach Nayak, P. C.; Venkatesh, B.; Krishna, B. Journal of Hydrology, Vol. 493 https://doi.org/10.1016/J.JHYDROL.2013.04.016	journal	June 2013
An overview of current applications, challenges, and future trends in distributed process-based models in hydrology Fatichi, Simone; Vivoni, Enrique R.; Ogden, Fred L. Journal of Hydrology, Vol. 537 https://doi.org/10.1016/J.JHYDROL.2016.03.026	journal	June 2016
Low flow hydrology: a review Smakhtin, V. U. Journal of Hydrology, Vol. 240, Issue 3-4 https://doi.org/10.1016/S0022-1694(00)00340-1	journal	January 2001
A physically based and machine learning hybrid approach for accurate rainfall-runoff modeling during extreme typhoon events Young, Chih-Chieh; Liu, Wen-Cheng; Wu, Ming-Chang Applied Soft Computing, Vol. 53 https://doi.org/10.1016/j.asoc.2016.12.052	journal	April 2017
Simulation of river stage using artificial neural network and MIKE 11 hydrodynamic model Panda, Rabindra K.; Pramanik, Niranjan; Bala, Biplab Computers & Geosciences, Vol. 36, Issue 6 https://doi.org/10.1016/j.cageo.2009.07.012	journal	June 2010
Accounting for surface–groundwater interactions and their uncertainty in river and groundwater models: A case study in the Namoi River, Australia Rassam, David W.; Peeters, Luk; Pickett, Trevor Environmental Modelling & Software, Vol. 50 https://doi.org/10.1016/j.envsoft.2013.09.004	journal	December 2013
Integrated approach to model decomposed flow hydrograph using artificial neural network and conceptual techniques Jain, Ashu; Srinivasulu, Sanaga Journal of Hydrology, Vol. 317, Issue 3-4 https://doi.org/10.1016/j.jhydrol.2005.05.022	journal	February 2006
Integration of artificial neural networks with conceptual models in rainfall-runoff modeling Chen, Jieyun; Adams, Barry J. Journal of Hydrology, Vol. 318, Issue 1-4 https://doi.org/10.1016/j.jhydrol.2005.06.017	journal	March 2006
Intercomparison of lumped versus distributed hydrologic model ensemble simulations on operational forecast scales Carpenter, Theresa M.; Georgakakos, Konstantine P. Journal of Hydrology, Vol. 329, Issue 1-2 https://doi.org/10.1016/j.jhydrol.2006.02.013	journal	September 2006
Comparison of four regionalisation methods for a distributed hydrological model Götzinger, J.; Bárdossy, A. Journal of Hydrology, Vol. 333, Issue 2-4 https://doi.org/10.1016/j.jhydrol.2006.09.008	journal	February 2007
Examining geological controls on baseflow index (BFI) using regression analysis: An illustration from the Thames Basin, UK Bloomfield, J. P.; Allen, D. J.; Griffiths, K. J. Journal of Hydrology, Vol. 373, Issue 1-2 https://doi.org/10.1016/j.jhydrol.2009.04.025	journal	June 2009
Implementation of surface soil moisture data assimilation with watershed scale distributed hydrological model Han, Eunjin; Merwade, Venkatesh; Heathman, Gary C. Journal of Hydrology, Vol. 416-417 https://doi.org/10.1016/j.jhydrol.2011.11.039	journal	January 2012
A hybrid approach to monthly streamflow forecasting: Integrating hydrological model outputs into a Bayesian artificial neural network Humphrey, Greer B.; Gibbs, Matthew S.; Dandy, Graeme C. Journal of Hydrology, Vol. 540 https://doi.org/10.1016/j.jhydrol.2016.06.026	journal	September 2016
Modeling and simulating of reservoir operation using the artificial neural network, support vector regression, deep learning algorithm Zhang, Di; Lin, Junqiang; Peng, Qidong Journal of Hydrology, Vol. 565 https://doi.org/10.1016/j.jhydrol.2018.08.050	journal	October 2018
Random Forests Breiman, Leo Machine Learning, Vol. 45, Issue 1, p. 5-32 https://doi.org/10.1023/A:1010933404324	journal	January 2001
A process-based rejectionist framework for evaluating catchment runoff model structure: PROCESS-BASED MODEL REJECTION Vaché, Kellie B.; McDonnell, Jeffrey J. Water Resources Research, Vol. 42, Issue 2 https://doi.org/10.1029/2005WR004247	journal	February 2006
On the value of soil moisture measurements in vadose zone hydrology: A review: SOIL MOISTURE AND HYDROLOGY Vereecken, H.; Huisman, J. A.; Bogena, H. Water Resources Research, Vol. 44, Issue 4 https://doi.org/10.1029/2008WR006829	journal	April 2008
Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors: IDENTIFIABILITY OF INPUT AND STRUCTURAL ERRORS Renard, Benjamin; Kavetski, Dmitri; Kuczera, George Water Resources Research, Vol. 46, Issue 5 https://doi.org/10.1029/2009WR008328	journal	May 2010
Are seemingly physically similar catchments truly hydrologically similar?: ARE PHYSICALLY SIMILAR CATCHMENTS HYDROLOGICALLY SIMILAR? Oudin, Ludovic; Kay, Alison; Andréassian, Vazken Water Resources Research, Vol. 46, Issue 11 https://doi.org/10.1029/2009WR008887	journal	November 2010
Effect of glaciers on streamflow trends in the Swiss Alps Pellicciotti, F.; Bauder, A.; Parola, M. Water Resources Research, Vol. 46, Issue 10 https://doi.org/10.1029/2009WR009039	journal	October 2010
Assessment of hydrological model predictive ability given multiple conceptual geological models: CONCEPTUAL GEOLOGICAL MODEL UNCERTAINTY Seifert, Dorte; Sonnenborg, Torben O.; Refsgaard, Jens Christian Water Resources Research, Vol. 48, Issue 6 https://doi.org/10.1029/2011WR011149	journal	June 2012
Hydrologic consistency as a basis for assessing complexity of monthly water balance models for the continental United States: HYDROLOGIC CONSISTENCY AND MODEL COMPLEXITY Martinez, Guillermo F.; Gupta, Hoshin V. Water Resources Research, Vol. 47, Issue 12 https://doi.org/10.1029/2011WR011229	journal	December 2011
Crash testing hydrological models in contrasted climate conditions: An experiment on 216 Australian catchments: TESTING HYDROLOGICAL MODELS IN CONTRASTED CLIMATE Coron, L.; Andréassian, V.; Perrin, C. Water Resources Research, Vol. 48, Issue 5 https://doi.org/10.1029/2011WR011721	journal	May 2012
Could Machine Learning Break the Convection Parameterization Deadlock? Gentine, P.; Pritchard, M.; Rasp, S. Geophysical Research Letters, Vol. 45, Issue 11 https://doi.org/10.1029/2018GL078202	journal	June 2018
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists Shen, Chaopeng Water Resources Research, Vol. 54, Issue 11 https://doi.org/10.1029/2018WR022643	journal	November 2018
Combining Physically Based Modeling and Deep Learning for Fusing GRACE Satellite Data: Can We Learn From Mismatch? Sun, Alexander Y.; Scanlon, Bridget R.; Zhang, Zizhan Water Resources Research, Vol. 55, Issue 2 https://doi.org/10.1029/2018WR023333	journal	February 2019
Downscaling SMAP Radiometer Soil Moisture Over the CONUS Using an Ensemble Learning Method Abbaszadeh, Peyman; Moradkhani, Hamid; Zhan, Xiwu Water Resources Research, Vol. 55, Issue 1 https://doi.org/10.1029/2018WR023354	journal	January 2019
Gradient‐Based Inverse Estimation for a Rainfall‐Runoff Model Krapu, Christopher; Borsuk, Mark; Kumar, Mukesh Water Resources Research, Vol. 55, Issue 8 https://doi.org/10.1029/2018WR024461	journal	August 2019
Quantifying Climate and Catchment Control on Hydrological Drought in the Continental United States Konapala, Goutam; Mishra, Ashok Water Resources Research, Vol. 56, Issue 1 https://doi.org/10.1029/2018WR024620	journal	January 2020
AquaSat: A Data Set to Enable Remote Sensing of Water Quality for Inland Waters Ross, Matthew R. V.; Topp, Simon N.; Appling, Alison P. Water Resources Research, Vol. 55, Issue 11 https://doi.org/10.1029/2019WR024883	journal	November 2019
Identifying Subsurface Drainage using Satellite Big Data and Machine Learning via Google Earth Engine Cho, Eunsang; Jacobs, Jennifer M.; Jia, Xinhua Water Resources Research, Vol. 55, Issue 10 https://doi.org/10.1029/2019WR024892	journal	October 2019
Process‐Guided Deep Learning Predictions of Lake Water Temperature Read, Jordan S.; Jia, Xiaowei; Willard, Jared Water Resources Research, Vol. 55, Issue 11 https://doi.org/10.1029/2019WR024922	journal	November 2019
A Rainfall‐Runoff Model With LSTM‐Based Sequence‐to‐Sequence Learning Xiang, Zhongrun; Yan, Jun; Demir, Ibrahim Water Resources Research, Vol. 56, Issue 1 https://doi.org/10.1029/2019WR025326	journal	January 2020
Using Information Flow for Whole System Understanding From Component Dynamics Jiang, Peishi; Kumar, Praveen Water Resources Research, Vol. 55, Issue 11 https://doi.org/10.1029/2019WR025820	journal	November 2019
Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Global patterns of drought recovery Schwalm, Christopher R.; Anderegg, William R. L.; Michalak, Anna M. Nature, Vol. 548, Issue 7666 https://doi.org/10.1038/nature23021	journal	August 2017
Flow‐Duration Curves. I: New Interpretation and Confidence Intervals Vogel, Richard M.; Fennessey, Neil M. Journal of Water Resources Planning and Management, Vol. 120, Issue 4 https://doi.org/10.1061/(ASCE)0733-9496(1994)120:4(485)	journal	July 1994
Performance Enhancement of a Conceptual Hydrological Model by Integrating Artificial Intelligence Kumanlioglu, Ahmet Ali; Fistikoglu, Okan Journal of Hydrologic Engineering, Vol. 24, Issue 11 https://doi.org/10.1061/(ASCE)HE.1943-5584.0001850	journal	November 2019
Definitions, methods, and applications in interpretable machine learning Murdoch, W. James; Singh, Chandan; Kumbier, Karl Proceedings of the National Academy of Sciences, Vol. 116, Issue 44 https://doi.org/10.1073/pnas.1900654116	journal	October 2019
Hybrid modelling approach to prairie hydrology: fusing data-driven and process-based hydrological models Mekonnen, Balew A.; Nazemi, Alireza; Mazurek, Kerry A. Hydrological Sciences Journal, Vol. 60, Issue 9 https://doi.org/10.1080/02626667.2014.935778	journal	June 2015
Forecasting monthly precipitation using sequential modelling Kumar, Deepak; Singh, Anshuman; Samui, Pijush Hydrological Sciences Journal, Vol. 64, Issue 6 https://doi.org/10.1080/02626667.2019.1595624	journal	April 2019
Operational testing of hydrological simulation models KlemeŠ, V. Hydrological Sciences Journal, Vol. 31, Issue 1 https://doi.org/10.1080/02626668609491024	journal	March 1986
The Parable of Google Flu: Traps in Big Data Analysis Lazer, D.; Kennedy, R.; King, G. Science, Vol. 343, Issue 6176 https://doi.org/10.1126/science.1248506	journal	March 2014
Learning to Forget: Continual Prediction with LSTM Gers, Felix A.; Schmidhuber, Jürgen; Cummins, Fred Neural Computation, Vol. 12, Issue 10 https://doi.org/10.1162/089976600300015015	journal	October 2000
Benchmarking of a Physically Based Hydrologic Model Newman, Andrew J.; Mizukami, Naoki; Clark, Martyn P. Journal of Hydrometeorology, Vol. 18, Issue 8 https://doi.org/10.1175/JHM-D-16-0284.1	journal	August 2017
Bias in random forest variable importance measures: Illustrations, sources and a solution Strobl, Carolin; Boulesteix, Anne-Laure; Zeileis, Achim BMC Bioinformatics, Vol. 8, Issue 1 https://doi.org/10.1186/1471-2105-8-25	journal	January 2007
Variable Importance Assessment in Regression: Linear Regression versus Random Forest Grömping, Ulrike The American Statistician, Vol. 63, Issue 4 https://doi.org/10.1198/tast.2009.08199	journal	November 2009
Deep Learning: A Bayesian Perspective Polson, Nicholas G.; Sokolov, Vadim Bayesian Analysis, Vol. 12, Issue 4 https://doi.org/10.1214/17-BA1082	journal	December 2017
Incorporating groundwater recharge and discharge functions into an existing monthly rainfall–runoff model/Incorporation de fonctions de recharge et de vidange superficielle de nappes au sein d’un modèle pluie-débit mensuel existant Hughes, D. A. Hydrological Sciences Journal, Vol. 49, Issue 2 https://doi.org/10.1623/hysj.49.2.297.34834	journal	April 2004
A comparative study of three neural network forecast combination methods for simulated river flows of different rainfall—runoff models Shamseldin, Asaad Y.; O'Connor, Kieran M.; Nasr, Ahmed E. Hydrological Sciences Journal, Vol. 52, Issue 5 https://doi.org/10.1623/hysj.52.5.896	journal	October 2007
Soil Moisture Measurement for Ecological and Hydrological Watershed-Scale Observatories: A Review Robinson, D. A.; Campbell, C. S.; Hopmans, J. W. Vadose Zone Journal, Vol. 7, Issue 1 https://doi.org/10.2136/vzj2007.0143	journal	January 2008
Integration of a Parsimonious Hydrological Model with Recurrent Neural Networks for Improved Streamflow Forecasting Tian, Ye; Xu, Yue-Ping; Yang, Zongliang Water, Vol. 10, Issue 11 https://doi.org/10.3390/w10111655	journal	November 2018
A large-sample watershed-scale hydrometeorological dataset for the contiguous USA Newman, Andrew; Sampson, Kevin; Clark, Martyn UCAR/NCAR https://doi.org/10.5065/D6MW2F4D	dataset	January 2014
Assimilating scatterometer soil moisture data into conceptual hydrologic models at the regional scale Parajka, J.; Naeimi, V.; Blöschl, G. Hydrology and Earth System Sciences, Vol. 10, Issue 3 https://doi.org/10.5194/hess-10-353-2006	journal	January 2006
Combining semi-distributed process-based and data-driven models in flow simulation: a case study of the Meuse river basin Corzo, G. A.; Solomatine, D. P.; de Wit, M. Hydrology and Earth System Sciences, Vol. 13, Issue 9 https://doi.org/10.5194/hess-13-1619-2009	journal	January 2009
The influence of soil moisture on threshold runoff generation processes in an alpine headwater catchment Penna, D.; Tromp-van Meerveld, H. J.; Gobbi, A. Hydrology and Earth System Sciences, Vol. 15, Issue 3 https://doi.org/10.5194/hess-15-689-2011	journal	January 2011
Advancing catchment hydrology to deal with predictions under change Ehret, U.; Gupta, H. V.; Sivapalan, M. Hydrology and Earth System Sciences, Vol. 18, Issue 2 https://doi.org/10.5194/hess-18-649-2014	journal	January 2014
The CAMELS data set: catchment attributes and meteorology for large-sample studies Addor, Nans; Newman, Andrew J.; Mizukami, Naoki Hydrology and Earth System Sciences, Vol. 21, Issue 10 https://doi.org/10.5194/hess-21-5293-2017	journal	January 2017
Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks Kratzert, Frederik; Klotz, Daniel; Brenner, Claire Hydrology and Earth System Sciences, Vol. 22, Issue 11 https://doi.org/10.5194/hess-22-6005-2018	journal	January 2018
Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets Kratzert, Frederik; Klotz, Daniel; Shalev, Guy Hydrology and Earth System Sciences, Vol. 23, Issue 12 https://doi.org/10.5194/hess-23-5089-2019	journal	January 2019

Similar Records

HydroEcoLSTM: A Python package with graphical user interface for hydro-ecological modeling with long short-term memory neural network

Journal Article · Fri Jan 03 19:00:00 EST 2025 · Ecological Informatics · OSTI ID:2510856

Machine Learning Assisted Reservoir Operation Model for Long–Term Water Management Simulation

Journal Article · Thu Sep 01 20:00:00 EDT 2022 · Journal of the American Water Resources Association · OSTI ID:1908076

Related Subjects

54 ENVIRONMENTAL SCIENCES
Long Short-Term Memory (LSTM) network
Process-based hydrologic models
Sacramento Soil Moisture Accounting Model (SAC)
machine learning

Machine learning assisted hybrid models can improve streamflow simulation in diverse catchments across the conterminous US

Citation Formats

References (73)

Similar Records

Related Subjects