A Multiscale Deep Learning Model for Soil Moisture Integrating Satellite and In Situ Data

Liu, Jiangtao; Rahmani, Farshid; Lawson, Kathryn; Shen, Chaopeng

doi:10.1029/2021gl096847

A Multiscale Deep Learning Model for Soil Moisture Integrating Satellite and In Situ Data

Journal Article · Mon Mar 14 00:00:00 EDT 2022 · Geophysical Research Letters

DOI:https://doi.org/10.1029/2021gl096847· OSTI ID:1978523

^[1]; ^[2]; ^[2]; ^[2]

Pennsylvania State Univ., University Park, PA (United States); OSTI
Pennsylvania State Univ., University Park, PA (United States)

Deep learning (DL) models trained on hydrologic observations can perform extraordinarily well, but they can inherit deficiencies of the training data, such as limited coverage of in situ data or low resolution/accuracy of satellite data. In this work, we propose a novel multiscale DL scheme learning simultaneously from satellite and in situ data to predict 9 km daily soil moisture (5 cm depth). Based on spatial cross-validation over sites in the conterminous United States, the multiscale scheme obtained a median correlation of 0.901 and root-mean-square error of 0.034 m³/m³. It outperformed the Soil Moisture Active Passive satellite mission's 9 km product, DL models trained on in situ data alone, and land surface models. Our 9 km product showed better accuracy than previous 1 km satellite downscaling products, highlighting limited impacts of improving resolution. Not only is our product useful for planning against floods, droughts, and pests, our scheme is generically applicable to geoscientific domains with data on multiple scales, breaking the confines of individual data sets.

View Accepted Manuscript (DOE)

Research Organization:: Univ. of California, Davis, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC); National Science Foundation (NSF)

Grant/Contract Number:: SC0016605

OSTI ID:: 1978523

Alternate ID(s):: OSTI ID: 1862507

Journal Information:: Geophysical Research Letters, Journal Name: Geophysical Research Letters Journal Issue: 7 Vol. 49; ISSN 0094-8276

Publisher:: American Geophysical Union (AGU)Copyright Statement

Country of Publication:: United States

Language:: English

References (67)

NLDAS Primary Forcing Data L4 Hourly 0.125 x 0.125 degree, Version 002 Xia, Y.; Al., Et NASA Goddard Earth Sciences Data and Information Services Center https://doi.org/10.5067/6j5lhhohzhn4	dataset	January 2009
SMAP L3 Radiometer Global Daily 36 km EASE-Grid Soil Moisture, Version 7 ONeill, Peggy E.; Chan, Steven; Njoku, Eni G. NASA National Snow and Ice Data Center DAAC https://doi.org/10.5067/hh4sz2pxsp6a	dataset	January 2020
High-resolution modeling of the spatial heterogeneity of soil moisture: Applications in network design Chaney, Nathaniel W.; Roundy, Joshua K.; Herrera-Estrada, Julio E. Water Resources Research, Vol. 51, Issue 1 https://doi.org/10.1002/2013WR014964	journal	January 2015
An initial assessment of SMAP soil moisture retrievals using high-resolution model simulations and in situ observations: SMAP Comparisons Pan, Ming; Cai, Xitian; Chaney, Nathaniel W. Geophysical Research Letters, Vol. 43, Issue 18 https://doi.org/10.1002/2016gl069964	journal	September 2016
Joint Sentinel‐1 and SMAP data assimilation to improve soil moisture estimates Lievens, H.; Reichle, R. H.; Liu, Q. Geophysical Research Letters, Vol. 44, Issue 12 https://doi.org/10.1002/2017gl073904	journal	June 2017
Prolongation of SMAP to Spatiotemporally Seamless Coverage of Continental U.S. Using a Deep Learning Neural Network Fang, Kuai; Shen, Chaopeng; Kifer, Daniel Geophysical Research Letters, Vol. 44, Issue 21 https://doi.org/10.1002/2017gl075619	journal	November 2017
Deep learning approaches for improving prediction of daily stream temperature in data‐scarce, unmonitored, and dammed basins Rahmani, Farshid; Shen, Chaopeng; Oliver, Samantha Hydrological Processes, Vol. 35, Issue 11 https://doi.org/10.1002/hyp.14400	journal	November 2021
Connecting Satellite Observations with Water Cycle Variables Through Land Data Assimilation: Examples Using the NASA GEOS-5 LDAS Reichle, Rolf H.; De Lannoy, Gabriëlle J. M.; Forman, Barton A. Surveys in Geophysics, Vol. 35, Issue 3 https://doi.org/10.1007/s10712-013-9220-8	journal	February 2013
Downscaling of AMSR-E soil moisture with MODIS products using machine learning approaches Im, Jungho; Park, Seonyoung; Rhee, Jinyoung Environmental Earth Sciences, Vol. 75, Issue 15 https://doi.org/10.1007/s12665-016-5917-6	journal	July 2016
Data assimilation methods in the Earth sciences Reichle, Rolf H. Advances in Water Resources, Vol. 31, Issue 11 https://doi.org/10.1016/j.advwatres.2008.01.001	journal	November 2008
Temporal evolution of soil moisture statistical fractal and controls by soil texture and regional groundwater flow Ji, Xinye; Shen, Chaopeng; Riley, William J. Advances in Water Resources, Vol. 86 https://doi.org/10.1016/j.advwatres.2015.09.027	journal	December 2015
Four decades of microwave satellite soil moisture observations: Part 2. Product validation and inter-satellite comparisons Karthikeyan, L.; Pan, Ming; Wanders, Niko Advances in Water Resources, Vol. 109 https://doi.org/10.1016/j.advwatres.2017.09.010	journal	November 2017
Relationships of pest grasshopper populations in Alberta, Canada to soil moisture and climate variables Powell, L. R.; Berg, A. A.; Johnson, D. L. Agricultural and Forest Meteorology, Vol. 144, Issue 1-2 https://doi.org/10.1016/j.agrformet.2007.01.013	journal	May 2007
Modelling desert locust presences using 32-year soil moisture data on a large-scale Gómez, Diego; Salvador, Pablo; Sanz, Julia Ecological Indicators, Vol. 117 https://doi.org/10.1016/j.ecolind.2020.106655	journal	October 2020
Catchment scale soil moisture spatial–temporal variability Brocca, L.; Tullo, T.; Melone, F. Journal of Hydrology, Vol. 422-423 https://doi.org/10.1016/j.jhydrol.2011.12.039	journal	February 2012
Combining SMOS with visible and near/shortwave/thermal infrared satellite data for high resolution soil moisture estimates Sánchez-Ruiz, Sergio; Piles, María; Sánchez, Nilda Journal of Hydrology, Vol. 516 https://doi.org/10.1016/j.jhydrol.2013.12.047	journal	August 2014
Soil moisture at local scale: Measurements and simulations Romano, Nunzio Journal of Hydrology, Vol. 516 https://doi.org/10.1016/j.jhydrol.2014.01.026	journal	August 2014
Evaluation of multi-model simulated soil moisture in NLDAS-2 Xia, Youlong; Sheffield, Justin; Ek, Michael B. Journal of Hydrology, Vol. 512 https://doi.org/10.1016/j.jhydrol.2014.02.027	journal	May 2014
Hydrologic model calibration using remotely sensed soil moisture and discharge measurements: The impact on predictions at gauged and ungauged locations Li, Yuan; Grimaldi, Stefania; Pauwels, Valentijn R. N. Journal of Hydrology, Vol. 557 https://doi.org/10.1016/j.jhydrol.2018.01.013	journal	February 2018
Influence of changes in rainfall and soil moisture on trends in flooding Wasko, Conrad; Nathan, Rory Journal of Hydrology, Vol. 575 https://doi.org/10.1016/j.jhydrol.2019.05.054	journal	August 2019
Evaluation and validation of a high spatial resolution satellite soil moisture product over the Continental United States Fang, Bin; Lakshmi, Venkataraman; Bindlish, Rajat Journal of Hydrology, Vol. 588 https://doi.org/10.1016/j.jhydrol.2020.125043	journal	September 2020
Continental-scale streamflow modeling of basins with reservoirs: Towards a coherent deep-learning-based strategy Ouyang, Wenyu; Lawson, Kathryn; Feng, Dapeng Journal of Hydrology, Vol. 599 https://doi.org/10.1016/j.jhydrol.2021.126455	journal	August 2021
Improved daily SMAP satellite soil moisture prediction over China using deep learning model with transfer learning Li, Qingliang; Wang, Ziyu; Shangguan, Wei Journal of Hydrology, Vol. 600 https://doi.org/10.1016/j.jhydrol.2021.126698	journal	September 2021
Validation of SMAP surface soil moisture products with core validation sites Colliander, A.; Jackson, T. J.; Bindlish, R. Remote Sensing of Environment, Vol. 191 https://doi.org/10.1016/j.rse.2017.01.021	journal	March 2017
Estimating surface soil moisture from SMAP observations using a Neural Network technique Kolassa, J.; Reichle, R. H.; Liu, Q. Remote Sensing of Environment, Vol. 204 https://doi.org/10.1016/j.rse.2017.10.045	journal	January 2018
Generation of spatially complete and daily continuous surface soil moisture of high spatial resolution Long, Di; Bai, Liangliang; Yan, La Remote Sensing of Environment, Vol. 233 https://doi.org/10.1016/j.rse.2019.111364	journal	November 2019
Soil moisture-based index for agricultural drought assessment: SMADI application in Pernambuco State-Brazil Souza, Alzira Gabrielle Soares Saraiva; Ribeiro Neto, Alfredo; Souza, Laio Lucas de Remote Sensing of Environment, Vol. 252 https://doi.org/10.1016/j.rse.2020.112124	journal	January 2021
Estimation and evaluation of high-resolution soil moisture from merged model and Earth observation data in the Great Britain Peng, Jian; Tanguy, Maliko; Robinson, Emma L. Remote Sensing of Environment, Vol. 264 https://doi.org/10.1016/j.rse.2021.112610	journal	October 2021
Evaluating the applicability of soil moisture-based metrics for gauging the resiliency of rainfed agricultural systems in the midwestern United States Eeswaran, Rasu; Nejadhashemi, A. Pouyan; Alves, Filipe Couto Soil and Tillage Research, Vol. 205 https://doi.org/10.1016/j.still.2020.104818	journal	January 2021
From Hydrometeorology to River Water Quality: Can a Deep Learning Model Predict Dissolved Oxygen at the Continental Scale? Zhi, Wei; Feng, Dapeng; Tsai, Wen-Ping Environmental Science & Technology, Vol. 55, Issue 4 https://doi.org/10.1021/acs.est.0c06783	journal	February 2021
Field observations of soil moisture variability across scales: SOIL MOISTURE VARIABILITY ACROSS SCALES Famiglietti, James S.; Ryu, Dongryeol; Berg, Aaron A. Water Resources Research, Vol. 44, Issue 1 https://doi.org/10.1029/2006wr005804	journal	January 2008
Downscaling soil moisture in the southern Great Plains through a calibrated multifractal model for land surface modeling applications: DOWNSCALING SOIL MOISTURE IN THE GREAT PLAINS Mascaro, Giuseppe; Vivoni, Enrique R.; Deidda, Roberto Water Resources Research, Vol. 46, Issue 8 https://doi.org/10.1029/2009wr008855	journal	August 2010
Continental-scale water and energy flux analysis and validation for the North American Land Data Assimilation System project phase 2 (NLDAS-2): 1. Intercomparison and application of model products: WATER AND ENERGY FLUX ANALYSIS Xia, Youlong; Mitchell, Kenneth; Ek, Michael Journal of Geophysical Research: Atmospheres, Vol. 117, Issue D3 https://doi.org/10.1029/2011JD016048	journal	February 2012
Downscaling SMAP Radiometer Soil Moisture Over the CONUS Using an Ensemble Learning Method Abbaszadeh, Peyman; Moradkhani, Hamid; Zhan, Xiwu Water Resources Research, Vol. 55, Issue 1 https://doi.org/10.1029/2018wr023354	journal	January 2019
A Global Assessment of Added Value in the SMAP Level 4 Soil Moisture Product Relative to Its Baseline Land Surface Model Dong, Jianzhi; Crow, Wade; Reichle, Rolf Geophysical Research Letters, Vol. 46, Issue 12 https://doi.org/10.1029/2019GL083398	journal	June 2019
A Rainfall‐Runoff Model With LSTM‐Based Sequence‐to‐Sequence Learning Xiang, Zhongrun; Yan, Jun; Demir, Ibrahim Water Resources Research, Vol. 56, Issue 1 https://doi.org/10.1029/2019WR025326	journal	January 2020
Enhancing Streamflow Forecast and Extracting Insights Using Long‐Short Term Memory Networks With Data Integration at Continental Scales Feng, Dapeng; Fang, Kuai; Shen, Chaopeng Water Resources Research, Vol. 56, Issue 9 https://doi.org/10.1029/2019WR026793	journal	September 2020
Gap Filling of High‐Resolution Soil Moisture for SMAP/Sentinel‐1: A Two‐Layer Machine Learning‐Based Framework Mao, Hanzi; Kathuria, Dhruva; Duffield, Nick Water Resources Research, Vol. 55, Issue 8 https://doi.org/10.1029/2019wr024902	journal	August 2019
What Role Does Hydrological Science Play in the Age of Machine Learning? Nearing, Grey S.; Kratzert, Frederik; Sampson, Alden Keefe Water Resources Research, Vol. 57, Issue 3 https://doi.org/10.1029/2020WR028091	journal	March 2021
Evaluating the Potential and Challenges of an Uncertainty Quantification Method for Long Short‐Term Memory Models for Soil Moisture Predictions Fang, Kuai; Kifer, Daniel; Lawson, Kathryn Water Resources Research, Vol. 56, Issue 12 https://doi.org/10.1029/2020wr028095	journal	December 2020
Transferring Hydrologic Data Across Continents – Leveraging Data‐Rich Regions to Improve Hydrologic Prediction in Data‐Sparse Regions Ma, Kai; Feng, Dapeng; Lawson, Kathryn Water Resources Research, Vol. 57, Issue 5 https://doi.org/10.1029/2020wr028600	journal	April 2021
Mitigating Prediction Error of Deep Learning Streamflow Models in Large Data‐Sparse Regions With Ensemble Modeling and Soft Data Feng, Dapeng; Lawson, Kathryn; Shen, Chaopeng Geophysical Research Letters, Vol. 48, Issue 14 https://doi.org/10.1029/2021GL092999	journal	July 2021
From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling Tsai, Wen-Ping; Feng, Dapeng; Pan, Ming Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-26107-z	journal	October 2021
Global soil moisture data derived through machine learning trained with in-situ measurements O., Sungmin; Orth, Rene Scientific Data, Vol. 8, Issue 1 https://doi.org/10.1038/s41597-021-00964-1	journal	July 2021
Exploring the exceptional performance of a deep learning stream temperature model and the value of streamflow data Rahmani, Farshid; Lawson, Kathryn; Ouyang, Wenyu Environmental Research Letters https://doi.org/10.1088/1748-9326/abd501	journal	December 2020
Very High Spatial Resolution Downscaled SMAP Radiometer Soil Moisture in the CONUS Using VIIRS/MODIS Data Fang, Bin; Lakshmi, Venkat; Cosh, Michael IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 14 https://doi.org/10.1109/JSTARS.2021.3076026	journal	January 2021
Assessment and Combination of SMAP and Sentinel-1A/B-Derived Soil Moisture Estimates With Land Surface Model Outputs in the Mid-Atlantic Coastal Plain, USA Kim, Hyunglok; Lee, Sangchul; Cosh, Michael H. IEEE Transactions on Geoscience and Remote Sensing, Vol. 59, Issue 2 https://doi.org/10.1109/TGRS.2020.2991665	journal	February 2021
The Value of SMAP for Long-Term Soil Moisture Estimation With the Help of Deep Learning Fang, Kuai; Pan, Ming; Shen, Chaopeng IEEE Transactions on Geoscience and Remote Sensing, Vol. 57, Issue 4 https://doi.org/10.1109/tgrs.2018.2872131	journal	April 2019
Soil moisture from remote sensing to forecast desert locust presence Piou, Cyril; Gay, Pierre‐Emmanuel; Benahi, Ahmed Salem Journal of Applied Ecology, Vol. 56, Issue 4 https://doi.org/10.1111/1365-2664.13323	journal	January 2019
A Multiscale Ensemble Filtering System for Hydrologic Data Assimilation. Part I: Implementation and Synthetic Experiment Pan, Ming; Wood, Eric F.; McLaughlin, Dennis B. Journal of Hydrometeorology, Vol. 10, Issue 3 https://doi.org/10.1175/2009JHM1088.1	journal	June 2009
Estimating Spatial Sampling Errors in Coarse-Scale Soil Moisture Estimates Derived from Point-Scale Observations Miralles, Diego G.; Crow, Wade T.; Cosh, Michael H. Journal of Hydrometeorology, Vol. 11, Issue 6 https://doi.org/10.1175/2010JHM1285.1	journal	December 2010
U.S. Climate Reference Network after One Decade of Operations: Status and Assessment Diamond, Howard J.; Karl, Thomas R.; Palecki, Michael A. Bulletin of the American Meteorological Society, Vol. 94, Issue 4 https://doi.org/10.1175/BAMS-D-12-00170.1	journal	April 2013
High-Resolution SMAP Satellite Soil Moisture Product: Exploring the Opportunities Abbaszadeh, Peyman; Moradkhani, Hamid; Gavahi, Keyhan Bulletin of the American Meteorological Society, Vol. 102, Issue 4 https://doi.org/10.1175/BAMS-D-21-0016.1	journal	April 2021
U.S. Climate Reference Network Soil Moisture and Temperature Observations Bell, Jesse E.; Palecki, Michael A.; Baker, C. Bruce Journal of Hydrometeorology, Vol. 14, Issue 3 https://doi.org/10.1175/JHM-D-12-0146.1	journal	June 2013
Development of a Daily Multilayer Cropland Soil Moisture Dataset for China Using Machine Learning and Application to Cropping Patterns Liu, Yaling; Chen, Dongdong; Mouatadid, Soukayna Journal of Hydrometeorology, Vol. 22, Issue 2 https://doi.org/10.1175/JHM-D-19-0301.1	journal	February 2021
Different Rates of Soil Drying after Rainfall Are Observed by the SMOS Satellite and the South Fork in situ Soil Moisture Network Rondinelli, Wesley J.; Hornbuckle, Brian K.; Patton, Jason C. Journal of Hydrometeorology, Vol. 16, Issue 2 https://doi.org/10.1175/jhm-d-14-0137.1	journal	April 2015
Near-Real-Time Forecast of Satellite-Based Soil Moisture Using Long Short-Term Memory with an Adaptive Data Integration Kernel Fang, Kuai; Shen, Chaopeng Journal of Hydrometeorology, Vol. 21, Issue 3 https://doi.org/10.1175/jhm-d-19-0169.1	journal	March 2019
Temporal Stability of Spatially Measured Soil Water Probability Density Function1 Vachaud, G.; Passerat De Silans, A.; Balabanis, P. Soil Science Society of America Journal, Vol. 49, Issue 4 https://doi.org/10.2136/sssaj1985.03615995004900040006x	journal	January 1985
Letter to the Editor on “Rank Stability or Temporal Stability” Chen, You-jun Soil Science Society of America Journal, Vol. 70, Issue 1 https://doi.org/10.2136/sssaj2005.0290l	journal	January 2006
Groundwater Estimation from Major Physical Hydrology Components Using Artificial Neural Networks and Deep Learning Afzaal, Hassan; Farooque, Aitazaz A.; Abbas, Farhat Water, Vol. 12, Issue 1 https://doi.org/10.3390/w12010005	journal	December 2019
NLDAS Primary Forcing Data L4 Hourly 0.125 x 0.125 degree, Version 002 Xia, Y.; Al., Et NASA Goddard Earth Sciences Data and Information Services Center https://doi.org/10.5067/6J5LHHOHZHN4	dataset	January 2009
SMAP L3 Radiometer Global Daily 36 km EASE-Grid Soil Moisture, Version 7 ONeill, Peggy E.; Chan, Steven; Njoku, Eni G. NASA National Snow and Ice Data Center DAAC https://doi.org/10.5067/HH4SZ2PXSP6A	dataset	January 2020
A dynamic approach for evaluating coarse scale satellite soil moisture products Loew, A.; Schlenz, F. Hydrology and Earth System Sciences, Vol. 15, Issue 1 https://doi.org/10.5194/hess-15-75-2011	journal	January 2011
Characterizing coarse-resolution watershed soil moisture heterogeneity using fine-scale simulations Riley, W. J.; Shen, C. Hydrology and Earth System Sciences, Vol. 18, Issue 7 https://doi.org/10.5194/hess-18-2463-2014	journal	January 2014
Global downscaling of remotely sensed soil moisture using neural networks Alemohammad, Seyed Hamed; Kolassa, Jana; Prigent, Catherine Hydrology and Earth System Sciences, Vol. 22, Issue 10 https://doi.org/10.5194/hess-22-5341-2018	journal	January 2018
Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets Kratzert, Frederik; Klotz, Daniel; Shalev, Guy Hydrology and Earth System Sciences, Vol. 23, Issue 12 https://doi.org/10.5194/hess-23-5089-2019	journal	January 2019
Evaluation of 18 satellite- and model-based soil moisture products using in situ measurements from 826 sensors Beck, Hylke E.; Pan, Ming; Miralles, Diego G. Hydrology and Earth System Sciences, Vol. 25, Issue 1 https://doi.org/10.5194/hess-25-17-2021	journal	January 2021