A dynamic data-driven method for dealing with model structural error in soil moisture data assimilation

Zhang, Qiuru; Shi, Liangsheng; Holzman, Mauro; Ye, Ming; Wang, Yakun; Carmona, Facundo; Zha, Yuanyuan

doi:10.1016/j.advwatres.2019.103407

Title: A dynamic data-driven method for dealing with model structural error in soil moisture data assimilation

Journal Article · Thu Aug 22 00:00:00 EDT 2019 · Advances in Water Resources

DOI:https://doi.org/10.1016/j.advwatres.2019.103407· OSTI ID:1803818

Zhang, Qiuru ^[1]; Shi, Liangsheng ^[1];

^[2];

^[3]; Wang, Yakun ^[1]; Carmona, Facundo ^[2]; Zha, Yuanyuan ^[1]

Wuhan Univ. (China)
Inst. de Hidrología de Llanuras, Azul-Tandil (Argentina)
Florida State Univ., Tallahassee, FL (United States)

Attributing to the flexibility in considering various types of observation error and model error, data assimilation has been increasingly applied to dynamically improve soil moisture modeling in many hydrological practices. However, accurate characterization of model error, especially the part caused by defective model structure, presents a significant challenge to the successful implementation of data assimilation. Model structural error has received limited attention relative to parameter and input errors, mainly due to our poor understanding of structural inadequacy and the difficulties in parameterizing structural error. In this paper, we present a dynamic data-driven approach to estimate the model structural error in soil moisture data assimilation without the need for identifying error generation mechanism or specifying particular form for the error model. The error model is based on the Gaussian process regression and then integrated into the ensemble Kalman filter (EnKF) to form a hybrid method for dealing with multi-source model errors. Two variants of the hybrid method in terms of two different error correction manners are proposed. The effectiveness of the proposed method is tested through a suit of synthetic cases and a real-world case. Results demonstrate the potential of the proposed hybrid method for estimating model structural error and providing improved model predictions. Compared to the traditional EnKF without explicitly considering the model structural error, parameter compensation issue is obviously reduced and soil moisture retrieval is substantially improved.

View Accepted Manuscript (DOE)

View Accepted Manuscript (Publisher)

Cite

Export

Save

Research Organization:: Florida State Univ., Tallahassee, FL (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: SC0019438

OSTI ID:: 1803818

Alternate ID(s):: OSTI ID: 1566241

Journal Information:: Advances in Water Resources, Vol. 132; ISSN 0309-1708

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 29 works

Citation information provided by
Web of Science

References (63)

Soil moisture prediction with the ensemble Kalman filter: Handling uncertainty of soil hydraulic parameters Brandhorst, N.; Erdal, D.; Neuweiler, I. Advances in Water Resources, Vol. 110 https://doi.org/10.1016/j.advwatres.2017.10.022	journal	December 2017
Integrating a calibrated groundwater flow model with error-correcting data-driven models to improve predictions Demissie, Yonas K.; Valocchi, Albert J.; Minsker, Barbara S. Journal of Hydrology, Vol. 364, Issue 3-4 https://doi.org/10.1016/j.jhydrol.2008.11.007	journal	January 2009
Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 8 https://doi.org/10.1109/TPAMI.2005.159	journal	August 2005
Data assimilation in the presence of forecast bias Dee, Dick P.; Da Silva, Arlindo M. Quarterly Journal of the Royal Meteorological Society, Vol. 124, Issue 545 https://doi.org/10.1002/qj.49712454512	journal	January 1998
Links Between Root Length Density Profiles and Models of the Root System Architecture Pagès, Loïc; Bruchou, Claude; Garré, Sarah Vadose Zone Journal, Vol. 11, Issue 4 https://doi.org/10.2136/vzj2011.0152	journal	October 2012
Correcting for forecast bias in soil moisture assimilation with the ensemble Kalman filter: CORRECTING FOR FORECAST BIAS IN SOIL MOISTURE ASSIMILATION De Lannoy, Gabriëlle J. M.; Reichle, Rolf H.; Houser, Paul R. Water Resources Research, Vol. 43, Issue 9 https://doi.org/10.1029/2006WR005449	journal	September 2007
Spatial and temporal characteristics of soil moisture in an intensively monitored agricultural field (OPE3) De Lannoy, Gabriëlle J. M.; Verhoest, Niko E. C.; Houser, Paul R. Journal of Hydrology, Vol. 331, Issue 3-4 https://doi.org/10.1016/j.jhydrol.2006.06.016	journal	December 2006
Accounting for Model Errors in Ensemble Data Assimilation Li, Hong; Kalnay, Eugenia; Miyoshi, Takemasa Monthly Weather Review, Vol. 137, Issue 10 https://doi.org/10.1175/2009MWR2766.1	journal	October 2009
The Ensemble Kalman Filter: theoretical formulation and practical implementation Evensen, Geir Ocean Dynamics, Vol. 53, Issue 4 https://doi.org/10.1007/s10236-003-0036-9	journal	November 2003
Using a bias aware EnKF to account for unresolved structure in an unsaturated zone model: BIAS AWARE ENKF FOR UNRESOLVED STRUCTURE Erdal, D.; Neuweiler, I.; Wollschläger, U. Water Resources Research, Vol. 50, Issue 1 https://doi.org/10.1002/2012WR013443	journal	January 2014
Challenges of modifying root traits in crops for agriculture Meister, Robert; Rajani, M. S.; Ruzicka, Daniel Trends in Plant Science, Vol. 19, Issue 12 https://doi.org/10.1016/j.tplants.2014.08.005	journal	December 2014
Comparison of ensemble-based state and parameter estimation methods for soil moisture data assimilation Chen, Weijing; Huang, Chunlin; Shen, Huanfeng Advances in Water Resources, Vol. 86 https://doi.org/10.1016/j.advwatres.2015.08.003	journal	December 2015
Data assimilation methods in the Earth sciences Reichle, Rolf H. Advances in Water Resources, Vol. 31, Issue 11 https://doi.org/10.1016/j.advwatres.2008.01.001	journal	November 2008
Toward reduction of model uncertainty: Integration of Bayesian model averaging and data assimilation: TOWARD REDUCTION OF MODEL UNCERTAINTY Parrish, Mark A.; Moradkhani, Hamid; DeChant, Caleb M. Water Resources Research, Vol. 48, Issue 3 https://doi.org/10.1029/2011WR011116	journal	March 2012
Hydraulic parameter estimation by remotely-sensed top soil moisture observations with the particle filter Montzka, Carsten; Moradkhani, Hamid; Weihermüller, Lutz Journal of Hydrology, Vol. 399, Issue 3-4 https://doi.org/10.1016/j.jhydrol.2011.01.020	journal	March 2011
A Bayesian approach to improved calibration and prediction of groundwater models with structural error Xu, Tianfang; Valocchi, Albert J. Water Resources Research, Vol. 51, Issue 11 https://doi.org/10.1002/2015WR017912	journal	November 2015
A framework for dealing with uncertainty due to model structure error Refsgaard, Jens Christian; van der Sluijs, Jeroen P.; Brown, James Advances in Water Resources, Vol. 29, Issue 11 https://doi.org/10.1016/j.advwatres.2005.11.013	journal	November 2006
An improved variable selection method for support vector regression in NIR spectral modeling Xu, Shu; Lu, Bo; Baldea, Michael Journal of Process Control, Vol. 67 https://doi.org/10.1016/j.jprocont.2017.06.001	journal	July 2018
A short exploration of structural noise: A SHORT EXPLORATION OF STRUCTURAL NOISE Doherty, John; Welter, David Water Resources Research, Vol. 46, Issue 5 https://doi.org/10.1029/2009WR008377	journal	May 2010
Data assimilation of soil water flow via ensemble Kalman filter: Infusing soil moisture data at different scales Zhu, Penghui; Shi, Liangsheng; Zhu, Yan Journal of Hydrology, Vol. 555 https://doi.org/10.1016/j.jhydrol.2017.10.078	journal	December 2017
Treatment of bias in recursive filtering Friedland, B. IEEE Transactions on Automatic Control, Vol. 14, Issue 4 https://doi.org/10.1109/TAC.1969.1099223	journal	August 1969
Assessing parameter, precipitation, and predictive uncertainty in a distributed hydrological model using sequential data assimilation with the particle filter Salamon, Peter; Feyen, Luc Journal of Hydrology, Vol. 376, Issue 3-4 https://doi.org/10.1016/j.jhydrol.2009.07.051	journal	October 2009
An adaptive covariance inflation error correction algorithm for ensemble filters Anderson, Jeffrey L. Tellus A: Dynamic Meteorology and Oceanography, Vol. 59, Issue 2 https://doi.org/10.1111/j.1600-0870.2006.00216.x	journal	January 2007
Numerical Comparison of Iterative Ensemble Kalman Filters for Unsaturated Flow Inverse Modeling Song, Xuehang; Shi, Liangsheng; Ye, Ming Vadose Zone Journal, Vol. 13, Issue 2 https://doi.org/10.2136/vzj2013.05.0083	journal	January 2014
A generalized Ross method for two- and three-dimensional variably saturated flow Zha, Yuanyuan; Shi, Liangsheng; Ye, Ming Advances in Water Resources, Vol. 54 https://doi.org/10.1016/j.advwatres.2013.01.002	journal	April 2013
Data assimilation for unsaturated flow models with restart adaptive probabilistic collocation based Kalman filter Man, Jun; Li, Weixuan; Zeng, Lingzao Advances in Water Resources, Vol. 92 https://doi.org/10.1016/j.advwatres.2016.03.016	journal	June 2016
Modeling Soil Water and Solute Transport-Fast, Simplified Numerical Solutions Ross, P. J. Agronomy Journal, Vol. 95, Issue 6 https://doi.org/10.2134/agronj2003.1352	journal	November 2003
A Bayesian tutorial for data assimilation Wikle, Christopher K.; Berliner, L. Mark Physica D: Nonlinear Phenomena, Vol. 230, Issue 1-2 https://doi.org/10.1016/j.physd.2006.09.017	journal	June 2007
Determining soil moisture by assimilating soil temperature measurements using the Ensemble Kalman Filter Dong, Jianzhi; Steele-Dunne, Susan C.; Ochsner, Tyson E. Advances in Water Resources, Vol. 86 https://doi.org/10.1016/j.advwatres.2015.08.011	journal	December 2015
An integrated hydrologic Bayesian multimodel combination framework: Confronting input, parameter, and model structural uncertainty in hydrologic prediction: AN INTEGRATED BAYESIAN MULTIMODEL FRAMEWORK Ajami, Newsha K.; Duan, Qingyun; Sorooshian, Soroosh Water Resources Research, Vol. 43, Issue 1 https://doi.org/10.1029/2005WR004745	journal	January 2007
Multi-source data assimilation for physically based hydrological modeling of an experimental hillslope Botto, Anna; Belluco, Enrica; Camporese, Matteo Hydrology and Earth System Sciences, Vol. 22, Issue 8 https://doi.org/10.5194/hess-22-4251-2018	journal	January 2018
Accounting for the Error due to Unresolved Scales in Ensemble Data Assimilation: A Comparison of Different Approaches Hamill, Thomas M.; Whitaker, Jeffrey S. Monthly Weather Review, Vol. 133, Issue 11 https://doi.org/10.1175/MWR3020.1	journal	November 2005
Towards a comprehensive assessment of model structural adequacy: ASSESSMENT OF MODEL STRUCTURAL ADEQUACY Gupta, Hoshin V.; Clark, Martyn P.; Vrugt, Jasper A. Water Resources Research, Vol. 48, Issue 8 https://doi.org/10.1029/2011WR011044	journal	August 2012
Data‐Driven Model Uncertainty Estimation in Hydrologic Data Assimilation Pathiraja, S.; Moradkhani, H.; Marshall, L. Water Resources Research, Vol. 54, Issue 2 https://doi.org/10.1002/2018WR022627	journal	February 2018
Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter: UNCERTAINTY ASSESSMENT OF HYDROLOGIC MODEL Moradkhani, Hamid; Hsu, Kuo-Lin; Gupta, Hoshin Water Resources Research, Vol. 41, Issue 5 https://doi.org/10.1029/2004WR003604	journal	May 2005
Treatment of uncertainty using ensemble methods: Comparison of sequential data assimilation and Bayesian model averaging: SEQUENTIAL DATA ASSIMILATION Vrugt, Jasper A.; Robinson, Bruce A. Water Resources Research, Vol. 43, Issue 1 https://doi.org/10.1029/2005WR004838	journal	January 2007
Kalman filters for assimilating near-surface observations into the Richards equation – Part 3: Retrieving states and parameters from laboratory evaporation experiments Medina, H.; Romano, N.; Chirico, G. B. Hydrology and Earth System Sciences, Vol. 18, Issue 7 https://doi.org/10.5194/hess-18-2543-2014	journal	January 2014
Bias aware Kalman filters: Comparison and improvements Drécourt, Jean-Philippe; Madsen, Henrik; Rosbjerg, Dan Advances in Water Resources, Vol. 29, Issue 5 https://doi.org/10.1016/j.advwatres.2005.07.006	journal	May 2006
One-dimensional soil moisture profile retrieval by assimilation of near-surface observations: a comparison of retrieval algorithms Walker, Jeffrey P.; Willgoose, Garry R.; Kalma, Jetse D. Advances in Water Resources, Vol. 24, Issue 6 https://doi.org/10.1016/S0309-1708(00)00043-9	journal	June 2001
Dual state-parameter estimation of root zone soil moisture by optimal parameter estimation and extended Kalman filter data assimilation Lü, Haishen; Yu, Zhongbo; Zhu, Yonghua Advances in Water Resources, Vol. 34, Issue 3 https://doi.org/10.1016/j.advwatres.2010.12.005	journal	March 2011
Correcting Biased Observation Model Error in Data Assimilation Berry, Tyrus; Harlim, John Monthly Weather Review, Vol. 145, Issue 7 https://doi.org/10.1175/MWR-D-16-0428.1	journal	July 2017
Error covariance calculation for forecast bias estimation in hydrologic data assimilation Pauwels, Valentijn R. N.; De Lannoy, Gabriëlle J. M. Advances in Water Resources, Vol. 86 https://doi.org/10.1016/j.advwatres.2015.05.013	journal	December 2015
Dynamic modeling of predictive uncertainty by regression on absolute errors: DYNAMIC MODELING OF PREDICTIVE UNCERTAINTY Pianosi, F.; Raso, L. Water Resources Research, Vol. 48, Issue 3 https://doi.org/10.1029/2011WR010603	journal	March 2012
Real-time groundwater flow modeling with the Ensemble Kalman Filter: Joint estimation of states and parameters and the filter inbreeding problem: REAL-TIME GROUNDWATER FLOW MODELING Hendricks Franssen, H. J.; Kinzelbach, W. Water Resources Research, Vol. 44, Issue 9 https://doi.org/10.1029/2007WR006505	journal	September 2008
Vadose Zone Model-Data Fusion: State of the Art and Future Challenges Huisman, Johan A.; Vrugt, Jasper A.; Ferre, Ty P. A. Vadose Zone Journal, Vol. 11, Issue 4 https://doi.org/10.2136/vzj2012.0140	journal	November 2012
Kalman filters for assimilating near-surface observations into the Richards equation – Part 2: A dual filter approach for simultaneous retrieval of states and parameters Medina, H.; Romano, N.; Chirico, G. B. Hydrology and Earth System Sciences, Vol. 18, Issue 7 https://doi.org/10.5194/hess-18-2521-2014	journal	January 2014
State and bias estimation for soil moisture profiles by an ensemble Kalman filter: Effect of assimilation depth and frequency: STATE AND BIAS ESTIMATION FOR SOIL MOISTURE De Lannoy, Gabriëlle J. M.; Houser, Paul R.; Pauwels, Valentijn R. N. Water Resources Research, Vol. 43, Issue 6 https://doi.org/10.1029/2006WR005100	journal	June 2007
An Iterative Ensemble Kalman Filter Lorentzen, Rolf J.; Naevdal, Geir IEEE Transactions on Automatic Control, Vol. 56, Issue 8 https://doi.org/10.1109/TAC.2011.2154430	journal	August 2011
Developing joint probability distributions of soil water retention characteristics Carsel, Robert F.; Parrish, Rudolph S. Water Resources Research, Vol. 24, Issue 5 https://doi.org/10.1029/WR024i005p00755	journal	May 1988
A simulation model of water dynamics in winter wheat field and its application in a semiarid region Kang, Shaozhong; Zhang, Fucang; Zhang, Jianhua Agricultural Water Management, Vol. 49, Issue 2 https://doi.org/10.1016/S0378-3774(00)00137-2	journal	July 2001
Model Error Estimation Employing an Ensemble Data Assimilation Approach Zupanski, Dusanka; Zupanski, Milija Monthly Weather Review, Vol. 134, Issue 5 https://doi.org/10.1175/MWR3125.1	journal	May 2006
Soil water uptake and root distribution of different perennial and annual bioenergy crops Ferchaud, Fabien; Vitte, Guillaume; Bornet, Frédéric Plant and Soil, Vol. 388, Issue 1-2 https://doi.org/10.1007/s11104-014-2335-y	journal	November 2014
Estimating Model Parameters with Ensemble-Based Data Assimilation: A Review Ruiz, Juan Jose; Pulido, Manuel; Miyoshi, Takemasa Journal of the Meteorological Society of Japan. Ser. II, Vol. 91, Issue 2 https://doi.org/10.2151/jmsj.2013-201	journal	January 2013
An improved approach for estimating observation and model error parameters in soil moisture data assimilation: ESTIMATING DATA ASSIMILATION ERROR PARAMETERS Crow, W. T.; van den Berg, M. J. Water Resources Research, Vol. 46, Issue 12 https://doi.org/10.1029/2010WR009402	journal	December 2010
Richards Equation-Based Modeling to Estimate Flow and Nitrate Transport in a Deep Alluvial Vadose Zone Botros, Farag E.; Onsoy, Yuksel S.; Ginn, Timothy R. Vadose Zone Journal, Vol. 11, Issue 4 https://doi.org/10.2136/vzj2011.0145	journal	November 2012
Impacts of different types of measurements on estimating unsaturated flow parameters Shi, Liangsheng; Song, Xuehang; Tong, Juxiu Journal of Hydrology, Vol. 524 https://doi.org/10.1016/j.jhydrol.2015.01.078	journal	May 2015
Uncertainty in hydrologic modeling: Toward an integrated data assimilation framework: HYDROLOGIC DATA ASSIMILATION Liu, Yuqiong; Gupta, Hoshin V. Water Resources Research, Vol. 43, Issue 7 https://doi.org/10.1029/2006WR005756	journal	July 2007
A Closed-form Equation for Predicting the Hydraulic Conductivity of Unsaturated Soils1 van Genuchten, M. Th. Soil Science Society of America Journal, Vol. 44, Issue 5 https://doi.org/10.2136/sssaj1980.03615995004400050002x	journal	January 1980
Improved treatment of uncertainty in hydrologic modeling: Combining the strengths of global optimization and data assimilation: TREATMENT OF UNCERTAINTY IN HYDROLOGIC MODELING Vrugt, Jasper A.; Diks, Cees G. H.; Gupta, Hoshin V. Water Resources Research, Vol. 41, Issue 1 https://doi.org/10.1029/2004WR003059	journal	January 2005
Bias and data assimilation Dee, D. P. Quarterly Journal of the Royal Meteorological Society, Vol. 131, Issue 613 https://doi.org/10.1256/qj.05.137	journal	October 2005
An Information-Theoretic Framework for Improving Imperfect Dynamical Predictions Via Multi-Model Ensemble Forecasts Branicki, Michal; Majda, Andrew J. Journal of Nonlinear Science, Vol. 25, Issue 3 https://doi.org/10.1007/s00332-015-9233-1	journal	March 2015
A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts Anderson, Jeffrey L.; Anderson, Stephen L. Monthly Weather Review, Vol. 127, Issue 12 https://doi.org/10.1175/1520-0493(1999)127<2741:AMCIOT>2.0.CO;2	journal	December 1999
Modeling Root Water Uptake in Hydrological and Climate Models Feddes, Reinder A.; Hoff, Holger; Bruen, Michael Bulletin of the American Meteorological Society, Vol. 82, Issue 12 https://doi.org/10.1175/1520-0477(2001)082<2797:MRWUIH>2.3.CO;2	journal	December 2001

Cited By (1)

Surrogate-Based Bayesian Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Approximation Error Zhang, Jiangjiang; Zheng, Qiang; Chen, Dingjiang arXiv https://doi.org/10.48550/arxiv.1807.05187	text	January 2018

Similar Records

A Hybrid Global Ocean Data Assimilation System at NCEP

Journal Article · Thu Oct 29 00:00:00 EDT 2015 · Monthly Weather Review · OSTI ID:1803818

Penny, Stephen G.; Behringer, David W.; Carton, James A.; +1 more

Estimating surface carbon fluxes based on a local ensemble transform Kalman filter with a short assimilation window and a long observation window: an observing system simulation experiment test in GEOS-Chem 10.1

Journal Article · Fri Jul 12 00:00:00 EDT 2019 · Geoscientific Model Development (Online) · OSTI ID:1803818

Liu, Yun; Kalnay, Eugenia; Zeng, Ning; +3 more

An improved framework for the dynamic likelihood filtering approach to data assimilation

Journal Article · Tue May 10 00:00:00 EDT 2022 · Chaos: An Interdisciplinary Journal of Nonlinear Science · OSTI ID:1803818

Foster, Dallas; Restrepo, Juan M.

Related Subjects

54 ENVIRONMENTAL SCIENCES
soil moisture
data assimilation
model structural error
data-driving
machine learning

Title: A dynamic data-driven method for dealing with model structural error in soil moisture data assimilation

Citation Formats

References (63)

Cited By (1)

Similar Records

Related Subjects