A dynamic data-driven method for dealing with model structural error in soil moisture data assimilation
Abstract
Attributing to the flexibility in considering various types of observation error and model error, data assimilation has been increasingly applied to dynamically improve soil moisture modeling in many hydrological practices. However, accurate characterization of model error, especially the part caused by defective model structure, presents a significant challenge to the successful implementation of data assimilation. Model structural error has received limited attention relative to parameter and input errors, mainly due to our poor understanding of structural inadequacy and the difficulties in parameterizing structural error. In this paper, we present a dynamic data-driven approach to estimate the model structural error in soil moisture data assimilation without the need for identifying error generation mechanism or specifying particular form for the error model. The error model is based on the Gaussian process regression and then integrated into the ensemble Kalman filter (EnKF) to form a hybrid method for dealing with multi-source model errors. Two variants of the hybrid method in terms of two different error correction manners are proposed. The effectiveness of the proposed method is tested through a suit of synthetic cases and a real-world case. Results demonstrate the potential of the proposed hybrid method for estimating model structural error and providingmore »
- Authors:
-
[2];
[3];
- Wuhan Univ. (China)
- Inst. de Hidrología de Llanuras, Azul-Tandil (Argentina)
- Florida State Univ., Tallahassee, FL (United States)
- Publication Date:
- Research Org.:
- Florida State Univ., Tallahassee, FL (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Biological and Environmental Research (BER)
- OSTI Identifier:
- 1803818
- Alternate Identifier(s):
- OSTI ID: 1566241
- Grant/Contract Number:
- SC0019438
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Advances in Water Resources
- Additional Journal Information:
- Journal Volume: 132; Journal ID: ISSN 0309-1708
- Publisher:
- Elsevier
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 54 ENVIRONMENTAL SCIENCES; soil moisture; data assimilation; model structural error; data-driving; machine learning
Citation Formats
Zhang, Qiuru, Shi, Liangsheng, Holzman, Mauro, Ye, Ming, Wang, Yakun, Carmona, Facundo, and Zha, Yuanyuan. A dynamic data-driven method for dealing with model structural error in soil moisture data assimilation. United States: N. p., 2019.
Web. doi:10.1016/j.advwatres.2019.103407.
Zhang, Qiuru, Shi, Liangsheng, Holzman, Mauro, Ye, Ming, Wang, Yakun, Carmona, Facundo, & Zha, Yuanyuan. A dynamic data-driven method for dealing with model structural error in soil moisture data assimilation. United States. https://doi.org/10.1016/j.advwatres.2019.103407
Zhang, Qiuru, Shi, Liangsheng, Holzman, Mauro, Ye, Ming, Wang, Yakun, Carmona, Facundo, and Zha, Yuanyuan. Thu .
"A dynamic data-driven method for dealing with model structural error in soil moisture data assimilation". United States. https://doi.org/10.1016/j.advwatres.2019.103407. https://www.osti.gov/servlets/purl/1803818.
@article{osti_1803818,
title = {A dynamic data-driven method for dealing with model structural error in soil moisture data assimilation},
author = {Zhang, Qiuru and Shi, Liangsheng and Holzman, Mauro and Ye, Ming and Wang, Yakun and Carmona, Facundo and Zha, Yuanyuan},
abstractNote = {Attributing to the flexibility in considering various types of observation error and model error, data assimilation has been increasingly applied to dynamically improve soil moisture modeling in many hydrological practices. However, accurate characterization of model error, especially the part caused by defective model structure, presents a significant challenge to the successful implementation of data assimilation. Model structural error has received limited attention relative to parameter and input errors, mainly due to our poor understanding of structural inadequacy and the difficulties in parameterizing structural error. In this paper, we present a dynamic data-driven approach to estimate the model structural error in soil moisture data assimilation without the need for identifying error generation mechanism or specifying particular form for the error model. The error model is based on the Gaussian process regression and then integrated into the ensemble Kalman filter (EnKF) to form a hybrid method for dealing with multi-source model errors. Two variants of the hybrid method in terms of two different error correction manners are proposed. The effectiveness of the proposed method is tested through a suit of synthetic cases and a real-world case. Results demonstrate the potential of the proposed hybrid method for estimating model structural error and providing improved model predictions. Compared to the traditional EnKF without explicitly considering the model structural error, parameter compensation issue is obviously reduced and soil moisture retrieval is substantially improved.},
doi = {10.1016/j.advwatres.2019.103407},
journal = {Advances in Water Resources},
number = ,
volume = 132,
place = {United States},
year = {Thu Aug 22 00:00:00 EDT 2019},
month = {Thu Aug 22 00:00:00 EDT 2019}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 29 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Soil moisture prediction with the ensemble Kalman filter: Handling uncertainty of soil hydraulic parameters
journal, December 2017
- Brandhorst, N.; Erdal, D.; Neuweiler, I.
- Advances in Water Resources, Vol. 110
Integrating a calibrated groundwater flow model with error-correcting data-driven models to improve predictions
journal, January 2009
- Demissie, Yonas K.; Valocchi, Albert J.; Minsker, Barbara S.
- Journal of Hydrology, Vol. 364, Issue 3-4
Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy
journal, August 2005
- Hanchuan Peng, ; Ding, C.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 8
Data assimilation in the presence of forecast bias
journal, January 1998
- Dee, Dick P.; Da Silva, Arlindo M.
- Quarterly Journal of the Royal Meteorological Society, Vol. 124, Issue 545
Links Between Root Length Density Profiles and Models of the Root System Architecture
journal, October 2012
- Pagès, Loïc; Bruchou, Claude; Garré, Sarah
- Vadose Zone Journal, Vol. 11, Issue 4
Correcting for forecast bias in soil moisture assimilation with the ensemble Kalman filter: CORRECTING FOR FORECAST BIAS IN SOIL MOISTURE ASSIMILATION
journal, September 2007
- De Lannoy, Gabriëlle J. M.; Reichle, Rolf H.; Houser, Paul R.
- Water Resources Research, Vol. 43, Issue 9
Spatial and temporal characteristics of soil moisture in an intensively monitored agricultural field (OPE3)
journal, December 2006
- De Lannoy, Gabriëlle J. M.; Verhoest, Niko E. C.; Houser, Paul R.
- Journal of Hydrology, Vol. 331, Issue 3-4
Accounting for Model Errors in Ensemble Data Assimilation
journal, October 2009
- Li, Hong; Kalnay, Eugenia; Miyoshi, Takemasa
- Monthly Weather Review, Vol. 137, Issue 10
The Ensemble Kalman Filter: theoretical formulation and practical implementation
journal, November 2003
- Evensen, Geir
- Ocean Dynamics, Vol. 53, Issue 4
Using a bias aware EnKF to account for unresolved structure in an unsaturated zone model: BIAS AWARE ENKF FOR UNRESOLVED STRUCTURE
journal, January 2014
- Erdal, D.; Neuweiler, I.; Wollschläger, U.
- Water Resources Research, Vol. 50, Issue 1
Challenges of modifying root traits in crops for agriculture
journal, December 2014
- Meister, Robert; Rajani, M. S.; Ruzicka, Daniel
- Trends in Plant Science, Vol. 19, Issue 12
Comparison of ensemble-based state and parameter estimation methods for soil moisture data assimilation
journal, December 2015
- Chen, Weijing; Huang, Chunlin; Shen, Huanfeng
- Advances in Water Resources, Vol. 86
Data assimilation methods in the Earth sciences
journal, November 2008
- Reichle, Rolf H.
- Advances in Water Resources, Vol. 31, Issue 11
Toward reduction of model uncertainty: Integration of Bayesian model averaging and data assimilation: TOWARD REDUCTION OF MODEL UNCERTAINTY
journal, March 2012
- Parrish, Mark A.; Moradkhani, Hamid; DeChant, Caleb M.
- Water Resources Research, Vol. 48, Issue 3
Hydraulic parameter estimation by remotely-sensed top soil moisture observations with the particle filter
journal, March 2011
- Montzka, Carsten; Moradkhani, Hamid; Weihermüller, Lutz
- Journal of Hydrology, Vol. 399, Issue 3-4
A Bayesian approach to improved calibration and prediction of groundwater models with structural error
journal, November 2015
- Xu, Tianfang; Valocchi, Albert J.
- Water Resources Research, Vol. 51, Issue 11
A framework for dealing with uncertainty due to model structure error
journal, November 2006
- Refsgaard, Jens Christian; van der Sluijs, Jeroen P.; Brown, James
- Advances in Water Resources, Vol. 29, Issue 11
An improved variable selection method for support vector regression in NIR spectral modeling
journal, July 2018
- Xu, Shu; Lu, Bo; Baldea, Michael
- Journal of Process Control, Vol. 67
A short exploration of structural noise: A SHORT EXPLORATION OF STRUCTURAL NOISE
journal, May 2010
- Doherty, John; Welter, David
- Water Resources Research, Vol. 46, Issue 5
Data assimilation of soil water flow via ensemble Kalman filter: Infusing soil moisture data at different scales
journal, December 2017
- Zhu, Penghui; Shi, Liangsheng; Zhu, Yan
- Journal of Hydrology, Vol. 555
Treatment of bias in recursive filtering
journal, August 1969
- Friedland, B.
- IEEE Transactions on Automatic Control, Vol. 14, Issue 4
Assessing parameter, precipitation, and predictive uncertainty in a distributed hydrological model using sequential data assimilation with the particle filter
journal, October 2009
- Salamon, Peter; Feyen, Luc
- Journal of Hydrology, Vol. 376, Issue 3-4
An adaptive covariance inflation error correction algorithm for ensemble filters
journal, January 2007
- Anderson, Jeffrey L.
- Tellus A: Dynamic Meteorology and Oceanography, Vol. 59, Issue 2
Numerical Comparison of Iterative Ensemble Kalman Filters for Unsaturated Flow Inverse Modeling
journal, January 2014
- Song, Xuehang; Shi, Liangsheng; Ye, Ming
- Vadose Zone Journal, Vol. 13, Issue 2
A generalized Ross method for two- and three-dimensional variably saturated flow
journal, April 2013
- Zha, Yuanyuan; Shi, Liangsheng; Ye, Ming
- Advances in Water Resources, Vol. 54
Data assimilation for unsaturated flow models with restart adaptive probabilistic collocation based Kalman filter
journal, June 2016
- Man, Jun; Li, Weixuan; Zeng, Lingzao
- Advances in Water Resources, Vol. 92
Modeling Soil Water and Solute Transport-Fast, Simplified Numerical Solutions
journal, November 2003
- Ross, P. J.
- Agronomy Journal, Vol. 95, Issue 6
A Bayesian tutorial for data assimilation
journal, June 2007
- Wikle, Christopher K.; Berliner, L. Mark
- Physica D: Nonlinear Phenomena, Vol. 230, Issue 1-2
Determining soil moisture by assimilating soil temperature measurements using the Ensemble Kalman Filter
journal, December 2015
- Dong, Jianzhi; Steele-Dunne, Susan C.; Ochsner, Tyson E.
- Advances in Water Resources, Vol. 86
An integrated hydrologic Bayesian multimodel combination framework: Confronting input, parameter, and model structural uncertainty in hydrologic prediction: AN INTEGRATED BAYESIAN MULTIMODEL FRAMEWORK
journal, January 2007
- Ajami, Newsha K.; Duan, Qingyun; Sorooshian, Soroosh
- Water Resources Research, Vol. 43, Issue 1
Multi-source data assimilation for physically based hydrological modeling of an experimental hillslope
journal, January 2018
- Botto, Anna; Belluco, Enrica; Camporese, Matteo
- Hydrology and Earth System Sciences, Vol. 22, Issue 8
Accounting for the Error due to Unresolved Scales in Ensemble Data Assimilation: A Comparison of Different Approaches
journal, November 2005
- Hamill, Thomas M.; Whitaker, Jeffrey S.
- Monthly Weather Review, Vol. 133, Issue 11
Towards a comprehensive assessment of model structural adequacy: ASSESSMENT OF MODEL STRUCTURAL ADEQUACY
journal, August 2012
- Gupta, Hoshin V.; Clark, Martyn P.; Vrugt, Jasper A.
- Water Resources Research, Vol. 48, Issue 8
Data‐Driven Model Uncertainty Estimation in Hydrologic Data Assimilation
journal, February 2018
- Pathiraja, S.; Moradkhani, H.; Marshall, L.
- Water Resources Research, Vol. 54, Issue 2
Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter: UNCERTAINTY ASSESSMENT OF HYDROLOGIC MODEL
journal, May 2005
- Moradkhani, Hamid; Hsu, Kuo-Lin; Gupta, Hoshin
- Water Resources Research, Vol. 41, Issue 5
Treatment of uncertainty using ensemble methods: Comparison of sequential data assimilation and Bayesian model averaging: SEQUENTIAL DATA ASSIMILATION
journal, January 2007
- Vrugt, Jasper A.; Robinson, Bruce A.
- Water Resources Research, Vol. 43, Issue 1
Kalman filters for assimilating near-surface observations into the Richards equation – Part 3: Retrieving states and parameters from laboratory evaporation experiments
journal, January 2014
- Medina, H.; Romano, N.; Chirico, G. B.
- Hydrology and Earth System Sciences, Vol. 18, Issue 7
Bias aware Kalman filters: Comparison and improvements
journal, May 2006
- Drécourt, Jean-Philippe; Madsen, Henrik; Rosbjerg, Dan
- Advances in Water Resources, Vol. 29, Issue 5
One-dimensional soil moisture profile retrieval by assimilation of near-surface observations: a comparison of retrieval algorithms
journal, June 2001
- Walker, Jeffrey P.; Willgoose, Garry R.; Kalma, Jetse D.
- Advances in Water Resources, Vol. 24, Issue 6
Dual state-parameter estimation of root zone soil moisture by optimal parameter estimation and extended Kalman filter data assimilation
journal, March 2011
- Lü, Haishen; Yu, Zhongbo; Zhu, Yonghua
- Advances in Water Resources, Vol. 34, Issue 3
Correcting Biased Observation Model Error in Data Assimilation
journal, July 2017
- Berry, Tyrus; Harlim, John
- Monthly Weather Review, Vol. 145, Issue 7
Error covariance calculation for forecast bias estimation in hydrologic data assimilation
journal, December 2015
- Pauwels, Valentijn R. N.; De Lannoy, Gabriëlle J. M.
- Advances in Water Resources, Vol. 86
Dynamic modeling of predictive uncertainty by regression on absolute errors: DYNAMIC MODELING OF PREDICTIVE UNCERTAINTY
journal, March 2012
- Pianosi, F.; Raso, L.
- Water Resources Research, Vol. 48, Issue 3
Real-time groundwater flow modeling with the Ensemble Kalman Filter: Joint estimation of states and parameters and the filter inbreeding problem: REAL-TIME GROUNDWATER FLOW MODELING
journal, September 2008
- Hendricks Franssen, H. J.; Kinzelbach, W.
- Water Resources Research, Vol. 44, Issue 9
Vadose Zone Model-Data Fusion: State of the Art and Future Challenges
journal, November 2012
- Huisman, Johan A.; Vrugt, Jasper A.; Ferre, Ty P. A.
- Vadose Zone Journal, Vol. 11, Issue 4
Kalman filters for assimilating near-surface observations into the Richards equation – Part 2: A dual filter approach for simultaneous retrieval of states and parameters
journal, January 2014
- Medina, H.; Romano, N.; Chirico, G. B.
- Hydrology and Earth System Sciences, Vol. 18, Issue 7
State and bias estimation for soil moisture profiles by an ensemble Kalman filter: Effect of assimilation depth and frequency: STATE AND BIAS ESTIMATION FOR SOIL MOISTURE
journal, June 2007
- De Lannoy, Gabriëlle J. M.; Houser, Paul R.; Pauwels, Valentijn R. N.
- Water Resources Research, Vol. 43, Issue 6
An Iterative Ensemble Kalman Filter
journal, August 2011
- Lorentzen, Rolf J.; Naevdal, Geir
- IEEE Transactions on Automatic Control, Vol. 56, Issue 8
Developing joint probability distributions of soil water retention characteristics
journal, May 1988
- Carsel, Robert F.; Parrish, Rudolph S.
- Water Resources Research, Vol. 24, Issue 5
A simulation model of water dynamics in winter wheat field and its application in a semiarid region
journal, July 2001
- Kang, Shaozhong; Zhang, Fucang; Zhang, Jianhua
- Agricultural Water Management, Vol. 49, Issue 2
Model Error Estimation Employing an Ensemble Data Assimilation Approach
journal, May 2006
- Zupanski, Dusanka; Zupanski, Milija
- Monthly Weather Review, Vol. 134, Issue 5
Soil water uptake and root distribution of different perennial and annual bioenergy crops
journal, November 2014
- Ferchaud, Fabien; Vitte, Guillaume; Bornet, Frédéric
- Plant and Soil, Vol. 388, Issue 1-2
Estimating Model Parameters with Ensemble-Based Data Assimilation: A Review
journal, January 2013
- Ruiz, Juan Jose; Pulido, Manuel; Miyoshi, Takemasa
- Journal of the Meteorological Society of Japan. Ser. II, Vol. 91, Issue 2
An improved approach for estimating observation and model error parameters in soil moisture data assimilation: ESTIMATING DATA ASSIMILATION ERROR PARAMETERS
journal, December 2010
- Crow, W. T.; van den Berg, M. J.
- Water Resources Research, Vol. 46, Issue 12
Richards Equation-Based Modeling to Estimate Flow and Nitrate Transport in a Deep Alluvial Vadose Zone
journal, November 2012
- Botros, Farag E.; Onsoy, Yuksel S.; Ginn, Timothy R.
- Vadose Zone Journal, Vol. 11, Issue 4
Impacts of different types of measurements on estimating unsaturated flow parameters
journal, May 2015
- Shi, Liangsheng; Song, Xuehang; Tong, Juxiu
- Journal of Hydrology, Vol. 524
Uncertainty in hydrologic modeling: Toward an integrated data assimilation framework: HYDROLOGIC DATA ASSIMILATION
journal, July 2007
- Liu, Yuqiong; Gupta, Hoshin V.
- Water Resources Research, Vol. 43, Issue 7
A Closed-form Equation for Predicting the Hydraulic Conductivity of Unsaturated Soils1
journal, January 1980
- van Genuchten, M. Th.
- Soil Science Society of America Journal, Vol. 44, Issue 5
Improved treatment of uncertainty in hydrologic modeling: Combining the strengths of global optimization and data assimilation: TREATMENT OF UNCERTAINTY IN HYDROLOGIC MODELING
journal, January 2005
- Vrugt, Jasper A.; Diks, Cees G. H.; Gupta, Hoshin V.
- Water Resources Research, Vol. 41, Issue 1
Bias and data assimilation
journal, October 2005
- Dee, D. P.
- Quarterly Journal of the Royal Meteorological Society, Vol. 131, Issue 613
An Information-Theoretic Framework for Improving Imperfect Dynamical Predictions Via Multi-Model Ensemble Forecasts
journal, March 2015
- Branicki, Michal; Majda, Andrew J.
- Journal of Nonlinear Science, Vol. 25, Issue 3
A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts
journal, December 1999
- Anderson, Jeffrey L.; Anderson, Stephen L.
- Monthly Weather Review, Vol. 127, Issue 12
Modeling Root Water Uptake in Hydrological and Climate Models
journal, December 2001
- Feddes, Reinder A.; Hoff, Holger; Bruen, Michael
- Bulletin of the American Meteorological Society, Vol. 82, Issue 12
Works referencing / citing this record:
Surrogate-Based Bayesian Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Approximation Error
text, January 2018
- Zhang, Jiangjiang; Zheng, Qiang; Chen, Dingjiang
- arXiv