Skip to main content
U.S. Department of Energy
Office of Scientific and Technical Information

A Novel Modeling Framework for Computationally Efficient and Accurate Real-Time Ensemble Flood Forecasting With Uncertainty Quantification

Journal Article · · Water Resources Research
DOI:https://doi.org/10.1029/2019wr025727· OSTI ID:1770783

A novel modeling framework that simultaneously improves accuracy, predictability, and computational efficiency is presented. It embraces the benefits of three modeling techniques integrated together for the first time: surrogate modeling, parameter inference, and data assimilation. The use of polynomial chaos expansion (PCE) surrogates significantly decreases computational time. Parameter inference allows for model faster convergence, reduced uncertainty, and superior accuracy of simulated results. Ensemble Kalman filters assimilate errors that occur during forecasting. To examine the applicability and effectiveness of the integrated framework, we developed 18 approaches according to how surrogate models are constructed, what type of parameter distributions are used as model inputs, and whether model parameters are updated during the data assimilation procedure. We conclude that (1) PCE must be built over various forcing and flow conditions, and in contrast to previous studies, it does not need to be rebuilt at each time step; (2) model parameter specification that relies on constrained, posterior information of parameters (so-called Selected specification) can significantly improve forecasting performance and reduce uncertainty bounds compared to Random specification using prior information of parameters; and (3) no substantial differences in results exist between single and dual ensemble Kalman filters, but the latter better simulates flood peaks. The use of PCE effectively compensates for the computational load added by the parameter inference and data assimilation (up to ~80 times faster). Therefore, the presented approach contributes to a shift in modeling paradigm arguing that complex, high-fidelity hydrologic and hydraulic models should be increasingly adopted for real-time and ensemble flood forecasting.

Research Organization:
Sandia National Laboratories (SNL-CA), Livermore, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); Ministry of Environment; National Research Foundation of Korea (NRF); Ministry of Science, ICT and Future Planning (MSIT); USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF)
Grant/Contract Number:
AC04-94AL85000; NA0003525
OSTI ID:
1770783
Report Number(s):
SAND--2021-2531J; 694484
Journal Information:
Water Resources Research, Journal Name: Water Resources Research Journal Issue: 3 Vol. 56; ISSN 0043-1397
Publisher:
American Geophysical Union (AGU)Copyright Statement
Country of Publication:
United States
Language:
English

References (101)

A partitioned update scheme for state-parameter estimation of distributed hydrologic models based on the ensemble Kalman filter: A Novel Scheme for State-Parameter Estimation journal November 2013
On the nonuniqueness of sediment yield at the catchment scale: The effects of soil antecedent conditions and surface shield: ON THE NONUNIQUENESS OF SEDIMENT YIELD journal February 2014
Internal variability and model uncertainty components in future hydrometeorological projections: The Alpine Durance basin journal April 2014
Systematic assessment of the uncertainty in integrated surface water-groundwater modeling based on the probabilistic collocation method journal July 2014
Abiotic and biotic controls of soil moisture spatiotemporal variability and the occurrence of hysteresis: Abiotic and biotic controls of soil moisture variability journal May 2015
Soil erosion assessment-Mind the gap: Soil Erosion Assessment-Mind the Gap journal December 2016
Improving Robustness of Hydrologic Ensemble Predictions Through Probabilistic Pre‐ and Post‐Processing in Sequential Data Assimilation journal March 2018
The future of distributed models: Model calibration and uncertainty prediction journal July 1992
Role and treatment of uncertainty in real-time flood forecasting journal October 2004
Methods for uncertainty assessment of climate models and model predictions over East Asia: UNCERTAINTY ASSESSMENT OF CLIMATE MODELS journal March 2013
A Stochastic Projection Method for Fluid Flow journal September 2002
General Review of Rainfall-Runoff Modeling: Model Calibration, Data Assimilation, and Uncertainty Analysis book January 2008
Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling? journal October 2008
A PCM-based stochastic hydrological model for uncertainty quantification in watershed systems journal October 2014
Quantifying climate internal variability using an hourly ensemble generator over South Korea journal October 2018
Quantification of predictive uncertainty with a metamodel: toward more efficient hydrologic simulations journal July 2019
Changing ideas in hydrology — The case of physically-based models journal January 1989
Automatic calibration of a conceptual rainfall–runoff model using multiple objectives journal August 2000
Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology journal August 2001
Non-uniqueness of inverse transmissivity field calibration and predictive transport modeling journal October 2003
Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates journal February 2001
Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems journal July 2003
Dual state–parameter estimation of hydrological models using ensemble Kalman filter journal February 2005
Generalized likelihood uncertainty estimation (GLUE) using adaptive Markov Chain Monte Carlo sampling journal April 2008
Hydrological data assimilation with the ensemble Kalman filter: Use of streamflow observations to update states in a distributed hydrological model journal October 2008
Data assimilation for distributed hydrological catchment modeling via ensemble Kalman filter journal June 2010
Hydrological data assimilation with the Ensemble Square-Root-Filter: Use of streamflow observations to update model states for real-time flash flood forecasting journal September 2013
Dual states estimation of a subsurface flow-transport coupled model using ensemble Kalman filtering journal October 2013
Polynomial Chaos expansion for subsurface flows with uncertain soil parameters journal December 2013
Uncertainty based modeling of rainfall-runoff: Combined differential evolution adaptive Metropolis (DREAM) and K-means clustering journal September 2015
On the non-uniqueness of the hydro-geomorphic responses in a zero-order catchment with respect to soil moisture journal June 2016
Assessing the relative importance of parameter and forcing uncertainty and their interactions in conceptual hydrological model simulations journal November 2016
Enhancing hydrologic data assimilation by evolutionary Particle Filter and Markov Chain Monte Carlo journal January 2018
Streamflow, stomata, and soil pits: Sources of inference for complex models with fast, robust uncertainty quantification journal March 2019
Assessment of a stochastic interpolation based parameter sampling scheme for efficient uncertainty analyses of hydrologic models journal May 2005
Sparse polynomial chaos expansions and adaptive stochastic finite elements using a regression approach journal June 2008
Markov chain Monte Carlo simulation using the DREAM software package: Theory, concepts, and MATLAB implementation journal January 2016
Parameter uncertainty and temporal dynamics of sensitivity for hydrologic models: A hybrid sequential data assimilation and probabilistic collocation method journal December 2016
Adaptive sparse polynomial chaos expansion based on least angle regression journal March 2011
Application of the coupled MIKE SHE/MIKE 11 modelling system to a lowland wet grassland in southeast England journal June 2004
An evaluation of the impact of model structure on hydrological modelling uncertainty for streamflow simulation journal October 2004
Adaptive state updating in real-time river flow forecasting—a combined filtering and error forecasting procedure journal July 2005
A manifesto for the equifinality thesis journal March 2006
Multi-period and multi-criteria model conditioning to reduce prediction uncertainty in an application of TOPMODEL within the GLUE framework journal January 2007
Parameter and modeling uncertainty simulated by GLUE and a formal Bayesian method for a conceptual hydrological model journal March 2010
Developing an integrated hydrograph separation and lumped modelling approach to quantifying hydrological pathways in Irish river catchments journal April 2013
Toward a reliable prediction of seasonal forecast uncertainty: Addressing model and initial condition uncertainty with ensemble data assimilation and Sequential Bayesian Combination journal November 2014
An integrated error parameter estimation and lag-aware data assimilation scheme for real-time flood forecasting journal November 2014
A holistic, multi-scale dynamic downscaling framework for climate impact assessments and challenges of addressing finer-scale watershed dynamics journal March 2015
A polynomial chaos ensemble hydrologic prediction system for efficient parameter inference and robust uncertainty assessment journal November 2015
Towards robust quantification and reduction of uncertainty in hydrologic predictions: Integration of particle Markov chain Monte Carlo and factorial polynomial chaos expansion journal May 2017
A back-fitting algorithm to improve real-time flood forecasting journal July 2018
Chaospy: An open source tool for designing methods of uncertainty quantification journal November 2015
An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis journal April 2010
Global sensitivity analysis using polynomial chaos expansions journal July 2008
Polynomial chaos expansion for sensitivity analysis journal July 2009
Monte Carlo and quasi-Monte Carlo methods journal January 1998
Convergence assessment of numerical Monte Carlo simulations in groundwater hydrology: TECHNICAL NOTE journal April 2004
Improved treatment of uncertainty in hydrologic modeling: Combining the strengths of global optimization and data assimilation: TREATMENT OF UNCERTAINTY IN HYDROLOGIC MODELING journal January 2005
Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter: UNCERTAINTY ASSESSMENT OF HYDROLOGIC MODEL journal May 2005
Climate model fidelity and projections of climate change journal January 2006
Particle filtering and ensemble Kalman filtering for state updating with hydrological conceptual rainfall-runoff models: PARTICLE AND ENSEMBLE KALMAN FILTERING journal September 2006
Formulation of capillary hysteresis with internal state variables: FORMULATION OF CAPILLARY HYSTERESIS journal July 2006
An integrated hydrologic Bayesian multimodel combination framework: Confronting input, parameter, and model structural uncertainty in hydrologic prediction: AN INTEGRATED BAYESIAN MULTIMODEL FRAMEWORK journal January 2007
Advancing the identification and evaluation of distributed rainfall-runoff models using global sensitivity analysis: DISTRIBUTED MODEL SENSITIVITY ANALYSIS journal June 2007
A spatial regularization approach to parameter estimation for a distributed watershed model: REGULARIZATION FOR A PARAMETER ESTIMATION journal December 2008
State and parameter estimation of hydrologic models using the constrained ensemble Kalman filter: CONSTRAINED ENSEMBLE KALMAN FILTER journal November 2009
Hysteresis of soil moisture spatial heterogeneity and the “homogenizing” effect of vegetation: SOIL MOISTURE SPATIAL HETEROGENEITY journal September 2010
Budget study of the internal variability in ensemble simulations of the Canadian Regional Climate Model at the seasonal scale journal January 2011
Examining the effectiveness and robustness of sequential data assimilation methods for quantification of uncertainty in hydrologic forecasting: ROBUSTNESS OF DA TO ESTIMATE UNCERTAINTY journal April 2012
Uncertainty in flood forecasting: A distributed modeling approach in a sparse data catchment: UNCERTAINTY IN FLOOD FORECASTING IN A SPARSE DATA CATCHMENT journal September 2012
On the basin-scale detection and attribution of human-induced climate change in monsoon precipitation and streamflow: DETECTION AND ATTRIBUTION OF HYDROLOGIC CHANGE journal October 2012
Evolution of ensemble data assimilation for uncertainty quantification using the particle filter-Markov chain Monte Carlo method: EVOLUTION OF ENSEMBLE DATA ASSIMILATION USING PF-MCMC journal December 2012
Uncertainty Quantification for Subsurface Flow and Transport: Coping With Nonlinearity/Irregularity via Polynomial Chaos Surrogate and Machine Learning journal October 2018
Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics journal January 1994
Real-time forecasting with a conceptual hydrologic model: 1. Analysis of uncertainty journal December 1980
Environmental stochasticity controls soil erosion variability journal March 2016
Stochastic Finite-Element Analysis of Seismic Soil–Structure Interaction journal January 2002
Advances in real–time flood forecasting
  • Young, Peter C.
  • Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, Vol. 360, Issue 1796 https://doi.org/10.1098/rsta.2002.1008
journal May 2002
A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking journal January 2002
Sensitivity Analysis for Importance Assessment journal June 2002
The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations journal January 2002
Measuring Dynamical Prediction Utility Using Relative Entropy journal July 2002
Verification of Forecasts Expressed in Terms of Probability journal January 1950
Analysis Scheme in the Ensemble Kalman Filter journal June 1998
Ensemble Data Assimilation without Perturbed Observations journal July 2002
Review of the Ensemble Kalman Filter for Atmospheric Data Assimilation journal December 2016
Surface Data Assimilation Using an Ensemble Kalman Filter Approach with Initial Condition and Model Physics Uncertainties journal May 2007
On Information and Sufficiency journal March 1951
Dimensionality Reduction for Complex Models via Bayesian Compressive Sensing journal January 2014
Using phase-state modelling for inferring forecasting uncertainty in nonlinear stochastic decision schemes journal October 1999
Numerical Simulation of the Rainfall-Runoff Process on a Daily Basis journal June 1973
Flood Hazard Impact Analysis in the Downstream of vu Gia-Thu bon River System, Quang nam Province, Central Vietnam journal January 2015
A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code journal May 1979
The Homogeneous Chaos journal October 1938
Stochastic finite element: a non intrusive approach by regression journal January 2006
Analysis of parameter uncertainty in hydrological and sediment modeling using GLUE method: a case study of SWAT model applied to Three Gorges Reservoir Region, China journal January 2012
Advancing data assimilation in operational hydrologic forecasting: progresses, challenges, and emerging opportunities journal January 2012
An integrated uncertainty and ensemble-based data assimilation approach for improved operational streamflow predictions journal January 2012
State and parameter estimation of two land surface models using the ensemble Kalman filter and the particle filter journal January 2017
Flood frequency estimation by continuous simulation (with likelihood based uncertainty estimation) journal January 2000