Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation

doi:https://doi.org/10.1016/j.jcp.2009.09.022

Title: Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation

Full Record
Other Related Research

Abstract

The filtering and predictive skill for turbulent signals is often limited by the lack of information about the true dynamics of the system and by our inability to resolve the assumed dynamics with sufficiently high resolution using the current computing power. The standard approach is to use a simple yet rich family of constant parameters to account for model errors through parameterization. This approach can have significant skill by fitting the parameters to some statistical feature of the true signal; however in the context of real-time prediction, such a strategy performs poorly when intermittent transitions to instability occur. Alternatively, we need a set of dynamic parameters. One strategy for estimating parameters on the fly is a stochastic parameter estimation through partial observations of the true signal. In this paper, we extend our newly developed stochastic parameter estimation strategy, the Stochastic Parameterization Extended Kalman Filter (SPEKF), to filtering sparsely observed spatially extended turbulent systems which exhibit abrupt stability transition from time to time despite a stable average behavior. For our primary numerical example, we consider a turbulent system of externally forced barotropic Rossby waves with instability introduced through intermittent negative damping. We find high filtering skill of SPEKF applied to thismore »« less

Authors:

Gershgorin, B ^[1]; Majda, A J ^[1]

Department of Mathematics and Center for Atmosphere and Ocean Science, Courant Institute of Mathematical Sciences, New York University, NY 10012 (United States)

Publication Date:: 2010-01-01

OSTI Identifier:: 21333900

Resource Type:: Journal Article

Journal Name:: Journal of Computational Physics

Additional Journal Information:: Journal Volume: 229; Journal Issue: 1; Other Information: DOI: 10.1016/j.jcp.2009.09.022; PII: S0021-9991(09)00514-2; Copyright (c) 2009 Elsevier Science B.V., Amsterdam, The Netherlands, All rights reserved; Country of input: International Atomic Energy Agency (IAEA); Journal ID: ISSN 0021-9991

Country of Publication:: United States

Language:: English

Subject:: 71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSICS; CORRECTIONS; ERRORS; FORECASTING; INSTABILITY; SIGNALS; STOCHASTIC PROCESSES; TURBULENCE

Citation Formats


                    Gershgorin, B, Harlim, J., and Majda, A J. Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation.  United States: N. p., 2010. 
        Web.  doi:10.1016/j.jcp.2009.09.022.

Copy to clipboard


                    Gershgorin, B, Harlim, J., & Majda, A J. Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation.  United States.  https://doi.org/10.1016/j.jcp.2009.09.022

Copy to clipboard


                    Gershgorin, B, Harlim, J., and Majda, A J. Fri .  
        "Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation".  United States.  https://doi.org/10.1016/j.jcp.2009.09.022.

Copy to clipboard


                    
@article{osti_21333900,

  title        = {Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation},

  author       = {Gershgorin, B and Harlim, J. and Majda, A J},

  abstractNote = {The filtering and predictive skill for turbulent signals is often limited by the lack of information about the true dynamics of the system and by our inability to resolve the assumed dynamics with sufficiently high resolution using the current computing power. The standard approach is to use a simple yet rich family of constant parameters to account for model errors through parameterization. This approach can have significant skill by fitting the parameters to some statistical feature of the true signal; however in the context of real-time prediction, such a strategy performs poorly when intermittent transitions to instability occur. Alternatively, we need a set of dynamic parameters. One strategy for estimating parameters on the fly is a stochastic parameter estimation through partial observations of the true signal. In this paper, we extend our newly developed stochastic parameter estimation strategy, the Stochastic Parameterization Extended Kalman Filter (SPEKF), to filtering sparsely observed spatially extended turbulent systems which exhibit abrupt stability transition from time to time despite a stable average behavior. For our primary numerical example, we consider a turbulent system of externally forced barotropic Rossby waves with instability introduced through intermittent negative damping. We find high filtering skill of SPEKF applied to this toy model even in the case of very sparse observations (with only 15 out of the 105 grid points observed) and with unspecified external forcing and damping. Additive and multiplicative bias corrections are used to learn the unknown features of the true dynamics from observations. We also present a comprehensive study of predictive skill in the one-mode context including the robustness toward variation of stochastic parameters, imperfect initial conditions and finite ensemble effect. Furthermore, the proposed stochastic parameter estimation scheme applied to the same spatially extended Rossby wave system demonstrates high predictive skill, comparable with the skill of the perfect model for a duration of many eddy turnover times especially in the unstable regime.},

  doi          = {10.1016/j.jcp.2009.09.022},

  url          = {https://www.osti.gov/biblio/21333900},
  journal      = {Journal of Computational Physics},

  issn         = {0021-9991},

  number       = 1,

  volume       = 229,

  place        = {United States},

  year         = {2010},

  month        = {1}

}

Copy to clipboard

Journal Article:

https://doi.org/10.1016/j.jcp.2009.09.022

Other availability

Find in Google Scholar

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Dynamic Stochastic Superresolution of sparsely observed turbulent systems

Journal Article Branicki, M., E-mail: branicki@cims.nyu.edu ; Majda, A. J. - Journal of Computational Physics

Real-time capture of the relevant features of the unresolved turbulent dynamics of complex natural systems from sparse noisy observations and imperfect models is a notoriously difficult problem. The resulting lack of observational resolution and statistical accuracy in estimating the important turbulent processes, which intermittently send significant energy to the large-scale fluctuations, hinders efficient parameterization and real-time prediction using discretized PDE models. This issue is particularly subtle and important when dealing with turbulent geophysical systems with an vast range of interacting spatio-temporal scales and rough energy spectra near the mesh scale of numerical models. Here, we introduce and study a suitemore »« less
https://doi.org/10.1016/J.JCP.2012.11.037
Mathematical strategies for filtering complex systems: Regularly spaced sparse observations

Journal Article Harlim, J. ; Majda, A J - Journal of Computational Physics

Real time filtering of noisy turbulent signals through sparse observations on a regularly spaced mesh is a notoriously difficult and important prototype filtering problem. Simpler off-line test criteria are proposed here as guidelines for filter performance for these stiff multi-scale filtering problems in the context of linear stochastic partial differential equations with turbulent solutions. Filtering turbulent solutions of the stochastically forced dissipative advection equation through sparse observations is developed as a stringent test bed for filter performance with sparse regular observations. The standard ensemble transform Kalman filter (ETKF) has poor skill on the test bed and even suffers from filtermore »« less
https://doi.org/10.1016/j.jcp.2008.01.049
Test models for improving filtering with model errors through stochastic parameter estimation

Journal Article Gershgorin, B ; Harlim, J. ; Majda, A J - Journal of Computational Physics

The filtering skill for turbulent signals from nature is often limited by model errors created by utilizing an imperfect model for filtering. Updating the parameters in the imperfect model through stochastic parameter estimation is one way to increase filtering skill and model performance. Here a suite of stringent test models for filtering with stochastic parameter estimation is developed based on the Stochastic Parameterization Extended Kalman Filter (SPEKF). These new SPEKF-algorithms systematically correct both multiplicative and additive biases and involve exact formulas for propagating the mean and covariance including the parameters in the test model. A comprehensive study is presented ofmore »« less
https://doi.org/10.1016/j.jcp.2009.08.019
Ensemble Kalman filters for dynamical systems with unresolved turbulence

Journal Article Grooms, Ian ; Lee, Yoonsang ; Majda, Andrew J. ; ... - Journal of Computational Physics

Ensemble Kalman filters are developed for turbulent dynamical systems where the forecast model does not resolve all the active scales of motion. Coarse-resolution models are intended to predict the large-scale part of the true dynamics, but observations invariably include contributions from both the resolved large scales and the unresolved small scales. The error due to the contribution of unresolved scales to the observations, called ‘representation’ or ‘representativeness’ error, is often included as part of the observation error, in addition to the raw measurement error, when estimating the large-scale part of the system. It is here shown how stochastic superparameterization (amore »« less
https://doi.org/10.1016/J.JCP.2014.05.037
Filtering a statistically exactly solvable test model for turbulent tracers from partial observations

Journal Article Gershgorin, B ; Majda, A J - Journal of Computational Physics

A statistically exactly solvable model for passive tracers is introduced as a test model for the authors' Nonlinear Extended Kalman Filter (NEKF) as well as other filtering algorithms. The model involves a Gaussian velocity field and a passive tracer governed by the advection-diffusion equation with an imposed mean gradient. The model has direct relevance to engineering problems such as the spread of pollutants in the air or contaminants in the water as well as climate change problems concerning the transport of greenhouse gases such as carbon dioxide with strongly intermittent probability distributions consistent with the actual observations of the atmosphere.more »« less
https://doi.org/10.1016/j.jcp.2010.11.024

Similar Records