Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows

Pawar, Suraj; Ahmed, Shady E.; San, Omer; Rasheed, Adil; Navon, Ionel M.

doi:10.1063/5.0012853

Title: Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows

Journal Article · Wed Jul 15 00:00:00 EDT 2020 · Physics of Fluids

DOI:https://doi.org/10.1063/5.0012853· OSTI ID:1737505

^[1];

^[2];

^[3]

Oklahoma State Univ., Stillwater, OK (United States). School of Mechanical and Aerospace Engineering
Norwegian Univ. of Science and Technology, Trondheim (Norway)
Florida State Univ., Tallahassee, FL (United States)

Reduced rank nonlinear filters are increasingly utilized in data assimilation of geophysical flows, but often require a set of ensemble forward simulations to estimate forecast covariance. On the other hand, predictor-corrector type nudging approaches are still attractive due to their simplicity of implementation when more complex methods need to be avoided. However, optimal estimate of nudging gain matrix might be cumbersome. In this paper, we put forth a fully nonintrusive recurrent neural network approach based on a long short-term memory (LSTM) embedding architecture to estimate the nudging term, which plays a role not only to force the state trajectories to the observations but also acts as a stabilizer. Furthermore, our approach relies on the power of archival data and the trained model can be retrained effectively due to power of transfer learning in any neural network applications. In order to verify the feasibility of the proposed approach, we perform twin experiments using Lorenz 96 system. Our results demonstrate that the proposed LSTM nudging approach yields more accurate estimates than both extended Kalman filter (EKF) and ensemble Kalman filter (EnKF) when only sparse observations are available. With the availability of emerging AI-friendly and modular hardware technologies and heterogeneous computing platforms, we articulate that our simplistic nudging framework turns out to be computationally more efficient than either the EKF or EnKF approaches.

View Accepted Manuscript (DOE)

View Accepted Manuscript (Publisher)

Cite

Export

Save

Research Organization:: Oklahoma State Univ., Stillwater, OK (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: SC0019290

OSTI ID:: 1737505

Alternate ID(s):: OSTI ID: 1638535

Journal Information:: Physics of Fluids, Vol. 32, Issue 7; ISSN 1070-6631

Publisher:: American Institute of Physics (AIP)Copyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 29 works

Citation information provided by
Web of Science

References (68)

Data-driven recovery of hidden physics in reduced order modeling of fluid flows Pawar, Suraj; Ahmed, Shady E.; San, Omer Physics of Fluids, Vol. 32, Issue 3 https://doi.org/10.1063/5.0002051	journal	March 2020
Data-assisted reduced-order modeling of extreme events in complex dynamical systems Wan, Zhong Yi; Vlachas, Pantelis; Koumoutsakos, Petros PLOS ONE, Vol. 13, Issue 5 https://doi.org/10.1371/journal.pone.0197704	journal	May 2018
Model error correction in data assimilation by integrating neural networks Zhu, Jiangcheng; Hu, Shuang; Arcucci, Rossella Big Data Mining and Analytics, Vol. 2, Issue 2 https://doi.org/10.26599/bdma.2018.9020033	journal	June 2019
Data-Driven Crowd Simulation with Generative Adversarial Networks Amirian, Javad; van Toll, Wouter; Hayet, Jean-Bernard CASA '19: Computer Animation and Social Agents, Proceedings of the 32nd International Conference on Computer Animation and Social Agents https://doi.org/10.1145/3328756.3328769	conference	July 2019
Cascades and statistical equilibrium in shell models of turbulence Ditlevsen, P. D.; Mogensen, I. A. Physical Review E, Vol. 53, Issue 5 https://doi.org/10.1103/physreve.53.4785	journal	May 1996
Optimal Spectral Nudging for Global Dynamic Downscaling Schubert-Frisius, Martina; Feser, Frauke; von Storch, Hans Monthly Weather Review, Vol. 145, Issue 3 https://doi.org/10.1175/mwr-d-16-0036.1	journal	March 2017
A comparative study of 4D-VAR and a 4D Ensemble Kalman Filter: perfect model simulations with Lorenz-96 Fertig, Elana J.; Harlim, John; Hunt, Brian R. Tellus A: Dynamic Meteorology and Oceanography, Vol. 59, Issue 1 https://doi.org/10.1111/j.1600-0870.2006.00205.x	journal	January 2007
Error growth and estimates of predictability from the ECMWF forecasting system Simmons, A. J.; Mureau, R.; Petroliagis, T. Quarterly Journal of the Royal Meteorological Society, Vol. 121, Issue 527 https://doi.org/10.1002/qj.49712152711	journal	October 1995
Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks Vlachas, Pantelis R.; Byeon, Wonmin; Wan, Zhong Y. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2213 https://doi.org/10.1098/rspa.2017.0844	journal	May 2018
A deterministic formulation of the ensemble Kalman filter: an alternative to ensemble square root filters Sakov, Pavel; Oke, Peter R. Tellus A: Dynamic Meteorology and Oceanography, Vol. 60, Issue 2 https://doi.org/10.1111/j.1600-0870.2007.00299.x	journal	January 2008
Optimal reduced space for Variational Data Assimilation Arcucci, Rossella; Mottet, Laetitia; Pain, Christopher Journal of Computational Physics, Vol. 379 https://doi.org/10.1016/j.jcp.2018.10.042	journal	February 2019
Physical initialization for numerical weather prediction over the tropics Krishnamurti, T. N.; Xue, Jishan; Bedi, H. S. Tellus B: Chemical and Physical Meteorology, Vol. 43, Issue 4 https://doi.org/10.3402/tellusb.v43i4.15398	journal	January 1991
Continuous Data Assimilation Using General Interpolant Observables Azouani, Abderrahim; Olson, Eric; Titi, Edriss S. Journal of Nonlinear Science, Vol. 24, Issue 2 https://doi.org/10.1007/s00332-013-9189-y	journal	November 2013
Data-driven modelling of nonlinear spatio-temporal fluid flows using a deep convolutional generative adversarial network Cheng, M.; Fang, F.; Pain, C. C. Computer Methods in Applied Mechanics and Engineering, Vol. 365 https://doi.org/10.1016/j.cma.2020.113000	journal	June 2020
Efficiency of a POD-based reduced second-order adjoint model in 4D-Var data assimilation Daescu, D. N.; Navon, I. M. International Journal for Numerical Methods in Fluids, Vol. 53, Issue 6 https://doi.org/10.1002/fld.1316	journal	January 2007
Determination of optimal nudging coefficients Vidard, P. A.; Le Dimet, F. -X.; Piacentini, A. Tellus A, Vol. 55, Issue 1 https://doi.org/10.1034/j.1600-0870.2003.201317.x	journal	January 2003
A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems Tang, Meng; Liu, Yimin; Durlofsky, Louis J. Journal of Computational Physics, Vol. 413 https://doi.org/10.1016/j.jcp.2020.109456	journal	July 2020
A nudging-based data assimilation method: the Back and Forth Nudging (BFN) algorithm Auroux, D.; Blum, J. Nonlinear Processes in Geophysics, Vol. 15, Issue 2 https://doi.org/10.5194/npg-15-305-2008	journal	January 2008
Rapid spatio-temporal flood prediction and uncertainty quantification using a deep learning method Hu, R.; Fang, F.; Pain, C. C. Journal of Hydrology, Vol. 575 https://doi.org/10.1016/j.jhydrol.2019.05.087	journal	August 2019
A reduced-order strategy for 4D-Var data assimilation Robert, C.; Durbiano, S.; Blayo, E. Journal of Marine Systems, Vol. 57, Issue 1-2 https://doi.org/10.1016/j.jmarsys.2005.04.003	journal	August 2005
Dynamic Data Assimilation Lewis, John M.; Lakshmivarahan, S.; Dhall, Sudarshan Cambridge University Press https://doi.org/10.1017/CBO9780511526480	book	January 2006
Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets Pérez-Ortiz, Juan Antonio; Gers, Felix A.; Eck, Douglas Neural Networks, Vol. 16, Issue 2 https://doi.org/10.1016/s0893-6080(02)00219-8	journal	March 2003
Local calibration of remotely sensed rainfall from the TRMM satellite for different periods and spatial scales in the Indus Basin Cheema, Muhammad Jehanzeb Masud; Bastiaanssen, Wim G. M. International Journal of Remote Sensing, Vol. 33, Issue 8 https://doi.org/10.1080/01431161.2011.617397	journal	September 2011
Comparison of sequential data assimilation methods for the Kuramoto-Sivashinsky equation Jardak, M.; Navon, I. M.; Zupanski, M. International Journal for Numerical Methods in Fluids https://doi.org/10.1002/fld.2020	journal	January 2009
Long Short-Term Memory Hochreiter, Sepp; Schmidhuber, Jürgen Neural Computation, Vol. 9, Issue 8 https://doi.org/10.1162/neco.1997.9.8.1735	journal	November 1997
A reduced-order approach to four-dimensional variational data assimilation using proper orthogonal decomposition Cao, Yanhua; Zhu, Jiang; Navon, I. M. International Journal for Numerical Methods in Fluids, Vol. 53, Issue 10 https://doi.org/10.1002/fld.1365	journal	January 2007
Parameterised non-intrusive reduced order methods for ensemble Kalman filter data assimilation Xiao, D.; Du, J.; Fang, F. Computers & Fluids, Vol. 177 https://doi.org/10.1016/j.compfluid.2018.10.006	journal	November 2018
A local ensemble Kalman filter for atmospheric data assimilation Ott, Edward; Hunt, Brian R.; Szunyogh, Istvan Tellus A: Dynamic Meteorology and Oceanography, Vol. 56, Issue 5 https://doi.org/10.3402/tellusa.v56i5.14462	journal	January 2004
Data Assimilation Evensen, Geir https://doi.org/10.1007/978-3-642-03711-5	book	January 2009
The structure of energy conserving low-order models Gluhovsky, Alexander; Tong, Christopher Physics of Fluids, Vol. 11, Issue 2 https://doi.org/10.1063/1.869883	journal	February 1999
pyROM: A computational framework for reduced order modeling Puzyrev, Vladimir; Ghommem, Mehdi; Meka, Shiv Journal of Computational Science, Vol. 30 https://doi.org/10.1016/j.jocs.2018.12.004	journal	January 2019
Data assimilation as a learning tool to infer ordinary differential equation representations of dynamical models Bocquet, Marc; Brajard, Julien; Carrassi, Alberto Nonlinear Processes in Geophysics, Vol. 26, Issue 3 https://doi.org/10.5194/npg-26-143-2019	journal	January 2019
An Optimal Nudging Data Assimilation Scheme Using Parameter Estimation Zou, X.; Navon, I. M.; Ledimet, F. X. Quarterly Journal of the Royal Meteorological Society, Vol. 118, Issue 508 https://doi.org/10.1002/qj.49711850808	journal	October 1992
Spectral nudging in a spectral regional climate model Radu, Raluca; Déqué, Michel; Somot, Samuel Tellus A: Dynamic Meteorology and Oceanography, Vol. 60, Issue 5 https://doi.org/10.1111/j.1600-0870.2008.00341.x	journal	January 2008
Use of Four-Dimensional Data Assimilation in a Limited-Area Mesoscale Model Part II: Effects of Data Assimilation within the Planetary Boundary Layer Stauffer, David R.; Seaman, Nelson L.; Binkowski, Francis S. Monthly Weather Review, Vol. 119, Issue 3 https://doi.org/10.1175/1520-0493(1991)119<0734:uofdda>2.0.co;2	journal	March 1991
Inferring flow parameters and turbulent configuration with physics-informed data assimilation and spectral nudging Clark Di Leoni, Patricio; Mazzino, Andrea; Biferale, Luca Physical Review Fluids, Vol. 3, Issue 10 https://doi.org/10.1103/physrevfluids.3.104604	journal	October 2018
Data-driven reduced order model with temporal convolutional neural network Wu, Pin; Sun, Junwu; Chang, Xuting Computer Methods in Applied Mechanics and Engineering, Vol. 360 https://doi.org/10.1016/j.cma.2019.112766	journal	March 2020
Hidden physics models: Machine learning of nonlinear partial differential equations Raissi, Maziar; Karniadakis, George Em Journal of Computational Physics, Vol. 357 https://doi.org/10.1016/j.jcp.2017.11.039	journal	March 2018
Sampling the posterior: An approach to non-Gaussian data assimilation Apte, A.; Hairer, M.; Stuart, A. M. Physica D: Nonlinear Phenomena, Vol. 230, Issue 1-2 https://doi.org/10.1016/j.physd.2006.06.009	journal	June 2007
Data Assimilation and Initialization of Hurricane Prediction Models Anthes, Richard A. Journal of the Atmospheric Sciences, Vol. 31, Issue 3 https://doi.org/10.1175/1520-0469(1974)031<0702:daaioh>2.0.co;2	journal	April 1974
Estimating the state of a geophysical system with sparse observations: time delay methods to achieve accurate initial states for prediction An, Zhe; Rey, Daniel; Ye, Jingxin Nonlinear Processes in Geophysics, Vol. 24, Issue 1 https://doi.org/10.5194/npg-24-9-2017	journal	January 2017
Synchronization to Big Data: Nudging the Navier-Stokes Equations for Data Assimilation of Turbulent Flows Clark Di Leoni, Patricio; Mazzino, Andrea; Biferale, Luca Physical Review X, Vol. 10, Issue 1 https://doi.org/10.1103/physrevx.10.011023	journal	February 2020
Data assimilation by delay-coordinate nudging: Data Assimilation by Delay-Coordinate Nudging Pazó, D.; Carrassi, A.; López, J. M. Quarterly Journal of the Royal Meteorological Society, Vol. 142, Issue 696 https://doi.org/10.1002/qj.2732	journal	March 2016
Comparison of the use of a physical-based model with data assimilation and machine learning methods for simulating soil water dynamics Li, Peijun; Zha, Yuanyuan; Shi, Liangsheng Journal of Hydrology, Vol. 584 https://doi.org/10.1016/j.jhydrol.2020.124692	journal	May 2020
Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data Pathak, Jaideep; Lu, Zhixin; Hunt, Brian R. Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 27, Issue 12 https://doi.org/10.1063/1.5010300	journal	December 2017
Machine Learning for Fluid Mechanics Brunton, Steven L.; Noack, Bernd R.; Koumoutsakos, Petros Annual Review of Fluid Mechanics, Vol. 52, Issue 1 https://doi.org/10.1146/annurev-fluid-010719-060214	journal	September 2019
Machine Learning: Deepest Learning as Statistical Data Assimilation Problems Abarbanel, Henry D. I.; Rozdeba, Paul J.; Shirman, Sasha Neural Computation, Vol. 30, Issue 8 https://doi.org/10.1162/neco_a_01094	journal	August 2018
POD/DEIM reduced-order strategies for efficient four dimensional variational data assimilation Ştefănescu, R.; Sandu, A.; Navon, I. M. Journal of Computational Physics, Vol. 295 https://doi.org/10.1016/j.jcp.2015.04.030	journal	August 2015
Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization Bocquet, Marc; Brajard, Julien; Carrassi, Alberto Foundations of Data Science, Vol. 2, Issue 1 https://doi.org/10.3934/fods.2020004	journal	January 2020
Effective low-order models for atmospheric dynamics and time series analysis Gluhovsky, Alexander; Grady, Kevin Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 26, Issue 2 https://doi.org/10.1063/1.4942586	journal	February 2016
Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks Puskorius, G. V.; Feldkamp, L. A. IEEE Transactions on Neural Networks, Vol. 5, Issue 2 https://doi.org/10.1109/72.279191	journal	March 1994
Back and forth nudging algorithm for data assimilation problems Auroux, Didier; Blum, Jacques Comptes Rendus Mathematique, Vol. 340, Issue 12 https://doi.org/10.1016/j.crma.2005.05.006	journal	June 2005
Continuous data assimilation reduced order models of fluid flow Zerfas, Camille; Rebholz, Leo G.; Schneier, Michael Computer Methods in Applied Mechanics and Engineering, Vol. 357 https://doi.org/10.1016/j.cma.2019.112596	journal	December 2019
Machine learning for observation bias correction with application to dust storm data assimilation Jin, Jianbing; Lin, Hai Xiang; Segers, Arjo Atmospheric Chemistry and Physics, Vol. 19, Issue 15 https://doi.org/10.5194/acp-19-10009-2019	journal	January 2019
The Meteorological Office analysis correction data assimilation scheme Lorenc, A. C.; Bell, R. S.; Macpherson, B. Quarterly Journal of the Royal Meteorological Society, Vol. 117, Issue 497 https://doi.org/10.1002/qj.49711749704	journal	January 1991
Optimal nonlinear eddy viscosity in Galerkin models of turbulent flows Protas, Bartosz; Noack, Bernd R.; Östh, Jan Journal of Fluid Mechanics, Vol. 766 https://doi.org/10.1017/jfm.2015.14	journal	February 2015
Nonintrusive reduced order modeling framework for quasigeostrophic turbulence Rahman, Sk. M.; Pawar, S.; San, O. Physical Review E, Vol. 100, Issue 5 https://doi.org/10.1103/physreve.100.053306	journal	November 2019
The Number of Required Observations in Data Assimilation for a Shallow-Water Flow Whartenby, William G.; Quinn, John C.; Abarbanel, Henry D. I. Monthly Weather Review, Vol. 141, Issue 7 https://doi.org/10.1175/mwr-d-12-00103.1	journal	July 2013
Accurate state and parameter estimation in nonlinear systems with sparse observations Rey, Daniel; Eldridge, Michael; Kostuk, Mark Physics Letters A, Vol. 378, Issue 11-12 https://doi.org/10.1016/j.physleta.2014.01.027	journal	February 2014
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics Vlachas, P. R.; Pathak, J.; Hunt, B. R. Neural Networks, Vol. 126 https://doi.org/10.1016/j.neunet.2020.02.016	journal	June 2020
Scaling and dissipation in the GOY shell model Kadanoff, Leo; Lohse, Detlef; Wang, Jane Physics of Fluids, Vol. 7, Issue 3 https://doi.org/10.1063/1.868775	journal	March 1995
Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders Lee, Kookjin; Carlberg, Kevin T. Journal of Computational Physics https://doi.org/10.1016/j.jcp.2019.108973	journal	November 2019
Exploration and prediction of fluid dynamical systems using auto-encoder technology Agostini, Lionel Physics of Fluids, Vol. 32, Issue 6 https://doi.org/10.1063/5.0012906	journal	June 2020
Physical initialization for numerical weather prediction over the tropics Krishnamurti, T. N.; Xue, Jishan; Bedi, H. S. Tellus B, Vol. 43, Issue 4 https://doi.org/10.1034/j.1600-0889.1991.t01-2-00007.x	journal	September 1991
A deterministic formulation of the ensemble Kalman filter: an alternative to ensemble square root filters Sakov, Pavel; Oke, Peter R. Tellus A https://doi.org/10.3402/tellusa.v60i2.15252	journal	March 2008
Spectral nudging in a spectral regional climate model Radu, Raluca; DÃ©quÃ©, Michel; Somot, Samuel Tellus A https://doi.org/10.3402/tellusa.v60i5.15501	journal	October 2008
A reduced-order strategy for 4D-Var data assimilation Robert, Céline; Durbiano, S.; Blayo, Eric arXiv https://doi.org/10.48550/arxiv.0709.2825	text	January 2007
A Local Ensemble Kalman Filter for Atmospheric Data Assimilation Ott, Edward; Hunt, Brian R.; Szunyogh, Istvan arXiv https://doi.org/10.48550/arxiv.physics/0203058	preprint	January 2002

Similar Records

Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows

Journal Article · Wed May 12 00:00:00 EDT 2021 · Physical Review Fluids (Online) · OSTI ID:1737505

Pawar, Suraj; San, Omer

Equation‐Free Surrogate Modeling of Geophysical Flows at the Intersection of Machine Learning and Data Assimilation

Journal Article · Thu Nov 03 00:00:00 EDT 2022 · Journal of Advances in Modeling Earth Systems · OSTI ID:1737505

Pawar, Suraj; San, Omer

Final report on "Carbon Data Assimilation with a Coupled Ensemble Kalman Filter"

Technical Report · Wed Jul 23 00:00:00 EDT 2014 · OSTI ID:1737505

Kalnay, Eugenia; Kang, Ji-Sun; Fung, Inez

Related Subjects

42 ENGINEERING
Nudging method
long short-term memory embedding
transfer learning
extended Kalman filter
ensemble Kalman filter
Lorenz system

Title: Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows

Citation Formats

References (68)

Similar Records

Related Subjects