DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows

Abstract

In the past couple of years, there has been a proliferation in the use of machine learning approaches to represent subgrid-scale processes in geophysical flows with an aim to improve the forecasting capability and to accelerate numerical simulations of these flows. Despite its success for different types of flow, the online deployment of a data-driven closure model can cause instabilities and biases in modeling the overall effect of subgrid-scale processes, which in turn leads to inaccurate prediction. To tackle this issue, we exploit the data assimilation technique to correct the physics-based model coupled with the neural network as a surrogate for unresolved flow dynamics in multiscale systems. In particular, we use a set of neural network architectures to learn the correlation between resolved flow variables and the parametrizations of unresolved flow dynamics and formulate a data assimilation approach to correct the hybrid model during their online deployment. We illustrate our framework in a set of applications of the multiscale Lorenz 96 system for which the parametrization model for unresolved scales is exactly known, and the two-dimensional Kraichnan turbulence system for which the parametrization model for unresolved scales is not known a priori. Our analysis, therefore, comprises a predictive dynamical coremore » empowered by (i) a data-driven closure model for subgrid-scale processes, (ii) a data assimilation approach for forecast error correction, and (iii) both data-driven closure and data assimilation procedures. We show significant improvement in the long-term prediction of the underlying chaotic dynamics with our framework compared to using only neural network parametrizations for future prediction. Moreover, we demonstrate that these data-driven parametrization models can handle the non-Gaussian statistics of subgrid-scale processes, and effectively improve the accuracy of outer data assimilation workflow loops in a modular nonintrusive way.« less

Authors:
ORCiD logo [1]; ORCiD logo [1]
  1. Oklahoma State Univ., Stillwater, OK (United States)
Publication Date:
Research Org.:
Oklahoma State Univ., Stillwater, OK (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1853026
Grant/Contract Number:  
SC0019290
Resource Type:
Accepted Manuscript
Journal Name:
Physical Review Fluids (Online)
Additional Journal Information:
Journal Name: Physical Review Fluids (Online); Journal Volume: 6; Journal Issue: 5; Journal ID: ISSN 2469-990X
Publisher:
American Physical Society (APS)
Country of Publication:
United States
Language:
English
Subject:
42 ENGINEERING; Physics; Chaos; Control & applications of chaos; Geophysical fluid dynamics; Mixing in geophysical flows; Spatiotemporal chaos

Citation Formats

Pawar, Suraj, and San, Omer. Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows. United States: N. p., 2021. Web. doi:10.1103/physrevfluids.6.050501.
Pawar, Suraj, & San, Omer. Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows. United States. https://doi.org/10.1103/physrevfluids.6.050501
Pawar, Suraj, and San, Omer. Wed . "Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows". United States. https://doi.org/10.1103/physrevfluids.6.050501. https://www.osti.gov/servlets/purl/1853026.
@article{osti_1853026,
title = {Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows},
author = {Pawar, Suraj and San, Omer},
abstractNote = {In the past couple of years, there has been a proliferation in the use of machine learning approaches to represent subgrid-scale processes in geophysical flows with an aim to improve the forecasting capability and to accelerate numerical simulations of these flows. Despite its success for different types of flow, the online deployment of a data-driven closure model can cause instabilities and biases in modeling the overall effect of subgrid-scale processes, which in turn leads to inaccurate prediction. To tackle this issue, we exploit the data assimilation technique to correct the physics-based model coupled with the neural network as a surrogate for unresolved flow dynamics in multiscale systems. In particular, we use a set of neural network architectures to learn the correlation between resolved flow variables and the parametrizations of unresolved flow dynamics and formulate a data assimilation approach to correct the hybrid model during their online deployment. We illustrate our framework in a set of applications of the multiscale Lorenz 96 system for which the parametrization model for unresolved scales is exactly known, and the two-dimensional Kraichnan turbulence system for which the parametrization model for unresolved scales is not known a priori. Our analysis, therefore, comprises a predictive dynamical core empowered by (i) a data-driven closure model for subgrid-scale processes, (ii) a data assimilation approach for forecast error correction, and (iii) both data-driven closure and data assimilation procedures. We show significant improvement in the long-term prediction of the underlying chaotic dynamics with our framework compared to using only neural network parametrizations for future prediction. Moreover, we demonstrate that these data-driven parametrization models can handle the non-Gaussian statistics of subgrid-scale processes, and effectively improve the accuracy of outer data assimilation workflow loops in a modular nonintrusive way.},
doi = {10.1103/physrevfluids.6.050501},
journal = {Physical Review Fluids (Online)},
number = 5,
volume = 6,
place = {United States},
year = {Wed May 12 00:00:00 EDT 2021},
month = {Wed May 12 00:00:00 EDT 2021}
}

Works referenced in this record:

Dynamical Subgrid-Scale Parameterizations from Direct Numerical Simulations
journal, November 2006

  • Frederiksen, Jorgen S.; Kepert, Steven M.
  • Journal of the Atmospheric Sciences, Vol. 63, Issue 11
  • DOI: 10.1175/JAS3795.1

Accelerating Progress in Global Atmospheric Model Development through Improved Parameterizations: Challenges, Opportunities, and Strategies
journal, July 2010

  • Jakob, Christian
  • Bulletin of the American Meteorological Society, Vol. 91, Issue 7
  • DOI: 10.1175/2009BAMS2898.1

A proposed modification of the Germano subgrid‐scale closure method
journal, March 1992

  • Lilly, D. K.
  • Physics of Fluids A: Fluid Dynamics, Vol. 4, Issue 3
  • DOI: 10.1063/1.858280

Deep learning
journal, May 2015

  • LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
  • Nature, Vol. 521, Issue 7553
  • DOI: 10.1038/nature14539

Two-Dimensional Turbulence
journal, January 2012


A Maximum Likelihood Ensemble Filter via a Modified Cholesky Decomposition for Non-Gaussian Data Assimilation
journal, February 2020

  • Nino-Ruiz, Elias David; Mancilla-Herrera, Alfonso; Lopez-Restrepo, Santiago
  • Sensors, Vol. 20, Issue 3
  • DOI: 10.3390/s20030877

A deterministic formulation of the ensemble Kalman filter: an alternative to ensemble square root filters
journal, January 2008


Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach
journal, January 2018


Optimal reduced space for Variational Data Assimilation
journal, February 2019

  • Arcucci, Rossella; Mottet, Laetitia; Pain, Christopher
  • Journal of Computational Physics, Vol. 379
  • DOI: 10.1016/j.jcp.2018.10.042

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019


Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the Lorenz 96 model
journal, July 2020


The role of large-scale spatial patterns in the chaotic amplification of perturbations in a Lorenz’96 model
journal, January 2011


Reynolds-averaged Navier–Stokes equations with explicit data-driven Reynolds stress closure can be ill-conditioned
journal, April 2019


Stochastic parameterization for large eddy simulation of geophysical flows
journal, April 2007

  • Duan, Jinqiao; Nadiga, Balasubramanya T.
  • Proceedings of the American Mathematical Society, Vol. 135, Issue 04
  • DOI: 10.1090/S0002-9939-06-08631-X

A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems
journal, July 2020


Review of the Ensemble Kalman Filter for Atmospheric Data Assimilation
journal, December 2016


Symmetries in the Lorenz-96 Model
journal, January 2019

  • van Kekem, Dirk L.; Sterk, Alef E.
  • International Journal of Bifurcation and Chaos, Vol. 29, Issue 01
  • DOI: 10.1142/S0218127419500081

Effects of stochastic parametrizations in the Lorenz '96 system
journal, January 2005

  • Wilks, Daniel S.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 131, Issue 606
  • DOI: 10.1256/qj.04.03

Dynamic Data Assimilation
book, January 2006


A New Approach to Linear Filtering and Prediction Problems
journal, March 1960

  • Kalman, R. E.
  • Journal of Basic Engineering, Vol. 82, Issue 1
  • DOI: 10.1115/1.3662552

Deep learning to represent subgrid processes in climate models
journal, September 2018

  • Rasp, Stephan; Pritchard, Michael S.; Gentine, Pierre
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 39
  • DOI: 10.1073/pnas.1810286115

Total variation diminishing Runge-Kutta schemes
journal, January 1998

  • Gottlieb, Sigal; Shu, Chi-Wang
  • Mathematics of Computation of the American Mathematical Society, Vol. 67, Issue 221
  • DOI: 10.1090/S0025-5718-98-00913-2

An adaptive covariance inflation error correction algorithm for ensemble filters
journal, January 2007


Approximate deconvolution large eddy simulation of a stratified two-layer quasigeostrophic ocean model
journal, March 2013


Deep neural networks for data-driven LES closure models
journal, December 2019


A Machine Learning‐Based Global Atmospheric Forecast Model
journal, May 2020

  • Arcomano, Troy; Szunyogh, Istvan; Pathak, Jaideep
  • Geophysical Research Letters, Vol. 47, Issue 9
  • DOI: 10.1029/2020GL087776

Data Assimilation
book, January 2009


Modeling subgrid-scale forces by spatial artificial neural networks in large eddy simulation of turbulence
journal, May 2020


Assessment and Propagation of Model Uncertainty
journal, January 1995


Turbulence model reduction by deep learning
journal, June 2020


Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems
journal, April 2020


Structure functions in two-dimensional turbulence
journal, February 1992

  • Grossmann, Siegfried; Mertens, Peter
  • Zeitschrift für Physik B Condensed Matter, Vol. 88, Issue 1
  • DOI: 10.1007/BF01573844

Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems
journal, May 2020


Statistical mechanics of two-dimensional and geophysical flows
journal, June 2012


Hidden physics models: Machine learning of nonlinear partial differential equations
journal, March 2018


Sampling the posterior: An approach to non-Gaussian data assimilation
journal, June 2007


Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data
journal, October 2017

  • Karpatne, Anuj; Atluri, Gowtham; Faghmous, James H.
  • IEEE Transactions on Knowledge and Data Engineering, Vol. 29, Issue 10
  • DOI: 10.1109/TKDE.2017.2720168

Predictive large-eddy-simulation wall modeling via physics-informed neural networks
journal, March 2019


Spatially Extended Tests of a Neural Network Parametrization Trained by Coarse‐Graining
journal, August 2019

  • Brenowitz, Noah D.; Bretherton, Christopher S.
  • Journal of Advances in Modeling Earth Systems, Vol. 11, Issue 8
  • DOI: 10.1029/2019MS001711

A turbulent eddy-viscosity surrogate modeling framework for Reynolds-averaged Navier-Stokes simulations
journal, September 2021


Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
journal, October 2016

  • Ling, Julia; Kurzawski, Andrew; Templeton, Jeremy
  • Journal of Fluid Mechanics, Vol. 807
  • DOI: 10.1017/jfm.2016.615

Inertial Ranges in Two-Dimensional Turbulence
journal, January 1967


Analysis Scheme in the Ensemble Kalman Filter
journal, June 1998


A proof of concept for scale-adaptive parametrizations: the case of the Lorenz '96 model: Proof for Scale-adaptive Parametrizations
journal, December 2017

  • Vissio, Gabriele; Lucarini, Valerio
  • Quarterly Journal of the Royal Meteorological Society, Vol. 144, Issue 710
  • DOI: 10.1002/qj.3184

Prognostic Validation of a Neural Network Unified Physics Parameterization
journal, June 2018

  • Brenowitz, N. D.; Bretherton, C. S.
  • Geophysical Research Letters, Vol. 45, Issue 12
  • DOI: 10.1029/2018GL078510

Assessing the scales in numerical weather and climate predictions: will exascale be the rescue?
journal, February 2019

  • Neumann, Philipp; Düben, Peter; Adamidis, Panagiotis
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 377, Issue 2142
  • DOI: 10.1098/rsta.2018.0148

Ensemble Kalman Filter for Assimilating Experimental Data into Large-Eddy Simulations of Turbulent Flows
journal, December 2019

  • Labahn, Jeffrey W.; Wu, Hao; Harris, Shaun R.
  • Flow, Turbulence and Combustion, Vol. 104, Issue 4
  • DOI: 10.1007/s10494-019-00093-1

Uncertainty in Model Climate Sensitivity Traced to Representations of Cumulus Precipitation Microphysics
journal, January 2016


An Improved Strategy for the Evaluation of Cloud Parameterizations in GCMS
journal, October 2003


A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence
journal, January 2020

  • Pawar, Suraj; San, Omer; Rasheed, Adil
  • Theoretical and Computational Fluid Dynamics, Vol. 34, Issue 4
  • DOI: 10.1007/s00162-019-00512-z

Gradient-based learning applied to document recognition
journal, January 1998

  • Lecun, Y.; Bottou, L.; Bengio, Y.
  • Proceedings of the IEEE, Vol. 86, Issue 11
  • DOI: 10.1109/5.726791

Earth observation satellite sensors for biodiversity monitoring: potentials and bottlenecks
journal, September 2014

  • Kuenzer, Claudia; Ottinger, Marco; Wegmann, Martin
  • International Journal of Remote Sensing, Vol. 35, Issue 18
  • DOI: 10.1080/01431161.2014.964349

Shallow neural networks for fluid flow reconstruction with limited sensors
journal, June 2020

  • Erichson, N. Benjamin; Mathelin, Lionel; Yao, Zhewei
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 476, Issue 2238
  • DOI: 10.1098/rspa.2020.0097

Super-resolution reconstruction of turbulent flows with machine learning
journal, May 2019

  • Fukami, Kai; Fukagata, Koji; Taira, Kunihiko
  • Journal of Fluid Mechanics, Vol. 870
  • DOI: 10.1017/jfm.2019.238

Ensemble-Based State Estimator for Aerodynamic Flows
journal, July 2018

  • da Silva, Andre F. C.; Colonius, Tim
  • AIAA Journal, Vol. 56, Issue 7
  • DOI: 10.2514/1.J056743

Predictions of turbulent shear flows using deep neural networks
journal, May 2019


Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders
journal, November 2019


Theory-Guided Machine Learning in Materials Science
journal, June 2016


Could Machine Learning Break the Convection Parameterization Deadlock?
journal, June 2018

  • Gentine, P.; Pritchard, M.; Rasp, S.
  • Geophysical Research Letters, Vol. 45, Issue 11
  • DOI: 10.1029/2018GL078202

Data-assisted reduced-order modeling of extreme events in complex dynamical systems
journal, May 2018


Towards a mesoscale eddy closure
journal, January 2008


A dynamic subgrid‐scale eddy viscosity model
journal, July 1991

  • Germano, Massimo; Piomelli, Ugo; Moin, Parviz
  • Physics of Fluids A: Fluid Dynamics, Vol. 3, Issue 7
  • DOI: 10.1063/1.857955

Turbulence Modeling in the Age of Data
journal, January 2019


Data-driven modelling of nonlinear spatio-temporal fluid flows using a deep convolutional generative adversarial network
journal, June 2020

  • Cheng, M.; Fang, F.; Pain, C. C.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 365
  • DOI: 10.1016/j.cma.2020.113000

Atmospheric Predictability and Two-Dimensional Turbulence
journal, March 1971


High-order methods for decaying two-dimensional homogeneous isotropic turbulence
journal, June 2012


A nonlinear dynamical perspective on model error: A proposal for non-local stochastic-dynamic parametrization in weather and climate prediction models
journal, January 2001

  • Palmer, T. N.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 127, Issue 572
  • DOI: 10.1002/qj.49712757202

Subgrid modelling for geophysical flows
journal, January 2013

  • Frederiksen, Jorgen S.; O'Kane, Terence J.; Zidikheri, Meelis J.
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 371, Issue 1982
  • DOI: 10.1098/rsta.2012.0166

Middle atmosphere dynamical sources of the semiannual oscillation in the thermosphere and ionosphere: MIDDLE ATMOSPHERE SOURCES OF THE T-I SAO
journal, January 2017

  • Jones, M.; Emmert, J. T.; Drob, D. P.
  • Geophysical Research Letters, Vol. 44, Issue 1
  • DOI: 10.1002/2016GL071741

Subgrid-Scale Parameterization with Conditional Markov Chains
journal, August 2008

  • Crommelin, Daan; Vanden-Eijnden, Eric
  • Journal of the Atmospheric Sciences, Vol. 65, Issue 8
  • DOI: 10.1175/2008JAS2566.1

A Non-Gaussian Ensemble Filter Update for Data Assimilation
journal, November 2010


A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts
journal, December 1999


A deep learning enabler for nonintrusive reduced order modeling of fluid flows
journal, August 2019

  • Pawar, S.; Rahman, S. M.; Vaddireddy, H.
  • Physics of Fluids, Vol. 31, Issue 8
  • DOI: 10.1063/1.5113494

Deep learning for physical processes: incorporating prior scientific knowledge
journal, December 2019

  • de Bézenac, Emmanuel; Pajot, Arthur; Gallinari, Patrick
  • Journal of Statistical Mechanics: Theory and Experiment, Vol. 2019, Issue 12
  • DOI: 10.1088/1742-5468/ab3195

Prediction of turbulent heat transfer using convolutional neural networks
journal, November 2019


Extensive chaos in the Lorenz-96 model
journal, December 2010

  • Karimi, A.; Paul, M. R.
  • Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 20, Issue 4
  • DOI: 10.1063/1.3496397

Parameterised non-intrusive reduced order methods for ensemble Kalman filter data assimilation
journal, November 2018


Artificial neural network approach to large-eddy simulation of compressible isotropic turbulence
journal, May 2019


Cloud parameterization for climate modeling: Status and prospects
journal, October 1989


Moisture Vertical Structure, Column Water Vapor, and Tropical Deep Convection
journal, June 2009

  • Holloway, Christopher E.; Neelin, J. David
  • Journal of the Atmospheric Sciences, Vol. 66, Issue 6
  • DOI: 10.1175/2008JAS2806.1

Combining machine learning with knowledge-based modeling for scalable forecasting and subgrid-scale closure of large, complex, spatiotemporal systems
journal, May 2020

  • Wikner, Alexander; Pathak, Jaideep; Hunt, Brian
  • Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 30, Issue 5
  • DOI: 10.1063/5.0005541

Harmonizing models and observations: Data assimilation in Earth system science
journal, June 2020


Deep learning and process understanding for data-driven Earth system science
journal, February 2019


Subgrid modelling for two-dimensional turbulence using neural networks
journal, November 2018


Theory-Guided Data Science for Climate Change
journal, November 2014

  • Faghmous, James H.; Banerjee, Arindam; Shekhar, Shashi
  • Computer, Vol. 47, Issue 11
  • DOI: 10.1109/MC.2014.335

State estimation in wall-bounded flow systems. Part 3. The ensemble Kalman filter
journal, August 2011

  • Colburn, C. H.; Cessna, J. B.; Bewley, T. R.
  • Journal of Fluid Mechanics, Vol. 682
  • DOI: 10.1017/jfm.2011.222

The maximum likelihood ensemble filter performances in chaotic systems
journal, January 2008

  • Carrassi, Alberto; Vannitsem, Stephane; Zupanski, Dusanka
  • Tellus A: Dynamic Meteorology and Oceanography, Vol. 61, Issue 5
  • DOI: 10.1111/j.1600-0870.2009.00408.x

Machine Learning for Fluid Mechanics
journal, September 2019


The Analog Data Assimilation
journal, October 2017

  • Lguensat, Redouane; Tandeo, Pierre; Ailliot, Pierre
  • Monthly Weather Review, Vol. 145, Issue 10
  • DOI: 10.1175/MWR-D-16-0441.1

Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data
journal, March 2017


Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization
journal, January 2020

  • Bocquet, Marc; Brajard, Julien; Carrassi, Alberto
  • Foundations of Data Science, Vol. 2, Issue 1
  • DOI: 10.3934/fods.2020004

Continuous data assimilation reduced order models of fluid flow
journal, December 2019

  • Zerfas, Camille; Rebholz, Leo G.; Schneier, Michael
  • Computer Methods in Applied Mechanics and Engineering, Vol. 357
  • DOI: 10.1016/j.cma.2019.112596

Filter accuracy for the Lorenz 96 model: Fixed versus adaptive observation operators
journal, June 2016


Reconstruction of unsteady viscous flows using data assimilation schemes
journal, July 2016


Deep neural networks for data-driven LES closure models
journal, December 2019


Neural networks based subgrid scale modeling in large eddy simulations
journal, January 2003


Reynolds-averaged Navier–Stokes equations with explicit data-driven Reynolds stress closure can be ill-conditioned
journal, April 2019


Perspective on machine learning for advancing fluid mechanics
journal, October 2019


Spatio-temporal interpolation of altimeter-derived SSH fields using analog data assimilation: A case-study in the south china sea
conference, July 2017

  • Lguensat, Redouane; Sun, Miao; Chen, Ge
  • 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)
  • DOI: 10.1109/igarss.2017.8127625

The role of large-scale spatial patterns in the chaotic amplification of perturbations in a Lorenz’96 model
journal, January 2011


Maximum Likelihood Ensemble Filter: Theoretical Aspects
journal, June 2005


Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model
preprint, January 2017


Super-resolution reconstruction of turbulent flows with machine learning
text, January 2018


Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors
text, January 2019


Recurrent Neural Network Architecture Search for Geophysical Emulation
preprint, January 2020