Learning subgrid-scale models with neural ordinary differential equations

Kang, Shinhoo; Constantinescu, Emil M.

doi:10.1016/j.compfluid.2023.105919

Learning subgrid-scale models with neural ordinary differential equations

Journal Article · Sat May 06 00:00:00 EDT 2023 · Computers and Fluids

DOI:https://doi.org/10.1016/j.compfluid.2023.105919· OSTI ID:2377337

^[1]; ^[1]

Argonne National Laboratory (ANL), Argonne, IL (United States)

We propose a new approach to learning the subgrid-scale model when simulating partial differential equations (PDEs) solved by the method of lines and their representation in chaotic ordinary differential equations, based on neural ordinary differential equations (NODEs). Solving systems with fine temporal and spatial grid scales is an ongoing computational challenge, and closure models are generally difficult to tune. Machine learning approaches have increased the accuracy and efficiency of computational fluid dynamics solvers. In this approach neural networks are used to learn the coarse- to fine-grid map, which can be viewed as subgrid-scale parameterization. We propose a strategy that uses the NODE and partial knowledge to learn the source dynamics at a continuous level. Our method inherits the advantages of NODEs and can be used to parameterize subgrid scales, approximate coupling operators, and improve the efficiency of low-order solvers. Numerical results with the two-scale Lorenz 96 ODE, the convection-diffusion PDE, and the viscous Burgers' PDE are used to illustrate this approach.

View Accepted Manuscript (DOE)

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: AC02-06CH11357

OSTI ID:: 2377337

Journal Information:: Computers and Fluids, Journal Name: Computers and Fluids Vol. 261; ISSN 0045-7930

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (25)

Neural network closures for nonlinear model order reduction San, Omer; Maulik, Romit Advances in Computational Mathematics, Vol. 44, Issue 6 https://doi.org/10.1007/s10444-018-9590-z	journal	January 2018
A family of embedded Runge-Kutta formulae Dormand, J. R.; Prince, P. J. Journal of Computational and Applied Mathematics, Vol. 6, Issue 1 https://doi.org/10.1016/0771-050X(80)90013-3	journal	March 1980
Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert Neural Networks, Vol. 3, Issue 5 https://doi.org/10.1016/0893-6080(90)90005-6	journal	January 1990
Galaxy morphology classification using neural ordinary differential equations Gupta, R.; Srijith, P. K.; Desai, S. Astronomy and Computing, Vol. 38 https://doi.org/10.1016/j.ascom.2021.100543	journal	January 2022
Explicit and implicit LES closures for Burgers turbulence Maulik, Romit; San, Omer Journal of Computational and Applied Mathematics, Vol. 327 https://doi.org/10.1016/j.cam.2017.06.003	journal	January 2018
Runge–Kutta pairs of order 5(4) satisfying only the first column simplifying assumption Tsitouras, Ch. Computers & Mathematics with Applications, Vol. 62, Issue 2 https://doi.org/10.1016/j.camwa.2011.06.002	journal	July 2011
Stabilized neural ordinary differential equations for long-time forecasting of dynamical systems Linot, Alec J.; Burby, Joshua W.; Tang, Qi Journal of Computational Physics, Vol. 474 https://doi.org/10.1016/j.jcp.2022.111838	journal	February 2023
Burgers turbulence Bec, J.; Khanin, K. Physics Reports, Vol. 447, Issue 1-2 https://doi.org/10.1016/j.physrep.2007.04.002	journal	August 2007
Subgrid modelling studies with Burgers’ equation Love, M. D. Journal of Fluid Mechanics, Vol. 100, Issue 01 https://doi.org/10.1017/S0022112080001024	journal	September 1980
Applications of Deep Learning to Ocean Data Inference and Subgrid Parameterization Bolton, Thomas; Zanna, Laure Journal of Advances in Modeling Earth Systems, Vol. 11, Issue 1 https://doi.org/10.1029/2018MS001472	journal	January 2019
Spatially Extended Tests of a Neural Network Parametrization Trained by Coarse‐Graining Brenowitz, Noah D.; Bretherton, Christopher S. Journal of Advances in Modeling Earth Systems, Vol. 11, Issue 8 https://doi.org/10.1029/2019MS001711	journal	August 2019
The quiet revolution of numerical weather prediction Bauer, Peter; Thorpe, Alan; Brunet, Gilbert Nature, Vol. 525, Issue 7567 https://doi.org/10.1038/nature14956	journal	September 2015
Climate goals and computing the future of clouds Schneider, Tapio; Teixeira, João; Bretherton, Christopher S. Nature Climate Change, Vol. 7, Issue 1 https://doi.org/10.1038/nclimate3190	journal	January 2017
Organizing principles for vegetation dynamics Franklin, Oskar; Harrison, Sandy P.; Dewar, Roderick Nature Plants, Vol. 6, Issue 5 https://doi.org/10.1038/s41477-020-0655-x	journal	May 2020
Towards neural Earth system modelling by integrating artificial intelligence in Earth system science Irrgang, Christopher; Boers, Niklas; Sonnewald, Maike Nature Machine Intelligence, Vol. 3, Issue 8 https://doi.org/10.1038/s42256-021-00374-3	journal	August 2021
A framework for large eddy simulation of Burgers turbulence based upon spatial and temporal statistical information LaBryer, A.; Attar, P. J.; Vedula, P. Physics of Fluids, Vol. 27, Issue 3 https://doi.org/10.1063/1.4916132	journal	March 2015
Deep learning to represent subgrid processes in climate models Rasp, Stephan; Pritchard, Michael S.; Gentine, Pierre Proceedings of the National Academy of Sciences, Vol. 115, Issue 39 https://doi.org/10.1073/pnas.1810286115	journal	September 2018
Can the dynamic eddy-viscosity class of subgrid-scale models capture inertial-range properties of Burgers turbulence? Basu, Sukanta Journal of Turbulence, Vol. 10 https://doi.org/10.1080/14685240902852719	journal	January 2009
Deep Residual Learning for Image Recognition He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.90	conference	June 2016
Improving Robustness and Uncertainty Modelling in Neural Ordinary Differential Equations Anumasa, Srinivas; Srijith, P. K. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) https://doi.org/10.1109/WACV48630.2021.00410	conference	January 2021
Neural ODEs as the deep limit of ResNets with constant weights Avelin, Benny; Nyström, Kaj Analysis and Applications, Vol. 19, Issue 03 https://doi.org/10.1142/S0219530520400023	journal	May 2020
Next-Generation Numerical Weather Prediction: Bridging Parameterization, Explicit Clouds, and Large Eddies Hong, Song-You; Dudhia, Jimy Bulletin of the American Meteorological Society, Vol. 93, Issue 1 https://doi.org/10.1175/2011BAMS3224.1	journal	January 2012
Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization Bocquet, Marc; Brajard, Julien; Carrassi, Alberto Foundations of Data Science, Vol. 2, Issue 1 https://doi.org/10.3934/fods.2020004	journal	January 2020
Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: general algorithms and Lorenz 96 case study (v1.0) Rasp, Stephan Geoscientific Model Development, Vol. 13, Issue 5 https://doi.org/10.5194/gmd-13-2185-2020	journal	January 2020
Lyapunov analysis of multiscale dynamics: the slow bundle of the two-scale Lorenz 96 model Carlu, Mallory; Ginelli, Francesco; Lucarini, Valerio Nonlinear Processes in Geophysics, Vol. 26, Issue 2 https://doi.org/10.5194/npg-26-73-2019	journal	January 2019

Similar Records

Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems

Technical Report · Thu Nov 05 23:00:00 EST 2020 · OSTI ID:1706214

Stiff neural ordinary differential equations

Journal Article · Mon Sep 20 00:00:00 EDT 2021 · Chaos: An Interdisciplinary Journal of Nonlinear Science · OSTI ID:1848383

Divide and conquer: Learning chaotic dynamical systems with multistep penalty neural ordinary differential equations

Journal Article · Mon Oct 14 00:00:00 EDT 2024 · Computer Methods in Applied Mechanics and Engineering · OSTI ID:3003857

Related Subjects

97 MATHEMATICS AND COMPUTING
acceleration
coupling
discontinuous Galerkin method
machine learning
neural ordinary differential equations

Learning subgrid-scale models with neural ordinary differential equations

Citation Formats

References (25)

Similar Records

Related Subjects