Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning

Carlberg, Kevin Thomas; Jameson, Antony; Kochenderfer, Mykel J.; Morton, Jeremy; Peng, Liqian; Witherden, Freddie D.

doi:10.1016/j.jcp.2019.05.041

Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning

Journal Article · Wed Jun 12 00:00:00 EDT 2019 · Journal of Computational Physics

DOI:https://doi.org/10.1016/j.jcp.2019.05.041· OSTI ID:1529308

Carlberg, Kevin Thomas ^[1]; Jameson, Antony ^[2]; ^[3]; Morton, Jeremy ^[3]; Peng, Liqian ^[1]; Witherden, Freddie D. ^[3]

Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Texas A & M Univ., College Station, TX (United States)
Stanford Univ., Stanford, CA (United States)

Data I/O poses a significant bottleneck in large-scale CFD simulations; thus, practitioners would like to significantly reduce the number of times the solution is saved to disk, yet retain the ability to recover any field quantity (at any time instance) a posteriori. The objective of this work is therefore to accurately recover missing CFD data a posteriori at any time instance, given that the solution has been written to disk at only a relatively small number of time instances. We consider in particular high-order discretizations (e.g., discontinuous Galerkin), as such techniques are becoming increasingly popular for the simulation of highly separated flows. To satisfy this objective, this work proposes a methodology consisting of two stages: 1) dimensionality reduction and 2) dynamics learning. For dimensionality reduction, we propose a novel hierarchical approach. First, the method reduces the number of degrees of freedom within each element of the high-order discretization by applying autoencoders from deep learning. Second, the methodology applies principal component analysis to compress the global vector of encodings. This leads to a low-dimensional state, which associates with a nonlinear embedding of the original CFD data. For dynamics learning, we propose to apply regression techniques (e.g., kernel methods) to learn the discrete-time velocity characterizing the time evolution of this low-dimensional state. In conclusion, a numerical example on a large-scale CFD example characterized by nearly 13 million degrees of freedom illustrates the suitability of the proposed method in an industrial setting.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-CA), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1529308

Report Number(s):: SAND--2018-13713J; 670930

Journal Information:: Journal of Computational Physics, Journal Name: Journal of Computational Physics Journal Issue: C Vol. 395; ISSN 0021-9991

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (39)

Heterogeneous computing on mixed unstructured grids with PyFR Witherden, F. D.; Vermeire, B. C.; Vincent, P. E. Computers & Fluids, Vol. 120 https://doi.org/10.1016/j.compfluid.2015.07.016	journal	October 2015
State representation learning for control: An overview Lesort, Timothée; Díaz-Rodríguez, Natalia; Goudou, Jean-Frano̧is Neural Networks, Vol. 108 https://doi.org/10.1016/j.neunet.2018.07.006	journal	December 2018
Gappy data and reconstruction procedures for flow past a cylinder Venturi, Daniele; Karniadakis, George Em Journal of Fluid Mechanics, Vol. 519 https://doi.org/10.1017/s0022112004001338	journal	January 1999
Dynamic mode decomposition of numerical and experimental data Schmid, Peter J. Journal of Fluid Mechanics, Vol. 656 https://doi.org/10.1017/s0022112010001217	journal	July 2010
Experimental and numerical studies of the flow over a circular cylinder at Reynolds number 3900 Parnaudeau, Philippe; Carlier, Johan; Heitz, Dominique Physics of Fluids, Vol. 20, Issue 8 https://doi.org/10.1063/1.2957018	journal	August 2008
EOF Calculations and Data Filling from Incomplete Oceanographic Datasets* Beckers, J. M.; Rixen, M. Journal of Atmospheric and Oceanic Technology, Vol. 20, Issue 12 https://doi.org/10.1175/1520-0426(2003)020<1839:ecadff>2.0.co;2	journal	December 2003
Correction: Deep Multilayer Convolution Frameworks for Data-Driven Learning of Fluid flow Dynamics Puligilla, Shivakanth Chary; Jayaraman, Balaji 2018 Fluid Dynamics Conference https://doi.org/10.2514/6.2018-3091.c1	conference	June 2018
Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data Karl, Maximilian; Soelch, Maximilian; Bayer, Justin arXiv https://doi.org/10.48550/arxiv.1605.06432	preprint	January 2016
Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition Takeishi, Naoya; Kawahara, Yoshinobu; Yairi, Takehisa arXiv https://doi.org/10.48550/arxiv.1710.04340	preprint	January 2017
Deep learning for universal linear embeddings of nonlinear dynamics Lusch, Bethany; Kutz, J. Nathan; Brunton, Steven L. arXiv https://doi.org/10.48550/arxiv.1712.09707	text	January 2017
Deep Dynamical Modeling and Control of Unsteady Fluid Flows Morton, Jeremy; Witherden, Freddie D.; Jameson, Antony arXiv https://doi.org/10.48550/arxiv.1805.07472	preprint	January 2018
Surrogate modeling of multiscale models using kernel methods: KERNEL SURROGATE MULTISCALE MODELS Wirtz, D.; Karajan, N.; Haasdonk, B. International Journal for Numerical Methods in Engineering, Vol. 101, Issue 1 https://doi.org/10.1002/nme.4767	journal	November 2014
A Staggered-Grid Multidomain Spectral Method for the Compressible Navier–Stokes Equations Kopriva, David A. Journal of Computational Physics, Vol. 143, Issue 1 https://doi.org/10.1006/jcph.1998.5956	journal	June 1998
Vortex Shedding from Bluff Bodies with Splitter Plates Nakamura, Y. Journal of Fluids and Structures, Vol. 10, Issue 2 https://doi.org/10.1006/jfls.1996.0010	journal	February 1996
A study on CFD data compression using hybrid supercompact wavelets Kang, Hyungmin; Lee, Dongho; Lee, Dohyung KSME International Journal, Vol. 17, Issue 11 https://doi.org/10.1007/BF02983609	journal	November 2003
A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition Williams, Matthew O.; Kevrekidis, Ioannis G.; Rowley, Clarence W. Journal of Nonlinear Science, Vol. 25, Issue 6 https://doi.org/10.1007/s00332-015-9258-5	journal	June 2015
Characterization by proper-orthogonal-decomposition of the passive controlled wake flow downstream of a half cylinder Cruz, A. Santa; David, L.; Pécheux, J. Experiments in Fluids, Vol. 39, Issue 4 https://doi.org/10.1007/s00348-005-0006-8	journal	July 2005
Autonomous Learning of State Representations for Control: An Emerging Field Aims to Autonomously Learn State Representations for Reinforcement Learning Agents from Their Real-World Sensor Observations Böhmer, Wendelin; Springenberg, Jost Tobias; Boedecker, Joschka KI - Künstliche Intelligenz, Vol. 29, Issue 4 https://doi.org/10.1007/s13218-015-0356-1	journal	March 2015
Computation of unsteady flow over a half-cylinder close to a moving wall Kumarasamy, Sanjay; Barlow, Jewel B. Journal of Wind Engineering and Industrial Aerodynamics, Vol. 69-71 https://doi.org/10.1016/S0167-6105(97)00158-X	journal	July 1997
Low-storage, explicit Runge–Kutta schemes for the compressible Navier–Stokes equations Kennedy, Christopher A.; Carpenter, Mark H.; Lewis, R. Michael Applied Numerical Mathematics, Vol. 35, Issue 3 https://doi.org/10.1016/S0168-9274(99)00141-5	journal	November 2000
Unsteady flow sensing and estimation via the gappy proper orthogonal decomposition Willcox, K. Computers & Fluids, Vol. 35, Issue 2 https://doi.org/10.1016/j.compfluid.2004.11.006	journal	February 2006
Parallel implementation of large-scale CFD data compression toward aeroacoustic analysis Sakai, Ryotaro; Sasaki, Daisuke; Nakahashi, Kazuhiro Computers & Fluids, Vol. 80 https://doi.org/10.1016/j.compfluid.2012.04.020	journal	July 2013
Gappy data: To Krig or not to Krig? Gunes, Hasan; Sirisup, Sirod; Karniadakis, George Em Journal of Computational Physics, Vol. 212, Issue 1 https://doi.org/10.1016/j.jcp.2005.06.023	journal	February 2006
Spectral difference method for unstructured grids I: Basic formulation Liu, Yen; Vinokur, Marcel; Wang, Z. J. Journal of Computational Physics, Vol. 216, Issue 2 https://doi.org/10.1016/j.jcp.2006.01.024	journal	August 2006
Galerkin v. least-squares Petrov–Galerkin projection in nonlinear model reduction Carlberg, Kevin; Barone, Matthew; Antil, Harbir Journal of Computational Physics, Vol. 330 https://doi.org/10.1016/j.jcp.2016.10.033	journal	February 2017
On the utility of GPU accelerated high-order methods for unsteady flow simulations: A comparison with industry-standard tools Vermeire, B. C.; Witherden, F. D.; Vincent, P. E. Journal of Computational Physics, Vol. 334 https://doi.org/10.1016/j.jcp.2016.12.049	journal	April 2017
Gappy data and reconstruction procedures for flow past a cylinder Venturi, Daniele; Karniadakis, George Em Journal of Fluid Mechanics, Vol. 519 https://doi.org/10.1017/S0022112004001338	journal	January 1999
Dynamic mode decomposition of numerical and experimental data Schmid, Peter J. Journal of Fluid Mechanics, Vol. 656 https://doi.org/10.1017/S0022112010001217	journal	July 2010
Random Forests Breiman, Leo Machine Learning, Vol. 45, Issue 1, p. 5-32 https://doi.org/10.1023/A:1010933404324	journal	January 2001
A tutorial on support vector regression Smola, Alex J.; Schölkopf, Bernhard Statistics and Computing, Vol. 14, Issue 3 https://doi.org/10.1023/B:STCO.0000035301.49549.88	journal	August 2004
The Runge-Kutta local projection $P^1$-discontinuous-Galerkin finite element method for scalar conservation laws Cockburn, Bernardo; Shu, Chi-Wang ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 25, Issue 3 https://doi.org/10.1051/m2an/1991250303371	journal	January 1991
Facilitating the Adoption of Unstructured High-Order Methods Amongst a Wider Community of Fluid Dynamicists Vincent, P. E.; Jameson, A. Mathematical Modelling of Natural Phenomena, Vol. 6, Issue 3 https://doi.org/10.1051/mmnp/20116305	journal	January 2011
Discovering governing equations from data by sparse identification of nonlinear dynamical systems Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan Proceedings of the National Academy of Sciences, Vol. 113, Issue 15 https://doi.org/10.1073/pnas.1517384113	journal	March 2016
Deep Residual Learning for Image Recognition He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.90	conference	June 2016
Regularization and variable selection via the elastic net Zou, Hui; Hastie, Trevor Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 67, Issue 2 https://doi.org/10.1111/j.1467-9868.2005.00503.x	journal	April 2005
Reducing the Dimensionality of Data with Neural Networks Hinton, G. E. Science, Vol. 313, Issue 5786 https://doi.org/10.1126/science.1127647	journal	July 2006
Optimization Methods for Large-Scale Machine Learning Bottou, Léon; Curtis, Frank E.; Nocedal, Jorge SIAM Review, Vol. 60, Issue 2 https://doi.org/10.1137/16M1080173	journal	January 2018
EOF Calculations and Data Filling from Incomplete Oceanographic Datasets* Beckers, J. M.; Rixen, M. Journal of Atmospheric and Oceanic Technology, Vol. 20, Issue 12 https://doi.org/10.1175/1520-0426(2003)020<1839:ECADFF>2.0.CO;2	journal	December 2003
Aerodynamic Data Reconstruction and Inverse Design Using Proper Orthogonal Decomposition Bui-Thanh, T.; Damodaran, M.; Willcox, K. AIAA Journal, Vol. 42, Issue 8 https://doi.org/10.2514/1.2159	journal	August 2004