Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems

Yang, Yibo; Perdikaris, Paris

doi:10.1007/s00466-019-01718-y

Title: Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems

Journal Article · 21 May 2019 · Computational Mechanics

DOI: https://doi.org/10.1007/s00466-019-01718-y · OSTI ID:1595804

Yang, Yibo ^[1];

^[1]

Univ. of Pennsylvania, Philadelphia, PA (United States)

Here, we present a probabilistic deep learning methodology that enables the construction of predictive data-driven surrogates for stochastic systems. Leveraging recent advances in variational inference with implicit distributions, we put forth a statistical inference framework that enables the end-to-end training of surrogate models on paired input–output observations that may be stochastic in nature, originate from different information sources of variable fidelity, or be corrupted by complex noise processes. The resulting surrogates can accommodate high-dimensional inputs and outputs and are able to return predictions with quantified uncertainty. Furthermore, the effectiveness our approach is demonstrated through a series of canonical studies, including the regression of noisy data, multi-fidelity modeling of stochastic processes, and uncertainty propagation in high-dimensional dynamical systems.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Univ. of Pennsylvania, Philadelphia, PA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: SC0019116; HR00111890034

OSTI ID:: 1595804

Journal Information:: Computational Mechanics, Vol. 64, Issue 2; Related Information: https://github.com/PredictiveIntelligenceLab/CADGMs; ISSN 0178-7675

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 21 works

Citation information provided by
Web of Science

References (46)

Multi-fidelity Gaussian process regression for prediction of random fields Parussini, L.; Venturi, D.; Perdikaris, P. Journal of Computational Physics, Vol. 336 https://doi.org/10.1016/j.jcp.2017.01.047	journal	May 2017
Multi-fidelity Gaussian process regression for prediction of random fields Parussini, L.; Venturi, D.; Perdikaris, P. Journal of Computational Physics, Vol. 336 https://doi.org/10.1016/j.jcp.2017.01.047	journal	May 2017
Active Learning with Statistical Models Cohn, D. A.; Ghahramani, Z.; Jordan, M. I. Journal of Artificial Intelligence Research, Vol. 4 https://doi.org/10.1613/jair.295	journal	January 1996
DDDAS-based multi-fidelity simulation framework for supply chain systems Celik, Nurcin; Lee, Seungho; Vasudevan, Karthik IIE Transactions, Vol. 42, Issue 5 https://doi.org/10.1080/07408170903394306	journal	February 2010
Surrogate-Based Optimization Using Multifidelity Models with Variable Parameterization and Corrected Space Mapping Robinson, T. D.; Eldred, M. S.; Willcox, K. E. AIAA Journal, Vol. 46, Issue 11 https://doi.org/10.2514/1.36043	journal	November 2008
Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Bayesian inference with optimal maps El Moselhy, Tarek A.; Marzouk, Youssef M. Journal of Computational Physics, Vol. 231, Issue 23 https://doi.org/10.1016/j.jcp.2012.07.022	journal	October 2012
Multifidelity Information Fusion Algorithms for High-Dimensional Systems and Massive Data sets Perdikaris, Paris; Venturi, Daniele; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 38, Issue 4 https://doi.org/10.1137/15M1055164	journal	January 2016
Information Theory and an Extension of the Maximum Likelihood Principle Akaike, Hirotogu Springer Series in Statistics https://doi.org/10.1007/978-1-4612-1694-0_15	book	January 1998
Deep Learning Ganatra, Sanket Zenodo https://doi.org/10.5281/zenodo.1189359	text	January 2018
Multi-Fidelity Uncertainty Quantification: Application to a Vertical Axis Wind Turbine Under an Extreme Gust Padron, Andres S.; Alonso, Juan J.; Palacios, Francisco 15th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference https://doi.org/10.2514/6.2014-3013	conference	June 2014
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules. Gómez-Bombarelli, Rafael; Wei, Jennifer N.; Duvenaud, David Apollo - University of Cambridge Repository https://doi.org/10.17863/cam.34134	text	January 2018
Predictive collective variable discovery with deep Bayesian models Schöberl, Markus; Zabaras, Nicholas; Koutsourelakis, Phaedon-Stelios The Journal of Chemical Physics, Vol. 150, Issue 2 https://doi.org/10.1063/1.5058063	journal	January 2019
Model inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond Perdikaris, Paris; Karniadakis, George Em Journal of The Royal Society Interface, Vol. 13, Issue 118 https://doi.org/10.1098/rsif.2015.1107	journal	May 2016
A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime Grigo, Constantin; Koutsourelakis, Phaedon-Stelios Journal of Computational Physics, Vol. 397 https://doi.org/10.1016/j.jcp.2019.05.053	journal	November 2019
Understanding deep convolutional networks Mallat, Stéphane Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 374, Issue 2065 https://doi.org/10.1098/rsta.2015.0203	journal	April 2016
Variational Inference: A Review for Statisticians Blei, David M.; Kucukelbir, Alp; McAuliffe, Jon D. Journal of the American Statistical Association, Vol. 112, Issue 518 https://doi.org/10.1080/01621459.2017.1285773	journal	July 2016
Active Learning with Statistical Models Cohn, D. A.; Ghahramani, Z.; Jordan, M. I. arXiv https://doi.org/10.48550/arxiv.cs/9603104	text	January 1996
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules Gómez-Bombarelli, Rafael; Wei, Jennifer N.; Duvenaud, David ACS Central Science, Vol. 4, Issue 2 https://doi.org/10.1021/acscentsci.7b00572	journal	January 2018
Surrogate-Based Optimization Using Multifidelity Models with Variable Parameterization and Corrected Space Mapping Robinson, T. D.; Eldred, M. S.; Willcox, K. E. AIAA Journal, Vol. 46, Issue 11 https://doi.org/10.2514/1.36043	journal	November 2008
Comparison of Non-Intrusive Polynomial Chaos and Stochastic Collocation Methods for Uncertainty Quantification Eldred, Michael; Burkardt, John 47th AIAA Aerospace Sciences Meeting including The New Horizons Forum and Aerospace Exposition https://doi.org/10.2514/6.2009-976	conference	June 2012
A Stochastic Collocation Algorithm with Multifidelity Models Narayan, Akil; Gittelson, Claude; Xiu, Dongbin SIAM Journal on Scientific Computing, Vol. 36, Issue 2 https://doi.org/10.1137/130929461	journal	January 2014
Optimal Model Management for Multifidelity Monte Carlo Estimation Peherstorfer, Benjamin; Willcox, Karen; Gunzburger, Max SIAM Journal on Scientific Computing, Vol. 38, Issue 5 https://doi.org/10.1137/15M1046472	journal	January 2016
Multifidelity importance sampling Peherstorfer, Benjamin; Cui, Tiangang; Marzouk, Youssef Computer Methods in Applied Mechanics and Engineering, Vol. 300 https://doi.org/10.1016/j.cma.2015.12.002	journal	March 2016
Multi-fidelity optimization via surrogate modelling Forrester, Alexander I. J.; Sóbester, András; Keane, Andy J. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 463, Issue 2088 https://doi.org/10.1098/rspa.2007.1900	journal	September 2007
Towards efficient uncertainty quantification in complex and large-scale biomechanical problems based on a Bayesian multi-fidelity scheme Biehler, Jonas; Gee, Michael W.; Wall, Wolfgang A. Biomechanics and Modeling in Mechanobiology, Vol. 14, Issue 3 https://doi.org/10.1007/s10237-014-0618-0	journal	September 2014
A two-stage multi-fidelity optimization procedure for honeycomb-type cellular materials Sun, Guangyong; Li, Guangyao; Stone, Michael Computational Materials Science, Vol. 49, Issue 3 https://doi.org/10.1016/j.commatsci.2010.05.041	journal	September 2010
Taking the Human Out of the Loop: A Review of Bayesian Optimization Shahriari, Bobak; Swersky, Kevin; Wang, Ziyu Proceedings of the IEEE, Vol. 104, Issue 1 https://doi.org/10.1109/JPROC.2015.2494218	journal	January 2016
Graphical Models, Exponential Families, and Variational Inference Wainwright, Martin J.; Jordan, Michael I. Foundations and Trends® in Machine Learning, Vol. 1, Issue 1–2 https://doi.org/10.1561/2200000001	journal	January 2007
Approximation and Model Management in Aerodynamic Optimization with Variable-Fidelity Models Alexandrov, Natalia M.; Lewis, Robert Michael; Gumbert, Clyde R. Journal of Aircraft, Vol. 38, Issue 6 https://doi.org/10.2514/2.2877	journal	November 2001
Computational Aspects of Stochastic Collocation with Multifidelity Models Zhu, Xueyu; Narayan, Akil; Xiu, Dongbin SIAM/ASA Journal on Uncertainty Quantification, Vol. 2, Issue 1 https://doi.org/10.1137/130949154	journal	January 2014
Extracting and composing robust features with denoising autoencoders Vincent, Pascal; Larochelle, Hugo; Bengio, Yoshua Proceedings of the 25th international conference on Machine learning - ICML '08 https://doi.org/10.1145/1390156.1390294	conference	January 2008
Multi-output separable Gaussian process: Towards an efficient, fully Bayesian paradigm for uncertainty quantification Bilionis, Ilias; Zabaras, Nicholas; Konomi, Bledar A. Journal of Computational Physics, Vol. 241 https://doi.org/10.1016/j.jcp.2013.01.011	journal	May 2013
Variational Inference: A Review for Statisticians Blei, David M.; Kucukelbir, Alp; McAuliffe, Jon D. arXiv https://doi.org/10.48550/arxiv.1601.00670	text	January 2016
Fourth-Order Time-Stepping for Stiff PDEs Kassam, Aly-Khan; Trefethen, Lloyd N. SIAM Journal on Scientific Computing, Vol. 26, Issue 4 https://doi.org/10.1137/S1064827502410633	journal	January 2005
Multi-fidelity optimization for sheet metal forming process Sun, Guangyong; Li, Guangyao; Zhou, Shiwei Structural and Multidisciplinary Optimization, Vol. 44, Issue 1 https://doi.org/10.1007/s00158-010-0596-5	journal	December 2010
Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach Gómez-Bombarelli, Rafael; Aguilera-Iparraguirre, Jorge; Hirzel, Timothy D. Nature Materials, Vol. 15, Issue 10 https://doi.org/10.1038/nmat4717	journal	August 2016
Variational Inference: A Review for Statisticians Blei, David M.; Kucukelbir, Alp; McAuliffe, Jon D. Taylor & Francis https://doi.org/10.6084/m9.figshare.5203696.v1	text	January 2017
A deep generative model for single-cell RNA sequencing with application to detecting differentially expressed genes Lopez, Romain; Regier, Jeffrey; Cole, Michael arXiv https://doi.org/10.48550/arxiv.1710.05086	preprint	January 2017
Data-Driven Parallel Scientific Computing: Multi-Fidelity Information Fusion Algorithms and Applications to Physical and Biological Systems Perdikaris, Paris Brown University https://doi.org/10.7301/z03n21s1	text	January 2015
Variational Inference: A Review for Statisticians Blei, David M.; Kucukelbir, Alp; McAuliffe, Jon D. Journal of the American Statistical Association, Vol. 112, Issue 518 https://doi.org/10.1080/01621459.2017.1285773	journal	July 2016
Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks Mescheder, Lars; Nowozin, Sebastian; Geiger, Andreas arXiv https://doi.org/10.48550/arxiv.1701.04722	preprint	January 2017
Active Learning with Statistical Models Cohn, D. A.; Ghahramani, Z.; Jordan, M. I. arXiv https://doi.org/10.48550/arxiv.cs/9603104	text	January 1996
Predicting the output from a complex computer code when fast approximations are available Kennedy, M. Biometrika, Vol. 87, Issue 1 https://doi.org/10.1093/biomet/87.1.1	journal	March 2000
Variational Inference: A Review for Statisticians Blei, David M.; Kucukelbir, Alp; McAuliffe, Jon D. Taylor & Francis https://doi.org/10.6084/m9.figshare.5203696	text	January 2017
Enabling Dark Energy Science with Deep Generative Models of Galaxy Images Ravanbakhsh, Siamak; Lanusse, Francois; Mandelbaum, Rachel Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31, Issue 1 https://doi.org/10.1609/aaai.v31i1.10755	journal	February 2017

Similar Records

Adversarial uncertainty quantification in physics-informed neural networks

Journal Article · 2019 · Journal of Computational Physics · OSTI ID:1595802

Yang, Yibo; Perdikaris, Paris

Multi-fidelity information fusion with concatenated neural networks

Journal Article · 2022 · Scientific Reports · OSTI ID:1904349

Pawar, Suraj; San, Omer; Vedula, Prakash; +2 more

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Journal Article · 2021 · Computer Methods in Applied Mechanics and Engineering · OSTI ID:1976970

Tsilifis, Panagiotis; Pandita, Piyush; Ghosh, Sayan; +3 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Probabilistic deep learning
Generative adversarial networks
Variational inference
Multi- delity modeling
Data-driven surrogates

Title: Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems

Citation Formats

References (46)

Similar Records

Related Subjects