Entropy-based closure for probabilistic learning on manifolds

Soize, C.; Ghanem, R.; Safta, C.; Huan, X.; Vane, Z. P.; Oefelein, J.; Lacaze, G.; Najm, H. N.; Tang, Q.; Chen, X.

doi:10.1016/j.jcp.2018.12.029

Title: Entropy-based closure for probabilistic learning on manifolds

Abstract

Abstract not provided.

Authors:

Soize, C. ^[1]; Ghanem, R. ^[2]; Safta, C. ^[3]; Huan, X. ^[3]; Vane, Z. P. ^[3]; Oefelein, J. ^[3]; Lacaze, G. ^[3]; Najm, H. N. ^[3]; Tang, Q. ^[4]; Chen, X. ^[4]

Univ. Paris, Marne-La-Vallee (France)
Univ. of Southern California, Los Angeles, CA (United States)
Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Publication Date:: Wed Mar 27 00:00:00 EDT 2019

Research Org.:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States); Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA); Defense Advanced Research Projects Agency (DARPA)

OSTI Identifier:: 1526161

Alternate Identifier(s):: OSTI ID: 1529282; OSTI ID: 1775899

Report Number(s):: LLNL-JRNL-744191; SAND-2018-0462J
Journal ID: ISSN 0021-9991; 899147

Grant/Contract Number:: AC52-07NA27344; AC04-94AL85000; AC02-06CH11357

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Computational Physics

Additional Journal Information:: Journal Volume: 388; Journal Issue: C; Journal ID: ISSN 0021-9991

Publisher:: Elsevier

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Soize, C., Ghanem, R., Safta, C., Huan, X., Vane, Z. P., Oefelein, J., Lacaze, G., Najm, H. N., Tang, Q., and Chen, X. Entropy-based closure for probabilistic learning on manifolds.  United States: N. p., 2019. 
Web.  doi:10.1016/j.jcp.2018.12.029.

Copy to clipboard


                    Soize, C., Ghanem, R., Safta, C., Huan, X., Vane, Z. P., Oefelein, J., Lacaze, G., Najm, H. N., Tang, Q., & Chen, X. Entropy-based closure for probabilistic learning on manifolds.  United States.  https://doi.org/10.1016/j.jcp.2018.12.029

Copy to clipboard


                    Soize, C., Ghanem, R., Safta, C., Huan, X., Vane, Z. P., Oefelein, J., Lacaze, G., Najm, H. N., Tang, Q., and Chen, X. Wed .  
"Entropy-based closure for probabilistic learning on manifolds".  United States.  https://doi.org/10.1016/j.jcp.2018.12.029.  https://www.osti.gov/servlets/purl/1526161.

Copy to clipboard


                    
@article{osti_1526161,

  title        = {Entropy-based closure for probabilistic learning on manifolds},

  author       = {Soize, C. and Ghanem, R. and Safta, C. and Huan, X. and Vane, Z. P. and Oefelein, J. and Lacaze, G. and Najm, H. N. and Tang, Q. and Chen, X.},

  abstractNote = {Abstract not provided.},

  doi          = {10.1016/j.jcp.2018.12.029},

  journal      = {Journal of Computational Physics},

  number       = C,

  volume       = 388,

  place        = {United States},

  year         = {Wed Mar 27 00:00:00 EDT 2019},

  month        = {Wed Mar 27 00:00:00 EDT 2019}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (Publisher)

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1016/j.jcp.2018.12.029

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 13 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Data-driven probability concentration and sampling on manifold
journal, September 2016

Soize, C.; Ghanem, R.
Journal of Computational Physics, Vol. 321
DOI: 10.1016/j.jcp.2016.05.044

Polynomial chaos representation of databases on manifolds
journal, April 2017

Soize, C.; Ghanem, R.
Journal of Computational Physics, Vol. 335
DOI: 10.1016/j.jcp.2017.01.031

Sparse regression learning by aggregation and Langevin Monte-Carlo
journal, September 2012

Dalalyan, A. S.; Tsybakov, A. B.
Journal of Computer and System Sciences, Vol. 78, Issue 5
DOI: 10.1016/j.jcss.2011.12.023

Probabilistic machine learning and artificial intelligence
journal, May 2015

Ghahramani, Zoubin
Nature, Vol. 521, Issue 7553
DOI: 10.1038/nature14541

Statistical learning and selective inference
journal, June 2015

Taylor, Jonathan; Tibshirani, Robert J.
Proceedings of the National Academy of Sciences, Vol. 112, Issue 25
DOI: 10.1073/pnas.1507583112

Efficient Global Optimization of Expensive Black-Box Functions
journal, January 1998

Jones, Donald R.; Schonlau, Matthias; Welch, William J.
Journal of Global Optimization, Vol. 13, Issue 4, p. 455-492
DOI: 10.1023/A:1008306431147

Surrogate-based analysis and optimization
journal, January 2005

Queipo, Nestor V.; Haftka, Raphael T.; Shyy, Wei
Progress in Aerospace Sciences, Vol. 41, Issue 1
DOI: 10.1016/j.paerosci.2005.02.001

On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning
journal, July 2011

Byrd, Richard H.; Chin, Gillian M.; Neveitt, Will
SIAM Journal on Optimization, Vol. 21, Issue 3
DOI: 10.1137/10079923X

Constrained optimization in expensive simulation: Novel approach
journal, April 2010

Kleijnen, Jack P. C.; Beers, Wim van; Nieuwenhuyse, Inneke van
European Journal of Operational Research, Vol. 202, Issue 1
DOI: 10.1016/j.ejor.2009.05.002

Bayesian Optimization in a Billion Dimensions via Random Embeddings
journal, January 2016

Wang, Ziyu; Hutter, Frank; Zoghi, Masrour
Journal of Artificial Intelligence Research, Vol. 55
DOI: 10.1613/jair.4806

Bayesian Optimization via Simulation with Pairwise Sampling and Correlated Prior Beliefs
journal, April 2016

Xie, Jing; Frazier, Peter I.; Chick, Stephen E.
Operations Research, Vol. 64, Issue 2
DOI: 10.1287/opre.2016.1480

Sequential Optimization and Reliability Assessment Method for Efficient Probabilistic Design
journal, March 2004

Du, Xiaoping; Chen, Wei
Journal of Mechanical Design, Vol. 126, Issue 2
DOI: 10.1115/1.1649968

Review of uncertainty-based multidisciplinary design optimization methods for aerospace vehicles
journal, August 2011

Yao, Wen; Chen, Xiaoqian; Luo, Wencai
Progress in Aerospace Sciences, Vol. 47, Issue 6
DOI: 10.1016/j.paerosci.2011.05.001

Probabilistic nonconvex constrained optimization with fixed number of function evaluations: Probabilistic nonconvex constrained optimization with fixed number of function evaluations
journal, September 2017

Ghanem, Roger; Soize, Christian
International Journal for Numerical Methods in Engineering, Vol. 113, Issue 4
DOI: 10.1002/nme.5632

Optimal Well-Placement Using Probabilistic Learning
journal, January 2018

Ghanem, Roger; Soize, Christian; Thimmisetty, Charanraj
Data-Enabled Discovery and Applications, Vol. 2, Issue 1
DOI: 10.1007/s41688-017-0014-x

Homogeneous chaos basis adaptation for design optimization under uncertainty: Application to the oil well placement problem
journal, August 2017

Thimmisetty, Charanraj; Tsilifis, Panagiotis; Ghanem, Roger
Artificial Intelligence for Engineering Design, Analysis and Manufacturing, Vol. 31, Issue 3
DOI: 10.1017/S0890060417000166

The atmospheric hydrologic cycle in the ACME v0.3 model
journal, July 2017

Terai, Christopher R.; Caldwell, Peter M.; Klein, Stephen A.
Climate Dynamics, Vol. 50, Issue 9-10
DOI: 10.1007/s00382-017-3803-x

Polynomial Chaos Expansion of a Multimodal Random Vector
journal, January 2015

Soize, C.
SIAM/ASA Journal on Uncertainty Quantification, Vol. 3, Issue 1
DOI: 10.1137/140968495

Construction of probability distributions in high dimension using the maximum entropy principle: Applications to stochastic processes, random fields and random matrices
journal, December 2008

Soize, Christian
International Journal for Numerical Methods in Engineering, Vol. 76, Issue 10
DOI: 10.1002/nme.2385

Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
journal, March 2011

Girolami, Mark; Calderhead, Ben
Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 2
DOI: 10.1111/j.1467-9868.2010.00765.x

Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps
journal, May 2005

Coifman, R. R.; Lafon, S.; Lee, A. B.
Proceedings of the National Academy of Sciences, Vol. 102, Issue 21
DOI: 10.1073/pnas.0500334102

Diffusion maps
journal, July 2006

Coifman, Ronald R.; Lafon, Stéphane
Applied and Computational Harmonic Analysis, Vol. 21, Issue 1
DOI: 10.1016/j.acha.2006.04.006

Information Theory and Statistical Mechanics
journal, May 1957

Jaynes, E. T.
Physical Review, Vol. 106, Issue 4
DOI: 10.1103/PhysRev.106.620

Information Theory and Statistical Mechanics. II
journal, October 1957

Jaynes, E. T.
Physical Review, Vol. 108, Issue 2
DOI: 10.1103/PhysRev.108.171

A Mathematical Theory of Communication
journal, July 1948

Shannon, C. E.
Bell System Technical Journal, Vol. 27, Issue 3
DOI: 10.1002/j.1538-7305.1948.tb01338.x

Construction of probability distributions in high dimension using the maximum entropy principle: Applications to stochastic processes, random fields and random matrices
journal, December 2008

Soize, Christian
International Journal for Numerical Methods in Engineering, Vol. 76, Issue 10
DOI: 10.1002/nme.2385

Constrained optimization in expensive simulation: Novel approach
journal, April 2010

Kleijnen, Jack P. C.; Beers, Wim van; Nieuwenhuyse, Inneke van
European Journal of Operational Research, Vol. 202, Issue 1
DOI: 10.1016/j.ejor.2009.05.002

Sparse regression learning by aggregation and Langevin Monte-Carlo
journal, September 2012

Dalalyan, A. S.; Tsybakov, A. B.
Journal of Computer and System Sciences, Vol. 78, Issue 5
DOI: 10.1016/j.jcss.2011.12.023

Surrogate-based analysis and optimization
journal, January 2005

Queipo, Nestor V.; Haftka, Raphael T.; Shyy, Wei
Progress in Aerospace Sciences, Vol. 41, Issue 1
DOI: 10.1016/j.paerosci.2005.02.001

Monte Carlo sampling-based methods for stochastic optimization
journal, January 2014

Homem-de-Mello, Tito; Bayraksan, Güzin
Surveys in Operations Research and Management Science, Vol. 19, Issue 1
DOI: 10.1016/j.sorms.2014.05.001

Statistical Active Learning Algorithms for Noise Tolerance and Differential Privacy
text, January 2014

Balcan, Maria-Florina; Feldman, Vitaly
Carnegie Mellon University
DOI: 10.1184/r1/6476360

Works referencing / citing this record:

Physics‐constrained non‐Gaussian probabilistic learning on manifolds
journal, September 2019

Soize, Christian; Ghanem, Roger
International Journal for Numerical Methods in Engineering, Vol. 121, Issue 1
DOI: 10.1002/nme.6202

Design optimization of a scramjet under uncertainty using probabilistic learning on manifolds
journal, December 2019

Ghanem, R. G.; Soize, C.; Safta, C.
Journal of Computational Physics, Vol. 399
DOI: 10.1016/j.jcp.2019.108930

Machine learning for detecting structural changes from dynamic monitoring using the probabilistic learning on manifolds
journal, September 2020

Soize, Christian; Orcesi, André
Structure and Infrastructure Engineering, Vol. 17, Issue 10
DOI: 10.1080/15732479.2020.1811991

Similar Records in DOE PAGES and OSTI.GOV collections:

Design optimization of a scramjet under uncertainty using probabilistic learning on manifolds

Journal Article Ghanem, R. G. ; Soize, C. ; Safta, C. ; ... - Journal of Computational Physics

Here, we demonstrate, on a scramjet combustion problem, a constrained probabilistic learning approach that augments physics-based datasets with realizations that adhere to underlying constraints and scatter. The constraints are captured and delineated through diffusion maps, while the scatter is captured and sampled through a projected stochastic differential equation. The objective function and constraints of the optimization problem are then efficiently framed as non-parametric conditional expectations. Different spatial resolutions of a large-eddy simulation filter are used to explore the robustness of the model to the training dataset and to gain insight into the significance of spatial resolution on optimal design.
https://doi.org/10.1016/j.jcp.2019.108930

Full Text Available
Enhancing Model Predictability for a Scramjet Using Probabilistic Learning on Manifolds

Journal Article Soize, Christian ; Ghanem, Roger ; Safta, Cosmin ; ... - AIAA Journal

In this study, the computational burden of a large-eddy simulation for reactive flows is exacerbated in the presence of uncertainty in flow conditions or kinetic variables. A comprehensive statistical analysis, with a sufficiently large number of samples, remains elusive. Statistical learning is an approach that allows for extracting more information using fewer samples. Such procedures, if successful, will greatly enhance the predictability of models in the sense of improving exploration and characterization of uncertainty due to model error and input dependencies, all while being constrained by the size of the associated statistical samples. In this paper, it is shown howmore »« less
https://doi.org/10.2514/1.j057069
Trajectory design via unsupervised probabilistic learning on optimal manifolds

Journal Article Safta, Cosmin ; Ghanem, Roger G. ; Grant, Michael J. ; ... - Data-Centric Engineering

Abstract This article illustrates the use of unsupervised probabilistic learning techniques for the analysis of planetary reentry trajectories. A three-degree-of-freedom model was employed to generate optimal trajectories that comprise the training datasets. The algorithm first extracts the intrinsic structure in the data via a diffusion map approach. We find that data resides on manifolds of much lower dimensionality compared to the high-dimensional state space that describes each trajectory. Using the diffusion coordinates on the graph of training samples, the probabilistic framework subsequently augments the original data with samples that are statistically consistent with the original set. The augmented samples aremore »« less
https://doi.org/10.1017/dce.2022.26
A Probabilistic Modeling Framework for Forecasting Gas Hydrate Using Geospatial Machine Learning.

Conference Frederick, Jennifer M ; Eymold, William Karl ; Nole, Michael ; ...

Abstract not provided.
Full Text Available
Sparse probabilistic learning of nonlinear differential equations.

Conference Sandhu, Rimple ; Khalil, Mohammad ; Pettit, Chris ; ...

Abstract not provided.
Full Text Available

Similar Records

Title: Entropy-based closure for probabilistic learning on manifolds

Abstract

Citation Formats

Data-driven probability concentration and sampling on manifold journal, September 2016

Polynomial chaos representation of databases on manifolds journal, April 2017

Sparse regression learning by aggregation and Langevin Monte-Carlo journal, September 2012

Probabilistic machine learning and artificial intelligence journal, May 2015

Statistical learning and selective inference journal, June 2015

Efficient Global Optimization of Expensive Black-Box Functions journal, January 1998

Surrogate-based analysis and optimization journal, January 2005

On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning journal, July 2011

Constrained optimization in expensive simulation: Novel approach journal, April 2010

Bayesian Optimization in a Billion Dimensions via Random Embeddings journal, January 2016

Bayesian Optimization via Simulation with Pairwise Sampling and Correlated Prior Beliefs journal, April 2016

Sequential Optimization and Reliability Assessment Method for Efficient Probabilistic Design journal, March 2004

Review of uncertainty-based multidisciplinary design optimization methods for aerospace vehicles journal, August 2011

Probabilistic nonconvex constrained optimization with fixed number of function evaluations: Probabilistic nonconvex constrained optimization with fixed number of function evaluations journal, September 2017

Optimal Well-Placement Using Probabilistic Learning journal, January 2018

Homogeneous chaos basis adaptation for design optimization under uncertainty: Application to the oil well placement problem journal, August 2017

The atmospheric hydrologic cycle in the ACME v0.3 model journal, July 2017

Polynomial Chaos Expansion of a Multimodal Random Vector journal, January 2015

Construction of probability distributions in high dimension using the maximum entropy principle: Applications to stochastic processes, random fields and random matrices journal, December 2008

Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods journal, March 2011

Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps journal, May 2005

Diffusion maps journal, July 2006

Information Theory and Statistical Mechanics journal, May 1957

Information Theory and Statistical Mechanics. II journal, October 1957

A Mathematical Theory of Communication journal, July 1948

Construction of probability distributions in high dimension using the maximum entropy principle: Applications to stochastic processes, random fields and random matrices journal, December 2008

Constrained optimization in expensive simulation: Novel approach journal, April 2010

Sparse regression learning by aggregation and Langevin Monte-Carlo journal, September 2012

Surrogate-based analysis and optimization journal, January 2005

Monte Carlo sampling-based methods for stochastic optimization journal, January 2014

Statistical Active Learning Algorithms for Noise Tolerance and Differential Privacy text, January 2014

Physics‐constrained non‐Gaussian probabilistic learning on manifolds journal, September 2019

Design optimization of a scramjet under uncertainty using probabilistic learning on manifolds journal, December 2019

Machine learning for detecting structural changes from dynamic monitoring using the probabilistic learning on manifolds journal, September 2020

Data-driven probability concentration and sampling on manifold
journal, September 2016

Polynomial chaos representation of databases on manifolds
journal, April 2017

Sparse regression learning by aggregation and Langevin Monte-Carlo
journal, September 2012

Probabilistic machine learning and artificial intelligence
journal, May 2015

Statistical learning and selective inference
journal, June 2015

Efficient Global Optimization of Expensive Black-Box Functions
journal, January 1998

Surrogate-based analysis and optimization
journal, January 2005

On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning
journal, July 2011

Constrained optimization in expensive simulation: Novel approach
journal, April 2010

Bayesian Optimization in a Billion Dimensions via Random Embeddings
journal, January 2016

Bayesian Optimization via Simulation with Pairwise Sampling and Correlated Prior Beliefs
journal, April 2016

Sequential Optimization and Reliability Assessment Method for Efficient Probabilistic Design
journal, March 2004

Review of uncertainty-based multidisciplinary design optimization methods for aerospace vehicles
journal, August 2011

Probabilistic nonconvex constrained optimization with fixed number of function evaluations: Probabilistic nonconvex constrained optimization with fixed number of function evaluations
journal, September 2017

Optimal Well-Placement Using Probabilistic Learning
journal, January 2018

Homogeneous chaos basis adaptation for design optimization under uncertainty: Application to the oil well placement problem
journal, August 2017

The atmospheric hydrologic cycle in the ACME v0.3 model
journal, July 2017

Polynomial Chaos Expansion of a Multimodal Random Vector
journal, January 2015

Construction of probability distributions in high dimension using the maximum entropy principle: Applications to stochastic processes, random fields and random matrices
journal, December 2008

Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
journal, March 2011

Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps
journal, May 2005

Diffusion maps
journal, July 2006

Information Theory and Statistical Mechanics
journal, May 1957

Information Theory and Statistical Mechanics. II
journal, October 1957

A Mathematical Theory of Communication
journal, July 1948

Construction of probability distributions in high dimension using the maximum entropy principle: Applications to stochastic processes, random fields and random matrices
journal, December 2008

Constrained optimization in expensive simulation: Novel approach
journal, April 2010

Sparse regression learning by aggregation and Langevin Monte-Carlo
journal, September 2012

Surrogate-based analysis and optimization
journal, January 2005

Monte Carlo sampling-based methods for stochastic optimization
journal, January 2014

Statistical Active Learning Algorithms for Noise Tolerance and Differential Privacy
text, January 2014

Physics‐constrained non‐Gaussian probabilistic learning on manifolds
journal, September 2019

Design optimization of a scramjet under uncertainty using probabilistic learning on manifolds
journal, December 2019

Machine learning for detecting structural changes from dynamic monitoring using the probabilistic learning on manifolds
journal, September 2020