Neural networks for parameter estimation in intractable models

Lenzi, Amanda; Bessac, Julie; Rudi, Johann; Stein, Michael L.

doi:10.1016/j.csda.2023.107762

Neural networks for parameter estimation in intractable models

Journal Article · Fri Apr 14 00:00:00 EDT 2023 · Computational Statistics and Data Analysis (Print)

DOI:https://doi.org/10.1016/j.csda.2023.107762· OSTI ID:2429559

^[1]; Bessac, Julie ^[2]; Rudi, Johann ^[3]; Stein, Michael L. ^[4]

Univ. of Edinburgh (United Kingdom)
Argonne National Laboratory (ANL), Argonne, IL (United States); National Renewable Energy Laboratory (NREL), Golden, CO (United States)
Argonne National Laboratory (ANL), Argonne, IL (United States); Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)
Rutgers Univ., Piscataway, NJ (United States)

The goal is to use deep learning models to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. For instance, inference for max-stable processes is exceptionally challenging even with small datasets, but simulation is straightforward. Data from model simulations are used to train deep neural networks and learn statistical parameters from max-stable models. The proposed neural network-based method provides a competitive alternative to current approaches, as demonstrated by considerable accuracy and computational time improvements. Finally, it serves as a proof of concept for deep learning in statistical parameter estimation and can be extended to other estimation problems.

View Accepted Manuscript (DOE)

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC)

Grant/Contract Number:: AC02-06CH11357

OSTI ID:: 2429559

Alternate ID(s):: OSTI ID: 2331353

Journal Information:: Computational Statistics and Data Analysis (Print), Journal Name: Computational Statistics and Data Analysis (Print) Vol. 185; ISSN 0167-9473

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (26)

Full likelihood inference for max‐stable data Huser, Raphaël; Dombry, Clément; Ribatet, Mathieu Stat, Vol. 8, Issue 1 https://doi.org/10.1002/sta4.218	journal	January 2019
Likelihood estimators for multivariate extremes Huser, Raphaël; Davison, Anthony C.; Genton, Marc G. Extremes, Vol. 19, Issue 1 https://doi.org/10.1007/s10687-015-0230-4	journal	November 2015
Reliability bounds through random sets: Non-parametric methods and geotechnical applications Oberguggenberger, Michael; Fellin, Wolfgang Computers & Structures, Vol. 86, Issue 10 https://doi.org/10.1016/j.compstruc.2007.05.040	journal	May 2008
Approximate Bayesian computing for spatial extremes Erhardt, Robert J.; Smith, Richard L. Computational Statistics & Data Analysis, Vol. 56, Issue 6 https://doi.org/10.1016/j.csda.2011.12.003	journal	June 2012
A review of uncertainty quantification in deep learning: Techniques, applications and challenges Abdar, Moloud; Pourpanah, Farhad; Hussain, Sadiq Information Fusion, Vol. 76 https://doi.org/10.1016/j.inffus.2021.05.008	journal	December 2021
On the robustness of maximum composite likelihood estimate Xu, Ximing; Reid, N. Journal of Statistical Planning and Inference, Vol. 141, Issue 9 https://doi.org/10.1016/j.jspi.2011.03.026	journal	September 2011
The multi-institution North American Land Data Assimilation System (NLDAS): Utilizing multiple GCIP products and partners in a continental distributed hydrological modeling system Mitchell, Kenneth E. Journal of Geophysical Research, Vol. 109, Issue D7 https://doi.org/10.1029/2003JD003823	journal	January 2004
Statistical inference for noisy nonlinear ecological dynamic systems Wood, Simon N. Nature, Vol. 466, Issue 7310 https://doi.org/10.1038/nature09319	journal	August 2010
The frontier of simulation-based inference Cranmer, Kyle; Brehmer, Johann; Louppe, Gilles Proceedings of the National Academy of Sciences, Vol. 117, Issue 48 https://doi.org/10.1073/pnas.1912789117	journal	May 2020
Estimation and Prediction in Spatial Models With Block Composite Likelihoods Eidsvik, Jo; Shaby, Benjamin A.; Reich, Brian J. Journal of Computational and Graphical Statistics, Vol. 23, Issue 2 https://doi.org/10.1080/10618600.2012.760460	journal	April 2014
High-Order Composite Likelihood Inference for Max-Stable Distributions and Processes Castruccio, Stefano; Huser, Raphaël; Genton, Marc G. Journal of Computational and Graphical Statistics, Vol. 25, Issue 4 https://doi.org/10.1080/10618600.2015.1086656	journal	October 2016
Bivariate extreme value theory: Models and estimation Tawn, Jonathan A. Biometrika, Vol. 75, Issue 3 https://doi.org/10.1093/biomet/75.3.397	journal	January 1988
Exploiting occurrence times in likelihood inference for componentwise maxima Stephenson, Alec; Tawn, Jonathan Biometrika, Vol. 92, Issue 1 https://doi.org/10.1093/biomet/92.1.213	journal	March 2005
Adaptive approximate Bayesian computation Beaumont, M. A.; Cornuet, J. -M.; Marin, J. -M. Biometrika, Vol. 96, Issue 4 https://doi.org/10.1093/biomet/asp052	journal	October 2009
On the likelihood function of Gaussian max-stable processes Genton, M. G.; Ma, Y.; Sang, H. Biometrika, Vol. 98, Issue 2 https://doi.org/10.1093/biomet/asr020	journal	May 2011
Efficient inference and simulation for elliptical Pareto processes Thibaud, Emeric; Opitz, Thomas Biometrika, Vol. 102, Issue 4 https://doi.org/10.1093/biomet/asv045	journal	October 2015
Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation: Semi-automatic Approximate Bayesian Computation Fearnhead, Paul; Prangle, Dennis Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 74, Issue 3 https://doi.org/10.1111/j.1467-9868.2011.01010.x	journal	May 2012
Statistical Inference for Max-Stable Processes in Space and Time Davis, Richard A.; Klüppelberg, Claudia; Steinkohl, Christina Journal of the Royal Statistical Society Series B: Statistical Methodology, Vol. 75, Issue 5 https://doi.org/10.1111/rssb.12012	journal	March 2013
Changes in the Extremes in an Ensemble of Transient Climate Simulations with a Coupled Atmosphere–Ocean GCM Kharin, Viatcheslav V.; Zwiers, Francis W. Journal of Climate, Vol. 13, Issue 21 https://doi.org/10.1175/1520-0442(2000)013<3760:CITEIA>2.0.CO;2	journal	November 2000
Strictly Proper Scoring Rules, Prediction, and Estimation Gneiting, Tilmann; Raftery, Adrian E. Journal of the American Statistical Association, Vol. 102, Issue 477 https://doi.org/10.1198/016214506000001437	journal	March 2007
Likelihood-Based Inference for Max-Stable Processes Padoan, S. A.; Ribatet, M.; Sisson, S. A. Journal of the American Statistical Association, Vol. 105, Issue 489 https://doi.org/10.1198/jasa.2009.tm08577	journal	March 2010
Stationary max-stable fields associated to negative definite functions Kabluchko, Zakhar; Schlather, Martin; de Haan, Laurens The Annals of Probability, Vol. 37, Issue 5 https://doi.org/10.1214/09-AOP455	journal	September 2009
Statistical Modeling of Spatial Extremes Davison, A. C.; Padoan, S. A.; Ribatet, M. Statistical Science, Vol. 27, Issue 2 https://doi.org/10.1214/11-STS376	journal	May 2012
Bayesian inference for the Brown–Resnick process, with an application to extreme low temperatures Thibaud, Emeric; Aalto, Juha; Cooley, Daniel S. The Annals of Applied Statistics, Vol. 10, Issue 4 https://doi.org/10.1214/16-AOAS980	journal	December 2016
Scalable Bayesian Inference for the Inverse Temperature of a Hidden Potts Model Moores, Matthew; Nicholls, Geoff; Pettitt, Anthony Bayesian Analysis, Vol. 15, Issue 1 https://doi.org/10.1214/18-BA1130	journal	March 2020
Estimating changes in temperature extremes from millennial-scale climate simulations using generalized extreme value (GEV) distributions Huang, Whitney K.; Stein, Michael L.; McInerney, David J. Advances in Statistical Climatology, Meteorology and Oceanography, Vol. 2, Issue 1 https://doi.org/10.5194/ascmo-2-79-2016	journal	January 2016

Similar Records

Physics-constrained deep neural network method for estimating parameters in a redox flow battery

Journal Article · Sun Mar 06 19:00:00 EST 2022 · Journal of Power Sources · OSTI ID:1862844

On Transfer Learning Of Neural Networks Using Bi-Fidelity Data For Uncertainty Propagation

Journal Article · Tue Dec 31 19:00:00 EST 2019 · International Journal for Uncertainty Quantification · OSTI ID:1769854

Computationally Efficient Training of Deep Neural Networks via Transfer Learning

Journal Article · Mon May 13 20:00:00 EDT 2019 · Proceedings of SPIE - The International Society for Optical Engineering · OSTI ID:1565903

Related Subjects

97 MATHEMATICS AND COMPUTING
deep neural networks
intractable likelihood
max-stable distributions
parameter estimation

Neural networks for parameter estimation in intractable models

Citation Formats

References (26)

Similar Records

Related Subjects