Rare Events via Cross-Entropy Population Monte Carlo

Miller, Caleb; Corcoran, Jem N.; Schneider, Michael D.

doi:10.1109/lsp.2021.3139572

Title: Rare Events via Cross-Entropy Population Monte Carlo

Journal Article · 31 December 2021 · IEEE Signal Processing Letters

DOI: https://doi.org/10.1109/lsp.2021.3139572 · OSTI ID:1860785

^[1];

^[1]; Schneider, Michael D. ^[2]

Univ. of Colorado, Boulder, CO (United States)
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Rare events are events that happen with very low frequency. Estimating rare event probabilities using Monte Carlo techniques is computationally expensive, often to the point of intractability, and special methods are required. Importance sampling (IS) is a well known technique that uses a proposal distribution in place of a target distribution to lower the variance of the estimator. Key to the success of IS methods is the choice of a proposal distribution, or the parameters governing the distribution. Adaptive importance sampling improves the parameters of a family or population of proposal distributions iteratively through trials. We present a novel cross-entropy population Monte Carlo algorithm, which adapts the parameters of proposals through the cross-entropy method. The proposed method stands apart from previous work in that we are not optimizing a mixture distribution. Instead, we leverage deterministic mixture weights and optimize the distributions individually through a reinterpretation of the typical derivation of the cross-entropy method. Demonstrations on rare event examples show that the algorithm can outperform existing resampling based population Monte Carlo methods, especially for higher-dimensional problems. Finally, we also demonstrate efficacy on a conjunction analysis problem.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC52-07NA27344

OSTI ID:: 1860785

Report Number(s):: LLNL-JRNL-826940; 1041892

Journal Information:: IEEE Signal Processing Letters, Journal Name: IEEE Signal Processing Letters Journal Issue: na Vol. 29; ISSN 1070-9908

Publisher:: IEEE Signal Processing SocietyCopyright Statement

Country of Publication:: United States

Language:: English

References (25)

A Tutorial on the Cross-Entropy Method de Boer, Pieter-Tjerk; Kroese, Dirk P.; Mannor, Shie Annals of Operations Research, Vol. 134, Issue 1 https://doi.org/10.1007/s10479-005-5724-z	journal	February 2005
Adaptive Importance Sampling for Control and Inference Kappen, H. J.; Ruiz, H. C. Journal of Statistical Physics, Vol. 162, Issue 5 https://doi.org/10.1007/s10955-016-1446-7	journal	January 2016
Rare and extreme events: the case of COVID-19 pandemic Machado, J. A. Tenreiro; Lopes, António M. Nonlinear Dynamics, Vol. 100, Issue 3 https://doi.org/10.1007/s11071-020-05680-w	journal	May 2020
Adaptive importance sampling in general mixture classes Cappé, Olivier; Douc, Randal; Guillin, Arnaud Statistics and Computing, Vol. 18, Issue 4 https://doi.org/10.1007/s11222-008-9059-x	journal	April 2008
Layered adaptive importance sampling Martino, L.; Elvira, V.; Luengo, D. Statistics and Computing, Vol. 27, Issue 3 https://doi.org/10.1007/s11222-016-9642-5	journal	March 2016
Estimation of small failure probabilities in high dimensions by subset simulation Au, Siu-Kui; Beck, James L. Probabilistic Engineering Mechanics, Vol. 16, Issue 4 https://doi.org/10.1016/S0266-8920(01)00019-4	journal	October 2001
Improving population Monte Carlo: Alternative weighting and resampling schemes Elvira, Víctor; Martino, Luca; Luengo, David Signal Processing, Vol. 131 https://doi.org/10.1016/j.sigpro.2016.07.012	journal	February 2017
Cross-entropy-based adaptive importance sampling using Gaussian mixture Kurtz, Nolan; Song, Junho Structural Safety, Vol. 42 https://doi.org/10.1016/j.strusafe.2013.01.006	journal	May 2013
Cross-entropy-based adaptive importance sampling using von Mises-Fisher mixture for high dimensional reliability analysis Wang, Ziqi; Song, Junho Structural Safety, Vol. 59 https://doi.org/10.1016/j.strusafe.2015.11.002	journal	March 2016
Cross entropy-based importance sampling using Gaussian densities revisited Geyer, Sebastian; Papaioannou, Iason; Straub, Daniel Structural Safety, Vol. 76 https://doi.org/10.1016/j.strusafe.2018.07.001	journal	January 2019
Practical rare event sampling for extreme mesoscale weather Webber, Robert J.; Plotkin, David A.; O’Neill, Morgan E. Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 29, Issue 5 https://doi.org/10.1063/1.5081461	journal	May 2019
A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code Mckay, M. D.; Beckman, R. J.; Conover, W. J. Technometrics, Vol. 42, Issue 1 https://doi.org/10.1080/00401706.2000.10485979	journal	February 2000
Nonparametric Importance Sampling Zhang, Ping Journal of the American Statistical Association, Vol. 91, Issue 435 https://doi.org/10.1080/01621459.1996.10476994	journal	September 1996
Population Monte Carlo schemes with reduced path degeneracy Elvira, Victor; Martino, Luca; Luengo, David 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP) https://doi.org/10.1109/CAMSAP.2017.8313090	conference	December 2017
A gradient adaptive population importance sampler Elvira, Victor; Martino, Luca; Luengo, David 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) https://doi.org/10.1109/ICASSP.2015.7178737	conference	April 2015
Langevin-based Strategy for Efficient Proposal Adaptation in Population Monte Carlo Elvira, Victor; Chouzenoux, Emilie ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) https://doi.org/10.1109/ICASSP.2019.8682284	conference	May 2019
Hamiltonian Adaptive Importance Sampling Mousavi, Ali; Monsefi, Reza; Elvira, Victor IEEE Signal Processing Letters, Vol. 28 https://doi.org/10.1109/LSP.2021.3068616	journal	January 2021
Adaptive Importance Sampling: The past, the present, and the future Bugallo, Monica F.; Elvira, Victor; Martino, Luca IEEE Signal Processing Magazine, Vol. 34, Issue 4 https://doi.org/10.1109/MSP.2017.2699226	journal	July 2017
An Adaptive Population Importance Sampler: Learning From Uncertainty Martino, Luca; Elvira, Victor; Luengo, David IEEE Transactions on Signal Processing, Vol. 63, Issue 16 https://doi.org/10.1109/TSP.2015.2440215	journal	August 2015
Policy Gradient Importance Sampling for Bayesian Inference El-Laham, Yousef; Bugallo, Monica F. IEEE Transactions on Signal Processing, Vol. 69 https://doi.org/10.1109/TSP.2021.3093792	journal	January 2021
Adaptive Multiple Importance Sampling: Adaptive multiple importance sampling Cornuet, Jean-Marie; Marin, Jean-Michel; Mira, Antonietta Scandinavian Journal of Statistics, Vol. 39, Issue 4 https://doi.org/10.1111/j.1467-9469.2011.00756.x	journal	February 2012
Population Monte Carlo Cappé, O.; Guillin, A.; Marin, J. M. Journal of Computational and Graphical Statistics, Vol. 13, Issue 4 https://doi.org/10.1198/106186004X12803	journal	December 2004
Computing Normalizing Constants for Finite Mixture Models via Incremental Mixture Importance Sampling (IMIS) Steele, Russell J.; Raftery, Adrian E.; Emond, Mary J. Journal of Computational and Graphical Statistics, Vol. 15, Issue 3 https://doi.org/10.1198/106186006X132358	journal	September 2006
Convergence of adaptive mixtures of importance sampling schemes Douc, R.; Guillin, A.; Marin, J. -M. The Annals of Statistics, Vol. 35, Issue 1 https://doi.org/10.1214/009053606000001154	journal	February 2007
Generalized Multiple Importance Sampling Elvira, Víctor; Martino, Luca; Luengo, David Statistical Science, Vol. 34, Issue 1 https://doi.org/10.1214/18-STS668	journal	February 2019

Similar Records

Estimation and uncertainty quantification for the output from quantum simulators

Journal Article · 2019 · Foundations of Data Science · OSTI ID:1513365

Bennink, Ryan; Lougovski, Pavel; Jasra, Ajay; +1 more

Cluster-Event Biasing in Monte Carlo Applications to Systems Reliability

Journal Article · 2002 · Nuclear Science and Engineering · OSTI ID:20804793

Khazen, Michael; Dubi, Arie

Maximum entropy method for expert system construction

Thesis/Dissertation · 1986 · OSTI ID:6764252

Lippman, A F

Related Subjects

97 MATHEMATICS AND COMPUTING
Monte Carlo methods
adaptive importance sampling
artificial intelligence
cross-entropy
optimization
proposals
rare events
signal processing algorithms
sociology
statistics

Title: Rare Events via Cross-Entropy Population Monte Carlo

Citation Formats

References (25)

Similar Records

Related Subjects