An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization

Deng, Wei; Lin, Guang; Liang, Faming

doi:10.1007/s11222-022-10120-3

An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization

Journal Article · 09 July 2022 · Statistics and Computing

DOI:https://doi.org/10.1007/s11222-022-10120-3· OSTI ID:1976710

Deng, Wei ^[1]; Lin, Guang;

OSTI

Not provided.

Research Organization:: Purdue Univ., West Lafayette, IN (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

DOE Contract Number:: SC0021142

OSTI ID:: 1976710

Journal Information:: Statistics and Computing, Journal Name: Statistics and Computing Journal Issue: 4 Vol. 32; ISSN 0960-3174

Publisher:: Springer

Country of Publication:: United States

Language:: English

References (34)

Stability of Stochastic Approximation under Verifiable Conditions Andrieu, Christophe; Moulines, Éric; Priouret, Pierre SIAM Journal on Control and Optimization, Vol. 44, Issue 1 https://doi.org/10.1137/S0363012902417267	journal	January 2005
Exponential Convergence of Langevin Distributions and Their Discrete Approximations Roberts, Gareth O.; Tweedie, Richard L. Bernoulli, Vol. 2, Issue 4 https://doi.org/10.2307/3318418	journal	December 1996
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks Liu, Chaoyue; Zhu, Libin; Belkin, Mikhail Applied and Computational Harmonic Analysis, Vol. 59 https://doi.org/10.1016/j.acha.2021.12.009	journal	July 2022
What is the best multi-stage architecture for object recognition? Jarrett, Kevin; Kavukcuoglu, Koray; Ranzato, Marc' Aurelio 2009 IEEE 12th International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2009.5459469	conference	September 2009
Semi-supervised Structured Prediction with Neural CRF Autoencoder Zhang, Xiao; Jiang, Yong; Peng, Hao Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing https://doi.org/10.18653/v1/D17-1179	conference	January 2017
Optimization by Simulated Annealing Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. Science, Vol. 220, Issue 4598 https://doi.org/10.1126/science.220.4598.671	journal	May 1983
Stochastic Approximation in Monte Carlo Computation Liang, Faming; Liu, Chuanhai; Carroll, Raymond J. Journal of the American Statistical Association, Vol. 102, Issue 477 https://doi.org/10.1198/016214506000001202	journal	March 2007
Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise Mattingly, J. C.; Stuart, A. M.; Higham, D. J. Stochastic Processes and their Applications, Vol. 101, Issue 2 https://doi.org/10.1016/S0304-4149(02)00150-3	journal	October 2002
Efficient, Multiple-Range Random Walk Algorithm to Calculate the Density of States Wang, Fugao; Landau, D. P. Physical Review Letters, Vol. 86, Issue 10 https://doi.org/10.1103/PhysRevLett.86.2050	journal	March 2001
Generalized1∕k-ensemble algorithm Liang, Faming Physical Review E, Vol. 69, Issue 6 https://doi.org/10.1103/PhysRevE.69.066701	journal	June 2004
Monte Carlo sampling methods using Markov chains and their applications Hastings, W. K. Biometrika, Vol. 57, Issue 1 https://doi.org/10.1093/biomet/57.1.97	journal	April 1970
Wide Residual Networks Zagoruyko, Sergey; Komodakis, Nikos Procedings of the British Machine Vision Conference 2016 https://doi.org/10.5244/C.30.87	conference	January 2016
Stochastic Gradient Markov Chain Monte Carlo Nemeth, Christopher; Fearnhead, Paul Journal of the American Statistical Association, Vol. 116, Issue 533 https://doi.org/10.1080/01621459.2020.1847120	journal	January 2021
Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference Geyer, Charles J.; Thompson, Elizabeth A. Journal of the American Statistical Association, Vol. 90, Issue 431 https://doi.org/10.1080/01621459.1995.10476590	journal	September 1995
Cyclical Learning Rates for Training Neural Networks Smith, Leslie N. 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) https://doi.org/10.1109/WACV.2017.58	conference	March 2017
Deep Residual Learning for Image Recognition He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.90	conference	June 2016
Convergence of Numerical Time-Averaging and Stationary Measures via Poisson Equations Mattingly, Jonathan C.; Stuart, Andrew M.; Tretyakov, M. V. SIAM Journal on Numerical Analysis, Vol. 48, Issue 2 https://doi.org/10.1137/090770527	journal	January 2010
On the use of stochastic approximation Monte Carlo for Monte Carlo integration Liang, Faming Statistics & Probability Letters, Vol. 79, Issue 5 https://doi.org/10.1016/j.spl.2008.10.007	journal	March 2009
Convergence of the Wang-Landau algorithm Fort, Gersende; Jourdain, Benjamin; Kuhn, Estelle Mathematics of Computation, Vol. 84, Issue 295 https://doi.org/10.1090/S0025-5718-2015-02952-4	journal	March 2015
Experimental Testing of Advanced Scatter Search Designs for Global Optimization of Multimodal Functions Laguna, Manuel; Martí, Rafael Journal of Global Optimization, Vol. 33, Issue 2 https://doi.org/10.1007/s10898-004-1936-z	journal	October 2005
Replica Monte Carlo Simulation of Spin-Glasses Swendsen, Robert H.; Wang, Jian-Sheng Physical Review Letters, Vol. 57, Issue 21 https://doi.org/10.1103/PhysRevLett.57.2607	journal	November 1986
A Generalized Wang–Landau Algorithm for Monte Carlo Computation Liang, Faming Journal of the American Statistical Association, Vol. 100, Issue 472 https://doi.org/10.1198/016214505000000259	journal	December 2005
Monte Carlo Simulation and Global Optimization without Parameters Hesselbo, Bobby; Stinchcombe, R. B. Physical Review Letters, Vol. 74, Issue 12 https://doi.org/10.1103/PhysRevLett.74.2151	journal	March 1995
Simulated Tempering: A New Monte Carlo Scheme Marinari, E.; Parisi, G. Europhysics Letters (EPL), Vol. 19, Issue 6 https://doi.org/10.1209/0295-5075/19/6/002	journal	July 1992
A Stochastic Approximation Method Robbins, Herbert; Monro, Sutton The Annals of Mathematical Statistics, Vol. 22, Issue 3 https://doi.org/10.1214/aoms/1177729586	journal	September 1951
Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods Girolami, Mark; Calderhead, Ben Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 2 https://doi.org/10.1111/j.1467-9868.2010.00765.x	journal	March 2011
Gradient-based learning applied to document recognition Lecun, Y.; Bottou, L.; Bengio, Y. Proceedings of the IEEE, Vol. 86, Issue 11 https://doi.org/10.1109/5.726791	journal	January 1998
Monte Carlo Statistical Methods Robert, Christian P.; Casella, George Springer Texts in Statistics https://doi.org/10.1007/978-1-4757-4145-2	book	January 2004
Equation of State Calculations by Fast Computing Machines Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N. The Journal of Chemical Physics, Vol. 21, Issue 6 https://doi.org/10.1063/1.1699114	journal	June 1953
Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation Belkin, Mikhail Acta Numerica, Vol. 30 https://doi.org/10.1017/S0962492921000039	journal	May 2021
Continuous Contour Monte Carlo for Marginal Density Estimation With an Application to a Spatial Statistical Model Liang, Faming Journal of Computational and Graphical Statistics, Vol. 16, Issue 3 https://doi.org/10.1198/106186007X238459	journal	September 2007
Random Erasing Data Augmentation Zhong, Zhun; Zheng, Liang; Kang, Guoliang Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, Issue 07 https://doi.org/10.1609/aaai.v34i07.7000	journal	April 2020
Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks Li, Chunyuan; Chen, Changyou; Carlson, David Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30, Issue 1 https://doi.org/10.1609/aaai.v30i1.10200	journal	February 2016
Multicanonical algorithms for first order phase transitions Berg, Bernd A.; Neuhaus, Thomas Physics Letters B, Vol. 267, Issue 2 https://doi.org/10.1016/0370-2693(91)91256-U	journal	September 1991

Similar Records

An adaptive Hessian approximated stochastic gradient MCMC method

Journal Article · 2021 · Journal of Computational Physics · OSTI ID:1853727

Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo

Journal Article · 2020 · SIAM Journal on Scientific Computing · OSTI ID:1853730

Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo

Journal Article · 2021 · SIAM Journal on Scientific Computing · OSTI ID:1866812

Related Subjects

Computer Science
Mathematics

An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization

Citation Formats

References (34)

Similar Records

Related Subjects