Correspondence between neuroevolution and gradient descent

Whitelam, Stephen; Selin, Viktor; Park, Sang-Won; Tamblyn, Isaac

doi:10.1038/s41467-021-26568-2

Correspondence between neuroevolution and gradient descent

Journal Article · Tue Nov 02 00:00:00 EDT 2021 · Nature Communications

DOI:https://doi.org/10.1038/s41467-021-26568-2· OSTI ID:1828630

; Selin, Viktor; Park, Sang-Won; Tamblyn, Isaac

Abstract

We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations, for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.

View Journal Article

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 1828630

Alternate ID(s):: OSTI ID: 1855195

Journal Information:: Nature Communications, Journal Name: Nature Communications Journal Issue: 1 Vol. 12; ISSN 2041-1723

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (28)

Taylor expansion of the accumulated rounding error Linnainmaa, Seppo BIT, Vol. 16, Issue 2 https://doi.org/10.1007/BF01931367	journal	June 1976
Approximation by superpositions of a sigmoidal function Cybenko, G. Mathematics of Control, Signals, and Systems, Vol. 2, Issue 4 https://doi.org/10.1007/BF02551274	journal	December 1989
Neuroevolution: from architectures to learning Floreano, Dario; Dürr, Peter; Mattiussi, Claudio Evolutionary Intelligence, Vol. 1, Issue 1 https://doi.org/10.1007/s12065-007-0002-4	journal	January 2008
Metropolis Monte Carlo method for Brownian dynamics simulation generalized to include hydrodynamic interactions Kikuchi, K.; Yoshida, M.; Maekawa, T. Chemical Physics Letters, Vol. 196, Issue 1-2 https://doi.org/10.1016/0009-2614(92)85928-4	journal	August 1992
On the effectiveness of crossover in simulated evolutionary optimization Fogel, David B.; Stayton, Lauren C. Biosystems, Vol. 32, Issue 3 https://doi.org/10.1016/0303-2647(94)90040-X	journal	January 1994
Theory of the Backpropagation Neural Network Hecht-Nielsen, Robert Neural Networks for Perception https://doi.org/10.1016/B978-0-12-741252-8.50010-8	book	January 1992
Metropolis Monte Carlo method as a numerical technique to solve the Fokker—Planck equation Kikuchi, K.; Yoshida, M.; Maekawa, T. Chemical Physics Letters, Vol. 185, Issue 3-4 https://doi.org/10.1016/S0009-2614(91)85070-D	journal	October 1991
Deep learning in neural networks: An overview Schmidhuber, Jürgen Neural Networks, Vol. 61 https://doi.org/10.1016/j.neunet.2014.09.003	journal	January 2015
Learning representations by back-propagating errors Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. Nature, Vol. 323, Issue 6088 https://doi.org/10.1038/323533a0	journal	October 1986
Designing neural networks through neuroevolution Stanley, Kenneth O.; Clune, Jeff; Lehman, Joel Nature Machine Intelligence, Vol. 1, Issue 1 https://doi.org/10.1038/s42256-018-0006-z	journal	January 2019
Genetic Algorithms Holland, John H. Scientific American, Vol. 267, Issue 1 https://doi.org/10.1038/scientificamerican0792-66	journal	July 1992
Equation of State Calculations by Fast Computing Machines Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N. The Journal of Chemical Physics, Vol. 21, Issue 6 https://doi.org/10.1063/1.1699114	journal	June 1953
Reversible self-assembly of patchy particles into monodisperse icosahedral clusters Wilber, Alex W.; Doye, Jonathan P. K.; Louis, Ard A. The Journal of Chemical Physics, Vol. 127, Issue 8 https://doi.org/10.1063/1.2759922	journal	August 2007
Avoiding unphysical kinetic traps in Monte Carlo simulations of strongly attractive particles Whitelam, Stephen; Geissler, Phillip L. The Journal of Chemical Physics, Vol. 127, Issue 15 https://doi.org/10.1063/1.2790421	journal	October 2007
Dynamic Monte Carlo versus Brownian dynamics: A comparison for self-diffusion and crystallization in colloidal fluids Sanz, E.; Marenduzzo, D. The Journal of Chemical Physics, Vol. 132, Issue 19 https://doi.org/10.1063/1.3414827	journal	May 2010
Coarse-grained Monte Carlo simulations of non-equilibrium systems Liu, Xiao; Crocker, John C.; Sinno, Talid The Journal of Chemical Physics, Vol. 138, Issue 24 https://doi.org/10.1063/1.4811656	journal	June 2013
Approximating the dynamical evolution of systems of strongly interacting overdamped particles Whitelam, Stephen Molecular Simulation, Vol. 37, Issue 7 https://doi.org/10.1080/08927022.2011.565758	journal	June 2011
Monte Carlo sampling methods using Markov chains and their applications Hastings, W. K. Biometrika, Vol. 57, Issue 1 https://doi.org/10.1093/biomet/57.1.97	journal	April 1970
Learning to grow: Control of material self-assembly using evolutionary reinforcement learning Whitelam, Stephen; Tamblyn, Isaac Physical Review E, Vol. 101, Issue 5 https://doi.org/10.1103/PhysRevE.101.052604	journal	May 2020
Revisiting the slow dynamics of a silica melt using Monte Carlo simulations Berthier, Ludovic Physical Review E, Vol. 76, Issue 1 https://doi.org/10.1103/PhysRevE.76.011507	journal	July 2007
Nonuniversal critical dynamics in Monte Carlo simulations Swendsen, Robert H.; Wang, Jian-Sheng Physical Review Letters, Vol. 58, Issue 2 https://doi.org/10.1103/PhysRevLett.58.86	journal	January 1987
Collective Monte Carlo Updating for Spin Systems Wolff, Ulli Physical Review Letters, Vol. 62, Issue 4 https://doi.org/10.1103/PhysRevLett.62.361	journal	January 1989
Rejection-Free Geometric Cluster Algorithm for Complex Fluids Liu, Jiwen; Luijten, Erik Physical Review Letters, Vol. 92, Issue 3 https://doi.org/10.1103/PhysRevLett.92.035504	journal	January 2004
How to simulate patchy particles Rovigatti, Lorenzo; Russo, John; Romano, Flavio The European Physical Journal E, Vol. 41, Issue 5 https://doi.org/10.1140/epje/i2018-11667-x	journal	May 2018
Simple Evolutionary Optimization Can Rival Stochastic Gradient Descent in Neural Networks Morse, Gregory; Stanley, Kenneth O. GECCO '16: Genetic and Evolutionary Computation Conference, Proceedings of the Genetic and Evolutionary Computation Conference 2016 https://doi.org/10.1145/2908812.2908916	conference	July 2016
Statistical Mechanics of Deep Learning Bahri, Yasaman; Kadmon, Jonathan; Pennington, Jeffrey Annual Review of Condensed Matter Physics, Vol. 11, Issue 1 https://doi.org/10.1146/annurev-conmatphys-031119-050745	journal	March 2020
Evolving Neural Networks through Augmenting Topologies Stanley, Kenneth O.; Miikkulainen, Risto Evolutionary Computation, Vol. 10, Issue 2 https://doi.org/10.1162/106365602320169811	journal	June 2002
Backpropagation Applied to Handwritten Zip Code Recognition LeCun, Y.; Boser, B.; Denker, J. S. Neural Computation, Vol. 1, Issue 4 https://doi.org/10.1162/neco.1989.1.4.541	journal	December 1989

Similar Records

Using the Metropolis algorithm to explore the loss surface of a recurrent neural network

Journal Article · Fri Dec 20 23:00:00 EST 2024 · Journal of Chemical Physics · OSTI ID:2574501

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Journal Article · Thu Oct 05 00:00:00 EDT 2023 · Communications on Applied Mathematics and Computation · OSTI ID:2007675

Correspondence between neuroevolution and gradient descent

Citation Formats

References (28)

Similar Records

Related Subjects