Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Evolutionary reinforcement learning of dynamical large deviations

Journal Article · · Journal of Chemical Physics
DOI:https://doi.org/10.1063/5.0015301· OSTI ID:1783100
 [1];  [2];  [3]
  1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Molecular Foundry
  2. California Institute of Technology (CalTech), Pasadena, CA (United States)
  3. National Research Council of Canada, Ottawa, ON (Canada); Vector Inst. for Artificial Intelligence, Toronto, ON (Canada)
+ Show Author Affiliations
In this work, we show how to bound and calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, potentially allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces, the evolutionary process acts directly on rates, and for models with large state spaces, the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.
Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Basic Energy Sciences (BES)
Grant/Contract Number:
AC02-05CH11231
OSTI ID:
1783100
Alternate ID(s):
OSTI ID: 1643200
Journal Information:
Journal of Chemical Physics, Journal Name: Journal of Chemical Physics Journal Issue: 4 Vol. 153; ISSN 0021-9606
Publisher:
American Institute of Physics (AIP)Copyright Statement
Country of Publication:
United States
Language:
English

References (58)

Neural Fitted Q Iteration – First Experiences with a Data Efficient Neural Reinforcement Learning Method book January 2005
Q-learning journal May 1992
Reinforcement learning for robot soccer journal May 2009
Finite Size Scaling of the Dynamical Free-Energy in a Kinetically Constrained Model journal March 2012
Adaptive Importance Sampling for Control and Inference journal January 2016
On the effectiveness of crossover in simulated evolutionary optimization journal January 1994
A policy gradient method for semi-Markov decision processes with application to call admission control journal May 2007
A basic formula for performance gradient estimation of semi-Markov decision processes journal January 2013
Sampling algorithms for validation of supervised learning models for Ising-like systems journal December 2017
The large deviation approach to statistical mechanics journal July 2009
Library-Based LAMMPS Implementation of High-Dimensional Neural Network Potentials journal January 2019
Exact stochastic simulation of coupled chemical reactions journal December 1977
Human-level control through deep reinforcement learning journal February 2015
Mastering the game of Go with deep neural networks and tree search journal January 2016
Mastering the game of Go without human knowledge journal October 2017
Quantum-chemical insights from deep tensor neural networks journal January 2017
Machine learning phases of matter journal February 2017
Identifying quantum phase transitions using artificial neural networks on experimental data journal July 2019
Genetic Algorithms journal July 1992
Machine learning and data science in materials design: a themed collection journal January 2018
The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics journal January 2018
Constructing first-principles phase diagrams of amorphous Li x Si using machine-learning-assisted sampling with an evolutionary algorithm journal June 2018
SchNet – A deep learning architecture for molecules and materials journal June 2018
A new approach for the prediction of partition functions using machine learning techniques journal July 2018
Machine learning and molecular design of self-assembling -conjugated oligopeptides journal April 2018
Density anomaly of water at negative pressures from first principles journal June 2018
The large deviation function for entropy production: the optimal trajectory and the role of fluctuations journal December 2012
Variational and optimal control representations of conditioned and driven processes journal December 2015
First-order dynamical phase transition in models of glasses: an approach based on ensembles of histories journal January 2009
A deep learning functional estimator of optimal dynamics for sampling large deviations journal July 2020
Monte Carlo sampling methods using Markov chains and their applications journal April 1970
Deep learning and the Schrödinger equation journal October 2017
Direct evaluation of dynamical large-deviation rate functions using a variational ansatz journal November 2019
Learning to grow: Control of material self-assembly using evolutionary reinforcement learning journal May 2020
Universal bounds on current fluctuations journal May 2016
Simple bounds on fluctuations and uncertainty relations for first-passage times of counting observables journal March 2017
Policy-guided Monte Carlo: Reinforcement-learning Markov chain dynamics journal December 2018
Dissipation Bounds All Steady-State Current Fluctuations journal March 2016
Finite-Size Scaling of a First-Order Dynamical Phase Transition: Adaptive Population Dynamics and an Effective Model journal March 2017
Exact Fluctuations of Nonequilibrium Steady States from Approximate Auxiliary Dynamics journal May 2018
Kinetic Ising Model of the Glass Transition journal September 1984
Geometrical Explanation and Scaling of Dynamical Heterogeneities in Glass Forming Systems journal July 2002
Entropy Production along a Stochastic Trajectory and an Integral Fluctuation Theorem journal July 2005
Direct Evaluation of Large-Deviation Functions journal March 2006
Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces journal April 2007
On large deviations theory and asymptotically efficient Monte Carlo estimation journal May 1990
Gradient-based learning applied to document recognition journal January 1998
Current Fluctuations in Systems with Diffusive Dynamics, in and out of Equilibrium journal January 2010
T RANSITION P ATH S AMPLING : Throwing Ropes Over Rough Mountain Passes, in the Dark journal October 2002
Activity-driven fluctuations in living cells journal May 2015
Canonical structure of dynamical fluctuations in mesoscopic nonequilibrium steady states journal April 2008
Large deviations of the empirical flow for continuous time Markov chains
  • Bertini, Lorenzo; Faggionato, Alessandra; Gabrielli, Davide
  • Annales de l'Institut Henri Poincaré, Probabilités et Statistiques, Vol. 51, Issue 3 https://doi.org/10.1214/14-AIHP601
journal August 2015
Importance Sampling for Stochastic Simulations journal November 1989
A Learning Algorithm for Risk-Sensitive Cost journal November 2008
Adaptive Importance Sampling Technique for Markov Chains Using Stochastic Approximation journal June 2006
The Arcade Learning Environment: An Evaluation Platform for General Agents journal May 2013
Monte Carlo simulation and large deviations theory for uniformly recurrent Markov chains journal March 1990
Peformance Analysis Conditioned on Rare Events: An Adaptive Simulation Scheme journal January 2003

Cited By (1)

A reinforcement learning approach to rare trajectory sampling text January 2020

Similar Records

Optimizing thermodynamic trajectories using evolutionary reinforcement learning
Journal Article · Wed Mar 20 00:00:00 EDT 2019 · arXiv.org Repository · OSTI ID:1601197
Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning
Journal Article · Sun Dec 19 23:00:00 EST 2021 · Physical Review. E · OSTI ID:1860343

Related Subjects

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY
Markov processes
Monte Carlo methods
artificial neural networks
computational physics
lattice models
machine learning
stochastic processes