Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning

Journal Article · · Physical Review. E
 [1];  [2];  [3];  [4];  [5];  [6]
  1. Univ. of Ottawa, ON (Canada); National Research Council of Canada, Ottawa (Canada)
  2. Univ. of Ontario Institute of Technology, Oshawa (Canada)
  3. Univ. of Victoria, BC (Canada)
  4. Univ. of Ontario Institute of Technology, Oshawa (Canada); Vector Institute for Artificial Intelligence, Toronto (Canada)
  5. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  6. Univ. of Ontario Institute of Technology, Oshawa (Canada); Vector Institute for Artificial Intelligence, Toronto (Canada); Univ. of Ottawa, ON (Canada)
+ Show Author Affiliations
Here using a model heat engine, we show that neural-network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters.
Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Basic Energy Sciences (BES)
Grant/Contract Number:
AC02-05CH11231
OSTI ID:
1860343
Journal Information:
Physical Review. E, Journal Name: Physical Review. E Journal Issue: 6 Vol. 104; ISSN 2470-0045
Publisher:
American Physical Society (APS)Copyright Statement
Country of Publication:
United States
Language:
English

References (31)

Q-learning journal May 1992
Reinforcement learning for robot soccer journal May 2009
Thermodynamic Formalism for Systems with Markov Dynamics journal January 2007
Image captioning via proximal policy optimization journal April 2021
Physics-informed reinforcement learning optimization of nuclear assembly design journal February 2021
Human-level control through deep reinforcement learning journal February 2015
Mastering the game of Go with deep neural networks and tree search journal January 2016
Mastering the game of Go without human knowledge journal October 2017
Optimization of Molecules via Deep Reinforcement Learning journal July 2019
Finding the ground state of spin Hamiltonians with reinforcement learning journal September 2020
Biomimetic ultra-broadband perfect absorbers optimised with reinforcement learning journal January 2020
Reversible self-assembly of patchy particles into monodisperse icosahedral clusters journal August 2007
Optimal paths for thermodynamic systems: The ideal Otto cycle journal January 1982
Allocating dissipation across a molecular machine cycle to maximize flux journal October 2017
Stochastic thermodynamics, fluctuation theorems and molecular machines journal November 2012
The large deviation function for entropy production: the optimal trajectory and the role of fluctuations journal December 2012
Fluctuations in interacting particle systems with memory journal July 2015
First-order dynamical phase transition in models of glasses: an approach based on ensembles of histories journal January 2009
Biochemical Machines for the Interconversion of Mutual Information and Work journal January 2017
Entropy Production along a Stochastic Trajectory and an Integral Fluctuation Theorem journal July 2005
ViZDoom Competitions: Playing Doom From Pixels journal September 2019
Deep Reinforcement Learning for Physics-Based Musculoskeletal Simulations of Healthy Subjects and Transfemoral Prostheses’ Users During Normal Walking journal January 2021
Deep reinforcement learning for de novo drug design journal July 2018
Crystallization by particle attachment in synthetic, biogenic, and geologic environments journal July 2015
Current Fluctuations in Systems with Diffusive Dynamics, in and out of Equilibrium journal January 2010
The Statistical Mechanics of Dynamic Pathways to Self-Assembly journal April 2015
Stochastic Simulation of Chemical Kinetics journal May 2007
Games in Culture journal August 1959
Dynamic Pathways for Viral Capsid Assembly journal July 2006
The Arcade Learning Environment: An Evaluation Platform for General Agents journal May 2013
Quantum error correction for the toric code using deep reinforcement learning journal September 2019

Cited By (2)

A reinforcement learning approach to rare trajectory sampling text January 2020
Accelerating GMRES with Deep Learning in Real-Time preprint January 2021

Similar Records

Optimizing thermodynamic trajectories using evolutionary reinforcement learning
Journal Article · Wed Mar 20 00:00:00 EDT 2019 · arXiv.org Repository · OSTI ID:1601197
Evolutionary reinforcement learning of dynamical large deviations
Journal Article · Mon Jul 27 20:00:00 EDT 2020 · Journal of Chemical Physics · OSTI ID:1783100

Related Subjects

71 CLASSICAL AND QUANTUM MECHANICS
GENERAL PHYSICS
heat engines
machine learning
thermodynamics