 
Summary: Reinforcement Learning, Neural Networks and PI Control Applied to a
Heating Coil
Charles W. Anderson1, Douglas C. Hittle2, Alon D. Katz2, and R. Matt Kretchmar1
1Department of Computer Science
Colorado State University
Fort Collins, CO 80523
fanderson,kretchmag@cs.colostate.edu
2Department of Mechanical Engineering
Colorado State University
Fort Collins, CO 80523
fhittle,along@lamar.colostate.edu
Abstract
An accurate simulation of a heating coil is used to compare the performance of a PI controller, a neural
network trained to predict the steadystate output of the PI controller, a neural network trained to minimize
the nstep ahead error between the coil output and the set point, and a reinforcement learning agent trained
to minimize the sum of the squared error over time. Although the PI controller works very well for this
task, the neural networks do result in improved performance.
1 Introduction
Typical methods for designing fixed feedback controllers results in suboptimal control performance. In
many situations, the degree of uncertainty in the model of the system being controlled limits the utility of
