| | |
Summary: A Generalization Model and Learning in Hardware
Alexander Nicholson
Department of Computer Science
California Institute of Technology
Pasadena, CA, 91125
zander@cs.caltech.edu
Abstract
We study two problems in the eld of machine learning. First, we propose
a novel theoretical framework for understanding learning and generaliza-
tion which we call the bin model. Using the bin model, a closed form is
derived for the generalization error that estimates the out-of-sample per-
formance in terms of the in-sample performance. We address the problem
of over tting, and show that using a simple exhaustive learning algorithm
it does not arise. This is independent of the target function, input distri-
bution and learning model, and remains true even with noisy data sets.
We apply our analysis to both classi cation and regression problems and
give an example of how it may be used eÆciently in practice. Second, we
investigate the use of learning and evolution in hardware for digital circuit
design. Using the reactive tabu search for discrete optimization, we show
that we can learn a multiplier circuit from a set of examples. The learned
|
