Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Bin Model for Neural Networks Yaser AbuMostafay, Xubo Songz

Summary: Bin Model for Neural Networks
Yaser Abu­Mostafay, Xubo Songz
y Dept. of Electrical Engineering and Dept. of Computer Science
California Institute of Technology
Pasedena, CA 91125 USA
z Dept. of Electrical Engineering
California Institute of Technology
Pasedena, CA 91125 USA
Abstract--- We propose a theoretical framework for the modeling of learning machines,
such as neural networks, which we call the bin model, in which we consider a function as
a Bernoulli distribution. Using this model, we study the issues related to generalization,
such as the expected test error given a certain training error, and the expected test error
given the best training error. Noise in the data is also captured by the bin model, and the
effect of noise on generalization is quantified.
1 Introduction
In learning, what we have is a finite and often noisy data set. To approximate the target function
embodied in the data set, we choose a class of functions, e.g., the functions that can be implemented
by a feed­forward neural network with certain architecture, as our candidates. Each candidate function
approximates the target function to certain precision. From the candidate functions, we will choose the
one that fits the data well according to some criterion. We have only access to training error. How well


Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology


Collections: Computer Technologies and Information Sciences