Home

About

Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network
FAQHELPSITE MAPCONTACT US


  Advanced Search  

 
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 11, NO. 3, MAY 2000 697 New Results on Recurrent Network Training
 

Summary: IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 11, NO. 3, MAY 2000 697
New Results on Recurrent Network Training:
Unifying the Algorithms and Accelerating
Convergence
Amir F. Atiya, Senior Member, IEEE, and Alexander G. Parlos, Senior Member, IEEE
Abstract--How to efficiently train recurrent networks remains
a challenging and active research topic. Most of the proposed
training approaches are based on computational ways to effi-
ciently obtain the gradient of the error function, and can be
generally grouped into five major groups. In this study we present
a derivation that unifies these approaches. We demonstrate that
the approaches are only five different ways of solving a particular
matrix equation. The second goal of this paper is develop a new
algorithm based on the insights gained from the novel formulation.
The new algorithm, which is based on approximating the error
gradient, has lower computational complexity in computing the
weight update than the competing techniques for most typical
problems. In addition, it reaches the error minimum in a much
smaller number of iterations. A desirable characteristic of re-
current network training algorithms is to be able to update the

  

Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology
Atiya, Amir - Computer Engineering Department, Cairo University
Parlos, Alexander - Department of Mechanical Engineering, Texas A&M University

 

Collections: Computer Technologies and Information Sciences; Engineering