skip to main content
DOE Patents title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Neural node network and model, and method of teaching same

Abstract

The present invention is a fully connected feed forward network that includes at least one hidden layer 16. The hidden layer 16 includes nodes 20 in which the output of the node is fed back to that node as an input with a unit delay produced by a delay device 24 occurring in the feedback path 22 (local feedback). Each node within each layer also receives a delayed output (crosstalk) produced by a delay unit 36 from all the other nodes within the same layer 16. The node performs a transfer function operation based on the inputs from the previous layer and the delayed outputs. The network can be implemented as analog or digital or within a general purpose processor. Two teaching methods can be used: (1) back propagation of weight calculation that includes the local feedback and the crosstalk or (2) more preferably a feed forward gradient decent which immediately follows the output computations and which also includes the local feedback and the crosstalk. Subsequent to the gradient propagation, the weights can be normalized, thereby preventing convergence to a local optimum. Education of the network can be incremental both on and off-line. An educated network is suitable for modelingmore » and controlling dynamic nonlinear systems and time series systems and predicting the outputs as well as hidden states and parameters. The educated network can also be further educated during on-line processing.« less

Inventors:
; ; ; ;
Issue Date:
Research Org.:
Texas A&M Univ., College Station, TX (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
870229
Patent Number(s):
5479571
Application Number:
08/046,936
Assignee:
The Texas A&M University System (College Station, TX)
Patent Classifications (CPCs):
G - PHYSICS G06 - COMPUTING G06N - COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
DOE Contract Number:  
FG07-89ER12893
Resource Type:
Patent
Resource Relation:
Patent File Date: 1993 Apr 13
Country of Publication:
United States
Language:
English
Subject:
/706/

Citation Formats

Parlos, Alexander G., Atiya, Amir F., Fernandez, Benito, Tsai, Wei K., and Chong, Kil T.. Neural node network and model, and method of teaching same. United States: N. p., 1995. Web.
Parlos, Alexander G., Atiya, Amir F., Fernandez, Benito, Tsai, Wei K., & Chong, Kil T.. Neural node network and model, and method of teaching same. United States.
Parlos, Alexander G., Atiya, Amir F., Fernandez, Benito, Tsai, Wei K., and Chong, Kil T.. Sun . "Neural node network and model, and method of teaching same". United States. https://www.osti.gov/servlets/purl/870229.
@article{osti_870229,
title = {Neural node network and model, and method of teaching same},
author = {Parlos, Alexander G. and Atiya, Amir F. and Fernandez, Benito and Tsai, Wei K. and Chong, Kil T.},
abstractNote = {The present invention is a fully connected feed forward network that includes at least one hidden layer 16. The hidden layer 16 includes nodes 20 in which the output of the node is fed back to that node as an input with a unit delay produced by a delay device 24 occurring in the feedback path 22 (local feedback). Each node within each layer also receives a delayed output (crosstalk) produced by a delay unit 36 from all the other nodes within the same layer 16. The node performs a transfer function operation based on the inputs from the previous layer and the delayed outputs. The network can be implemented as analog or digital or within a general purpose processor. Two teaching methods can be used: (1) back propagation of weight calculation that includes the local feedback and the crosstalk or (2) more preferably a feed forward gradient decent which immediately follows the output computations and which also includes the local feedback and the crosstalk. Subsequent to the gradient propagation, the weights can be normalized, thereby preventing convergence to a local optimum. Education of the network can be incremental both on and off-line. An educated network is suitable for modeling and controlling dynamic nonlinear systems and time series systems and predicting the outputs as well as hidden states and parameters. The educated network can also be further educated during on-line processing.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {1995},
month = {1}
}

Patent:

Save / Share:

Works referenced in this record:

Neural networks and physical systems with emergent collective computational abilities.
journal, April 1982


Generalization of back-propagation to recurrent neural networks
journal, November 1987


Approximation by superpositions of a sigmoidal function
journal, December 1989


Adaptation and tracking in system identification—A survey
journal, January 1990


Benefits of gain: speeded learning and minimal hidden layers in back-propagation networks
journal, January 1991


Complex temporal sequence learning based on short-term memory
journal, January 1990


Self-organizing network for optimum supervised learning
journal, March 1990


Chaotic dynamics of time-delay neural networks
conference, January 1990


Neural networks for system identification
journal, April 1990


Identification and control of dynamical systems using neural networks
journal, March 1990


Nonlinear dynamic system identification using artificial neural networks (ANNs)
conference, January 1990


A comparison of the backpropagation and recursive prediction error algorithms for training neural networks
journal, May 1991


An accelerated learning algorithm for multilayer perceptron networks
journal, May 1994


A recurrent time-delay neural network for improved phoneme recognition
conference, January 1991


Backpropagation through time: what it does and how to do it
journal, January 1990


Use of neural nets for dynamic modeling and control of chemical process systems
journal, May 1990


Bounding the states of systems with unknown-but-bounded disturbances
journal, October 1990


Identification of nonlinear systems–a survey
journal, January 1980


Learning by parallel forward propagation
conference, January 1990


A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
journal, June 1989


An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories
journal, December 1990