Neural node network and model, and method of teaching same

Parlos, Alexander G.; Atiya, Amir F.; Fernandez, Benito; Tsai, Wei K.; Chong, Kil T.

Title: Neural node network and model, and method of teaching same

Patent · Sun Jan 01 00:00:00 EST 1995

OSTI ID:870229

Parlos, Alexander G.; Atiya, Amir F.; Fernandez, Benito; Tsai, Wei K.; Chong, Kil T.

The present invention is a fully connected feed forward network that includes at least one hidden layer 16. The hidden layer 16 includes nodes 20 in which the output of the node is fed back to that node as an input with a unit delay produced by a delay device 24 occurring in the feedback path 22 (local feedback). Each node within each layer also receives a delayed output (crosstalk) produced by a delay unit 36 from all the other nodes within the same layer 16. The node performs a transfer function operation based on the inputs from the previous layer and the delayed outputs. The network can be implemented as analog or digital or within a general purpose processor. Two teaching methods can be used: (1) back propagation of weight calculation that includes the local feedback and the crosstalk or (2) more preferably a feed forward gradient decent which immediately follows the output computations and which also includes the local feedback and the crosstalk. Subsequent to the gradient propagation, the weights can be normalized, thereby preventing convergence to a local optimum. Education of the network can be incremental both on and off-line. An educated network is suitable for modeling and controlling dynamic nonlinear systems and time series systems and predicting the outputs as well as hidden states and parameters. The educated network can also be further educated during on-line processing.

View Patent

Cite

Export

Save

Research Organization:: Texas A&M Univ., College Station, TX (United States)

Sponsoring Organization:: USDOE

DOE Contract Number:: FG07-89ER12893

Assignee:: The Texas A&M University System (College Station, TX)

Patent Number(s):: 5,479,571

Application Number:: 08/046,936

OSTI ID:: 870229

Resource Relation:: Patent File Date: 1993 Apr 13

Country of Publication:: United States

Language:: English

References (27)

Neural networks and physical systems with emergent collective computational abilities. Hopfield, J. J. Proceedings of the National Academy of Sciences, Vol. 79, Issue 8 https://doi.org/10.1073/pnas.79.8.2554	journal	April 1982
Generalization of back-propagation to recurrent neural networks Pineda, Fernando J. Physical Review Letters, Vol. 59, Issue 19 https://doi.org/10.1103/PhysRevLett.59.2229	journal	November 1987
Approximation by superpositions of a sigmoidal function Cybenko, G. Mathematics of Control, Signals, and Systems, Vol. 2, Issue 4 https://doi.org/10.1007/BF02551274	journal	December 1989
Learning state space trajectories in recurrent neural networks Pearlmutter, Barak A. International Joint Conference on Neural Networks https://doi.org/10.1109/IJCNN.1989.118724	conference	January 1989
Adaptation and tracking in system identification—A survey Ljung, Lennart; Gunnarsson, Svante Automatica, Vol. 26, Issue 1 https://doi.org/10.1016/0005-1098(90)90154-A	journal	January 1990
Benefits of gain: speeded learning and minimal hidden layers in back-propagation networks Kruschke, J. K.; Movellan, J. R. IEEE Transactions on Systems, Man, and Cybernetics, Vol. 21, Issue 1 https://doi.org/10.1109/21.101159	journal	January 1991
Issues in system identification Ljung, L. IEEE Control Systems, Vol. 11, Issue 1, p. 25-29 https://doi.org/10.1109/37.103346	journal	January 1991
Complex temporal sequence learning based on short-term memory Wang, D.; Arbib, M. A. Proceedings of the IEEE, Vol. 78, Issue 9 https://doi.org/10.1109/5.58329	journal	January 1990
Self-organizing network for optimum supervised learning Tenorio, M. F.; Lee, W. -T. IEEE Transactions on Neural Networks, Vol. 1, Issue 1 https://doi.org/10.1109/72.80209	journal	March 1990
Structure identification of nonlinear dynamic systems—A survey on input/output approaches Haber, R.; Unbehauen, H. Automatica, Vol. 26, Issue 4 https://doi.org/10.1016/0005-1098(90)90044-I	journal	July 1990
Chaotic dynamics of time-delay neural networks Principe, J. C.; Lo, P. -C. 1990 IJCNN International Joint Conference on Neural Networks https://doi.org/10.1109/IJCNN.1990.137746	conference	January 1990
Neural networks for system identification Chu, S. R.; Shoureshi, R.; Tenorio, M. IEEE Control Systems Magazine, Vol. 10, Issue 3 https://doi.org/10.1109/37.55121	journal	April 1990
Identification and control of dynamical systems using neural networks Narendra, K. S.; Parthasarathy, K. IEEE Transactions on Neural Networks, Vol. 1, Issue 1 https://doi.org/10.1109/72.80202	journal	March 1990
Nonlinear dynamic system identification using artificial neural networks (ANNs) Fernandez, B.; Parlos, A. G.; Tsai, W. K. 1990 IJCNN International Joint Conference on Neural Networks https://doi.org/10.1109/IJCNN.1990.137706	conference	January 1990
Recurrent Backpropagation and the Dynamical Approach to Adaptive Neural Computation Pineda, Fernando J. Neural Computation, Vol. 1, Issue 2 https://doi.org/10.1162/neco.1989.1.2.161	journal	June 1989
A comparison of the backpropagation and recursive prediction error algorithms for training neural networks Billings, S. A.; Jamaluddin, H. B.; Chen, S. Mechanical Systems and Signal Processing, Vol. 5, Issue 3 https://doi.org/10.1016/0888-3270(91)90045-7	journal	May 1991
An accelerated learning algorithm for multilayer perceptron networks Parlos, A. G.; Fernandez, B.; Atiya, A. F. IEEE Transactions on Neural Networks, Vol. 5, Issue 3 https://doi.org/10.1109/72.286921	journal	May 1994
A recurrent time-delay neural network for improved phoneme recognition Greco, F.; Paoloni, A.; Ravaioli, G. [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing https://doi.org/10.1109/ICASSP.1991.150283	conference	January 1991
Backpropagation through time: what it does and how to do it Werbos, P. J. Proceedings of the IEEE, Vol. 78, Issue 10 https://doi.org/10.1109/5.58337	journal	January 1990
Principal component analysis by gradient descent on a constrained linear Hebbian cell Chauvin, K. M. International Joint Conference on Neural Networks https://doi.org/10.1109/IJCNN.1989.118611	conference	January 1989
Speaker-independent recognition of connected utterances using recurrent and non-recurrent neural networks Franzini, M.; Witbrock, Michael; Lee, T. H. International Joint Conference on Neural Networks https://doi.org/10.1109/IJCNN.1989.118670	conference	January 1989
Use of neural nets for dynamic modeling and control of chemical process systems Bhat, N.; McAvoy, T. J. Computers & Chemical Engineering, Vol. 14, Issue 4-5 https://doi.org/10.1016/0098-1354(90)87028-N	journal	May 1990
Bounding the states of systems with unknown-but-bounded disturbances Tsai, Wei K.; Parlos, Alexander G.; Verghese, George C. International Journal of Control, Vol. 52, Issue 4 https://doi.org/10.1080/00207179008953573	journal	October 1990
Identification of nonlinear systems–a survey Billings, S. A. IEE Proceedings D Control Theory and Applications, Vol. 127, Issue 6 https://doi.org/10.1049/ip-d.1980.0047	journal	January 1980
Learning by parallel forward propagation Abe, S. 1990 IJCNN International Joint Conference on Neural Networks https://doi.org/10.1109/IJCNN.1990.137830	conference	January 1990
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks Williams, Ronald J.; Zipser, David Neural Computation, Vol. 1, Issue 2 https://doi.org/10.1162/neco.1989.1.2.270	journal	June 1989
An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories Williams, Ronald J.; Peng, Jing Neural Computation, Vol. 2, Issue 4 https://doi.org/10.1162/neco.1990.2.4.490	journal	December 1990

Similar Records

Neural node network and model, and method of teaching same

Patent · Tue Dec 26 00:00:00 EST 1995 · OSTI ID:870229

Parlos, A G; Atiya, A F; Fernandez, B; +2 more

A neural network model to predict lung radiation-induced pneumonitis

Journal Article · Sat Sep 15 00:00:00 EDT 2007 · Medical Physics · OSTI ID:870229

Shifeng, Chen; Sumin, Zhou; Junan, Zhang; +3 more

Character Recognition Using Genetically Trained Neural Networks

Technical Report · Thu Oct 01 00:00:00 EDT 1998 · OSTI ID:870229

Diniz, C; Stantz, K M; Trahan, M W; +1 more

Related Subjects

/706/

Title: Neural node network and model, and method of teaching same

Citation Formats

References (27)

Similar Records

Related Subjects