skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A mathematical discussion concerning the performance of multilayer perceptron-type artificial neural networks through use of orthogonal bipolar vectors

Abstract

This work proposes the use of orthogonal bipolar vectors (OBV) as targets for multilayer perceptron (MLP) Artificial neural networks pertaining to the pattern recognition field. Implications involving this proposal are supported by a proper mathematical discussion. This study shows that the use of OBV as targets improves the class identification performance of MLP networks for pattern recognition with more than two classes when compared to the widely used conventional target bipolar vector (CBV). In addition to the overall performance improvement, it is possible to obtain high recognition rates with fewer training epochs, while keeping a feasible generalization with other data. This behavior is related to the Euclidean distance of the points in the output space, which is much higher in the case of OBVs. Therefore, with a greater distance between the points in the output space, the MLP network becomes less prone to associate trained outputs with incorrect targets. The mathematical discussion in this study is based on the backpropagation training algorithm. In addition to the mathematical discussion, results from experiments with real data are presented. These results follow those obtained by mathematical derivations. Real data sets used in the experiments are available from: (a) the Semeion Handwritten Digit ofmore » Machine Learning Repository, international repository; (b) Iris Image Database from the Chinese Academy of Sciences—CASIA; (c) Australian Sign Language, signs of Machine Learning Repository, international repository. Experiments were conducted to measure the mean Euclidean distance between the MLP network output subjected to a given input pattern and outputs related to other pattern classifications. Experimental results show that MLP networks trained with OBVs were able to ensure a Euclidean distance between the output and incorrect targets greater than MLP networks trained with CBVs (by up to 4 times for handwritten, by up to 12 times for iris images and by up to 10 times for Auslan signs language). As such, the adoption of OBVs as target vectors for MLPs reduces the computational effort for training the network.« less

Authors:
; ; ; ; ;
Publication Date:
OSTI Identifier:
22769363
Resource Type:
Journal Article
Journal Name:
Computational and Applied Mathematics
Additional Journal Information:
Journal Volume: 37; Journal Issue: 2; Other Information: Copyright (c) 2018 SBMAC - Sociedade Brasileira de Matemática Aplicada e Computacional; Country of input: International Atomic Energy Agency (IAEA); Journal ID: ISSN 0101-8205
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICAL METHODS AND COMPUTING; ALGORITHMS; EUCLIDEAN SPACE; NEURAL NETWORKS; PATTERN RECOGNITION; TRAINING; VECTORS

Citation Formats

Manzan, José Ricardo Gonçalves, E-mail: josericardo@iftm.edu.br, Yamanaka, Keiji, Peretta, Igor Santos, E-mail: iperetta@gmail.com, Pinto, Edmilson Rodrigues, E-mail: edmilson@famat.ufu.br, Oliveira, Tiago Elias Carvalho, E-mail: tiagoecoliveira@hotmail.com, and Nomura, Shigueo. A mathematical discussion concerning the performance of multilayer perceptron-type artificial neural networks through use of orthogonal bipolar vectors. United States: N. p., 2018. Web. doi:10.1007/S40314-016-0377-X.
Manzan, José Ricardo Gonçalves, E-mail: josericardo@iftm.edu.br, Yamanaka, Keiji, Peretta, Igor Santos, E-mail: iperetta@gmail.com, Pinto, Edmilson Rodrigues, E-mail: edmilson@famat.ufu.br, Oliveira, Tiago Elias Carvalho, E-mail: tiagoecoliveira@hotmail.com, & Nomura, Shigueo. A mathematical discussion concerning the performance of multilayer perceptron-type artificial neural networks through use of orthogonal bipolar vectors. United States. doi:10.1007/S40314-016-0377-X.
Manzan, José Ricardo Gonçalves, E-mail: josericardo@iftm.edu.br, Yamanaka, Keiji, Peretta, Igor Santos, E-mail: iperetta@gmail.com, Pinto, Edmilson Rodrigues, E-mail: edmilson@famat.ufu.br, Oliveira, Tiago Elias Carvalho, E-mail: tiagoecoliveira@hotmail.com, and Nomura, Shigueo. Tue . "A mathematical discussion concerning the performance of multilayer perceptron-type artificial neural networks through use of orthogonal bipolar vectors". United States. doi:10.1007/S40314-016-0377-X.
@article{osti_22769363,
title = {A mathematical discussion concerning the performance of multilayer perceptron-type artificial neural networks through use of orthogonal bipolar vectors},
author = {Manzan, José Ricardo Gonçalves, E-mail: josericardo@iftm.edu.br and Yamanaka, Keiji and Peretta, Igor Santos, E-mail: iperetta@gmail.com and Pinto, Edmilson Rodrigues, E-mail: edmilson@famat.ufu.br and Oliveira, Tiago Elias Carvalho, E-mail: tiagoecoliveira@hotmail.com and Nomura, Shigueo},
abstractNote = {This work proposes the use of orthogonal bipolar vectors (OBV) as targets for multilayer perceptron (MLP) Artificial neural networks pertaining to the pattern recognition field. Implications involving this proposal are supported by a proper mathematical discussion. This study shows that the use of OBV as targets improves the class identification performance of MLP networks for pattern recognition with more than two classes when compared to the widely used conventional target bipolar vector (CBV). In addition to the overall performance improvement, it is possible to obtain high recognition rates with fewer training epochs, while keeping a feasible generalization with other data. This behavior is related to the Euclidean distance of the points in the output space, which is much higher in the case of OBVs. Therefore, with a greater distance between the points in the output space, the MLP network becomes less prone to associate trained outputs with incorrect targets. The mathematical discussion in this study is based on the backpropagation training algorithm. In addition to the mathematical discussion, results from experiments with real data are presented. These results follow those obtained by mathematical derivations. Real data sets used in the experiments are available from: (a) the Semeion Handwritten Digit of Machine Learning Repository, international repository; (b) Iris Image Database from the Chinese Academy of Sciences—CASIA; (c) Australian Sign Language, signs of Machine Learning Repository, international repository. Experiments were conducted to measure the mean Euclidean distance between the MLP network output subjected to a given input pattern and outputs related to other pattern classifications. Experimental results show that MLP networks trained with OBVs were able to ensure a Euclidean distance between the output and incorrect targets greater than MLP networks trained with CBVs (by up to 4 times for handwritten, by up to 12 times for iris images and by up to 10 times for Auslan signs language). As such, the adoption of OBVs as target vectors for MLPs reduces the computational effort for training the network.},
doi = {10.1007/S40314-016-0377-X},
journal = {Computational and Applied Mathematics},
issn = {0101-8205},
number = 2,
volume = 37,
place = {United States},
year = {2018},
month = {5}
}