skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Pruning Neural Networks with Distribution Estimation Algorithms

Conference ·

This paper describes the application of four evolutionary algorithms to the pruning of neural networks used in classification problems. Besides of a simple genetic algorithm (GA), the paper considers three distribution estimation algorithms (DEAs): a compact GA, an extended compact GA, and the Bayesian Optimization Algorithm. The objective is to determine if the DEAs present advantages over the simple GA in terms of accuracy or speed in this problem. The experiments used a feed forward neural network trained with standard back propagation and public-domain and artificial data sets. The pruned networks seemed to have better or equal accuracy than the original fully-connected networks. Only in a few cases, pruning resulted in less accurate networks. We found few differences in the accuracy of the networks pruned by the four EAs, but found important differences in the execution time. The results suggest that a simple GA with a small population might be the best algorithm for pruning networks on the data sets we tested.

Research Organization:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Organization:
USDOE
DOE Contract Number:
W-7405-ENG-48
OSTI ID:
15004416
Report Number(s):
UCRL-JC-151521; TRN: US201015%%592
Resource Relation:
Journal Volume: 2723; Conference: Genetic and Evolutionary Computation Conference, Chicago, IL, Jul 12 - Jul 16, 2003
Country of Publication:
United States
Language:
English

References (10)

Pruning algorithms-a survey journal January 1993
Evolving artificial neural networks journal January 1999
Artificial Neural Networks Design using Evolutionary Algorithms book January 2003
Genetic algorithms and neural networks: optimizing connections and connectivity journal August 1990
Combined 5 × 2 cv F Test for Comparing Supervised Classification Learning Algorithms journal November 1999
The Equation for Response to Selection and Its Use for Prediction journal September 1997
Feature Subset Selection by Bayesian network-based optimization journal October 2000
The compact genetic algorithm
  • Harik, G. R.; Lobo, F. G.; Goldberg, D. E.
  • 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360) https://doi.org/10.1109/ICEC.1998.700083
conference January 1998
FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions journal December 1999
Scalability Problems of Simple Genetic Algorithms journal December 1999

Similar Records

Related Subjects

99
ACCURACY
ALGORITHMS
CLASSIFICATION
DISTRIBUTION
GENETICS
NEURAL NETWORKS
OPTIMIZATION
VELOCITY