Ramifications of Evolving Misbehaving Convolutional Neural Network Kernel and Batch Sizes
Deep-learners have many hyper-parameters including learning rate, batch size, kernel size - all playing a significant role toward estimating high quality models. Discovering useful hyper-parameter guidelines is an active area of research, though the state of the art generally uses a brute force, uniform grid approach or random search for finding ideal settings. We share the preliminary results of using an alternative approach to deep learner hyper-parameter tuning that uses an evolutionary algorithm to improve the accuracy of a deep-learner models used in satellite imagery building footprint detection. We found that the kernel and batch size hyper-parameters surprisingly differed from sizes arrived at via a brute force uniform grid approach. These differences suggest a novel role for evolutionary algorithms in determining the number of convolution layers, as well as smaller batch sizes in improving deep-learner models.
- Research Organization:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-00OR22725
- OSTI ID:
- 1495999
- Resource Relation:
- Conference: Machine Learning in HPC Environments (MLHPC 2018) - Dallas, Texas, United States of America - 11/11/2018 10:00:00 AM-11/16/2018 10:00:00 AM
- Country of Publication:
- United States
- Language:
- English
Similar Records
Integrated Model, Batch and Domain Parallelism in Training Neural Networks
Troubleshooting deep-learner training data problems using an evolutionary algorithm on Summit