skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Ramifications of Evolving Misbehaving Convolutional Neural Network Kernel and Batch Sizes

Abstract

Deep-learners have many hyper-parameters including learning rate, batch size, kernel size - all playing a significant role toward estimating high quality models. Discovering useful hyper-parameter guidelines is an active area of research, though the state of the art generally uses a brute force, uniform grid approach or random search for finding ideal settings. We share the preliminary results of using an alternative approach to deep learner hyper-parameter tuning that uses an evolutionary algorithm to improve the accuracy of a deep-learner models used in satellite imagery building footprint detection. We found that the kernel and batch size hyper-parameters surprisingly differed from sizes arrived at via a brute force uniform grid approach. These differences suggest a novel role for evolutionary algorithms in determining the number of convolution layers, as well as smaller batch sizes in improving deep-learner models.

Authors:
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Laboratory, Oak Ridge Leadership Computing Facility (OLCF); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1495999
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: Machine Learning in HPC Environments (MLHPC 2018) - Dallas, Texas, United States of America - 11/11/2018 10:00:00 AM-11/16/2018 10:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Coletti, Mark A., Lunga, Dalton D., Berres, Anne S., Sanyal, Jibonananda, and Rose, Amy. Ramifications of Evolving Misbehaving Convolutional Neural Network Kernel and Batch Sizes. United States: N. p., 2018. Web. doi:10.1109/MLHPC.2018.8638644.
Coletti, Mark A., Lunga, Dalton D., Berres, Anne S., Sanyal, Jibonananda, & Rose, Amy. Ramifications of Evolving Misbehaving Convolutional Neural Network Kernel and Batch Sizes. United States. doi:10.1109/MLHPC.2018.8638644.
Coletti, Mark A., Lunga, Dalton D., Berres, Anne S., Sanyal, Jibonananda, and Rose, Amy. Thu . "Ramifications of Evolving Misbehaving Convolutional Neural Network Kernel and Batch Sizes". United States. doi:10.1109/MLHPC.2018.8638644. https://www.osti.gov/servlets/purl/1495999.
@article{osti_1495999,
title = {Ramifications of Evolving Misbehaving Convolutional Neural Network Kernel and Batch Sizes},
author = {Coletti, Mark A. and Lunga, Dalton D. and Berres, Anne S. and Sanyal, Jibonananda and Rose, Amy},
abstractNote = {Deep-learners have many hyper-parameters including learning rate, batch size, kernel size - all playing a significant role toward estimating high quality models. Discovering useful hyper-parameter guidelines is an active area of research, though the state of the art generally uses a brute force, uniform grid approach or random search for finding ideal settings. We share the preliminary results of using an alternative approach to deep learner hyper-parameter tuning that uses an evolutionary algorithm to improve the accuracy of a deep-learner models used in satellite imagery building footprint detection. We found that the kernel and batch size hyper-parameters surprisingly differed from sizes arrived at via a brute force uniform grid approach. These differences suggest a novel role for evolutionary algorithms in determining the number of convolution layers, as well as smaller batch sizes in improving deep-learner models.},
doi = {10.1109/MLHPC.2018.8638644},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2018},
month = {11}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: