Evolving Larger Convolutional Layer Kernel Sizes for a Settlement Detection Deep-Learner on Summit

Conference · Fri Nov 01 00:00:00 EDT 2019

DOI:https://doi.org/10.1109/DLS49591.2019.00010· OSTI ID:1631245

Coletti, Mark  ^[1]; Lunga, Dalton  ^[1]; Bassett, Jeffrey K. ^[2]; Rose, Amy  ^[1]

---

1. ORNL
2. George Mason University, Virginia

Deep-learner hyper-parameters, such as kernel sizes, batch sizes, and learning rates, can significantly influence the quality of trained models. The state of the art for finding optimal hyper-parameters generally uses a brute force, grid search approach, random search, or Bayesian-based optimization among other techniques. We applied an evolutionary algorithm to optimize kernel sizes for a convolutional neural network used to detect settlements in satellite imagery. Usually convolutional layer kernel sizes are small - typically one, three, or five - but we found that the system converged at, or near, kernel sizes of nine for the last convolutional layer, and that this occurred for multiple runs using two different datasets. Moreover, the larger kernel sizes had fewer false positives than the 3x3 kernel sizes found as optimal via a brute force uniform grid search. This suggests that this large kernel size may be leveraging patterns found in larger areal features in the source imagery, and that this may be generalized as possible guidance for similar remote sensing deep-learning tasks.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1631245
• Resource Relation: Conference: The International Conference for High Performance Computing, Networking, Storage, and Analysis 2019 (SC19) - Denver, Colorado, United States of America - 11/17/2019 10:00:00 AM-11/22/2019 10:00:00 AM
• Country of Publication: United States
• Language: English