Evolving Deep Networks Using HPC
While a large number of deep learning networks have been studiedand published that produce outstanding results on natural imagedatasets, these datasets only make up a fraction of those to whichdeep learning can be applied. These datasets include text data, audiodata, and arrays of sensors that have very different characteristicsthan natural images. As these “best” networks for natural imageshave been largely discovered through experimentation and cannotbe proven optimal on some theoretical basis, there is no reasonto believe that they are the optimal network for these drasticallydifferent datasets. Hyperparameter search is thus often a very im-portant process when applying deep learning to a new problem. Inthis work we present an evolutionary approach to searching thepossible space of network hyperparameters and construction thatcan scale to 18, 000 nodes. This approach is applied to datasets ofvarying types and characteristics where we demonstrate the abilityto rapidly find best hyperparameters in order to enable practitionersto quickly iterate between idea and result.
- Research Organization:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
- Sponsoring Organization:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- DOE Contract Number:
- AC05-00OR22725
- OSTI ID:
- 1410912
- Resource Relation:
- Conference: 3rd Workshop on Machine Learning in High Performance Computing Environments (in conjuction with SC17) - Denver, Colorado, United States of America - 11/13/2017 10:00:00 AM-11/13/2017 10:00:00 AM
- Country of Publication:
- United States
- Language:
- English
Similar Records
Evolving Larger Convolutional Layer Kernel Sizes for a Settlement Detection Deep-Learner on Summit
Multiobjective Hyperparameter Optimization for Deep Learning Interatomic Potential Training Using NSGA-II