Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Evolving Deep Networks Using HPC

Conference ·
 [1];  [1];  [1];  [1];  [1];  [1];  [1];  [2];  [3]
  1. Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
  2. Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
  3. Universidad Técnica Federico Santa María, Valparaiso (Chile)
+ Show Author Affiliations
While a large number of deep learning networks have been studiedand published that produce outstanding results on natural imagedatasets, these datasets only make up a fraction of those to whichdeep learning can be applied. These datasets include text data, audiodata, and arrays of sensors that have very different characteristicsthan natural images. As these “best” networks for natural imageshave been largely discovered through experimentation and cannotbe proven optimal on some theoretical basis, there is no reasonto believe that they are the optimal network for these drasticallydifferent datasets. Hyperparameter search is thus often a very im-portant process when applying deep learning to a new problem. Inthis work we present an evolutionary approach to searching thepossible space of network hyperparameters and construction thatcan scale to 18, 000 nodes. This approach is applied to datasets ofvarying types and characteristics where we demonstrate the abilityto rapidly find best hyperparameters in order to enable practitionersto quickly iterate between idea and result.
Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
DOE Contract Number:
AC05-00OR22725
OSTI ID:
1410912
Country of Publication:
United States
Language:
English

References (14)

Recent advances in deep learning for speech research at Microsoft conference May 2013
Practical Recommendations for Gradient-Based Training of Deep Architectures book January 2012
The MINERνA data acquisition system and infrastructure
  • Perdue, G. N.; Bagby, L.; Baldin, B.
  • Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 694 https://doi.org/10.1016/j.nima.2012.08.024
journal December 2012
Deep learning for computational biology journal July 2016
Evolving Neural Networks through Augmenting Topologies journal June 2002
A survey on deep learning in medical image analysis journal December 2017
Optimizing deep learning hyper-parameters through an evolutionary algorithm
  • Young, Steven R.; Rose, Derek C.; Karnowski, Thomas P.
  • Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments - MLHPC '15 https://doi.org/10.1145/2834892.2834896
conference January 2015
Deep Residual Learning for Image Recognition conference June 2016
An empirical evaluation of deep architectures on problems with many factors of variation conference January 2007
A convolutional neural network neutrino event classifier journal September 2016
Going deeper with convolutions conference June 2015
ImageNet Large Scale Visual Recognition Challenge journal April 2015
Caffe: Convolutional Architecture for Fast Feature Embedding conference January 2014
Vertex reconstruction of neutrino interactions using deep learning conference May 2017

Similar Records

Related Subjects