skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Evolving Deep Networks Using HPC

Abstract

While a large number of deep learning networks have been studiedand published that produce outstanding results on natural imagedatasets, these datasets only make up a fraction of those to whichdeep learning can be applied. These datasets include text data, audiodata, and arrays of sensors that have very different characteristicsthan natural images. As these “best” networks for natural imageshave been largely discovered through experimentation and cannotbe proven optimal on some theoretical basis, there is no reasonto believe that they are the optimal network for these drasticallydifferent datasets. Hyperparameter search is thus often a very im-portant process when applying deep learning to a new problem. Inthis work we present an evolutionary approach to searching thepossible space of network hyperparameters and construction thatcan scale to 18, 000 nodes. This approach is applied to datasets ofvarying types and characteristics where we demonstrate the abilityto rapidly find best hyperparameters in order to enable practitionersto quickly iterate between idea and result.

Authors:
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1];  [2]; ORCiD logo [3]
  1. ORNL
  2. Fermi National Accelerator Laboratory (FNAL)
  3. Universidad Técnica Federico Santa María
Publication Date:
Research Org.:
Oak Ridge National Laboratory, Oak Ridge Leadership Computing Facility (OLCF); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1410912
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: 3rd Workshop on Machine Learning in High Performance Computing Environments (in conjuction with SC17) - Denver, Colorado, United States of America - 11/13/2017 10:00:00 AM-11/13/2017 10:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Young, Steven R., Rose, Derek, Johnston, Travis, Heller, William T., Karnowski, Thomas, Potok, Thomas E., Patton, Robert M., Perdue, Gabriel, and Miller, Jonathan. Evolving Deep Networks Using HPC. United States: N. p., 2017. Web. doi:10.1145/3146347.3146355.
Young, Steven R., Rose, Derek, Johnston, Travis, Heller, William T., Karnowski, Thomas, Potok, Thomas E., Patton, Robert M., Perdue, Gabriel, & Miller, Jonathan. Evolving Deep Networks Using HPC. United States. doi:10.1145/3146347.3146355.
Young, Steven R., Rose, Derek, Johnston, Travis, Heller, William T., Karnowski, Thomas, Potok, Thomas E., Patton, Robert M., Perdue, Gabriel, and Miller, Jonathan. Wed . "Evolving Deep Networks Using HPC". United States. doi:10.1145/3146347.3146355.
@article{osti_1410912,
title = {Evolving Deep Networks Using HPC},
author = {Young, Steven R. and Rose, Derek and Johnston, Travis and Heller, William T. and Karnowski, Thomas and Potok, Thomas E. and Patton, Robert M. and Perdue, Gabriel and Miller, Jonathan},
abstractNote = {While a large number of deep learning networks have been studiedand published that produce outstanding results on natural imagedatasets, these datasets only make up a fraction of those to whichdeep learning can be applied. These datasets include text data, audiodata, and arrays of sensors that have very different characteristicsthan natural images. As these “best” networks for natural imageshave been largely discovered through experimentation and cannotbe proven optimal on some theoretical basis, there is no reasonto believe that they are the optimal network for these drasticallydifferent datasets. Hyperparameter search is thus often a very im-portant process when applying deep learning to a new problem. Inthis work we present an evolutionary approach to searching thepossible space of network hyperparameters and construction thatcan scale to 18, 000 nodes. This approach is applied to datasets ofvarying types and characteristics where we demonstrate the abilityto rapidly find best hyperparameters in order to enable practitionersto quickly iterate between idea and result.},
doi = {10.1145/3146347.3146355},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2017},
month = {11}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Works referenced in this record:

The <mml:math altimg="si0011.gif" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:mi>MINER</mml:mi><mml:mi mathvariant="normal">ν</mml:mi><mml:mi mathvariant="normal">A</mml:mi></mml:math> data acquisition system and infrastructure
journal, December 2012

  • Perdue, G. N.; Bagby, L.; Baldin, B.
  • Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 694
  • DOI: 10.1016/j.nima.2012.08.024

A convolutional neural network neutrino event classifier
journal, September 2016


ImageNet Large Scale Visual Recognition Challenge
journal, April 2015

  • Russakovsky, Olga; Deng, Jia; Su, Hao
  • International Journal of Computer Vision, Vol. 115, Issue 3
  • DOI: 10.1007/s11263-015-0816-y

Evolving Neural Networks through Augmenting Topologies
journal, June 2002