skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Evolving Deep Networks Using HPC

Abstract

While a large number of deep learning networks have been studied and published that produce outstanding results on natural image datasets, these datasets only make up a fraction of those to which deep learning can be applied. These datasets include text data, audio data, and arrays of sensors that have very different characteristics than natural images. As these “best” networks for natural images have been largely discovered through experimentation and cannot be proven optimal on some theoretical basis, there is no reason to believe that they are the optimal network for these drastically different datasets. Hyperparameter search is thus often a very important process when applying deep learning to a new problem. In this work we present an evolutionary approach to searching the possible space of network hyperparameters and construction that can scale to 18, 000 nodes. This approach is applied to datasets of varying types and characteristics where we demonstrate the ability to rapidly find best hyperparameters in order to enable practitioners to quickly iterate between idea and result.

Authors:
 [1];  [1];  [1];  [1];  [1];  [1];  [1];  [2];  [3]
  1. ORNL, Oak Ridge
  2. Fermilab
  3. Santa Maria U., Valparaiso
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
OSTI Identifier:
1414394
Report Number(s):
FERMILAB-CONF-17-567-CD-ND
1644275
DOE Contract Number:  
AC02-07CH11359
Resource Type:
Conference
Country of Publication:
United States
Language:
English

Citation Formats

Young, Steven R., Rose, Derek C., Johnston, Travis, Heller, William T., Karnowski, thomas P., Potok, Thomas E., Patton, Robert M., Perdue, Gabriel, and Miller, Jonathan. Evolving Deep Networks Using HPC. United States: N. p., 2017. Web. doi:10.1145/3146347.3146355.
Young, Steven R., Rose, Derek C., Johnston, Travis, Heller, William T., Karnowski, thomas P., Potok, Thomas E., Patton, Robert M., Perdue, Gabriel, & Miller, Jonathan. Evolving Deep Networks Using HPC. United States. doi:10.1145/3146347.3146355.
Young, Steven R., Rose, Derek C., Johnston, Travis, Heller, William T., Karnowski, thomas P., Potok, Thomas E., Patton, Robert M., Perdue, Gabriel, and Miller, Jonathan. Sun . "Evolving Deep Networks Using HPC". United States. doi:10.1145/3146347.3146355. https://www.osti.gov/servlets/purl/1414394.
@article{osti_1414394,
title = {Evolving Deep Networks Using HPC},
author = {Young, Steven R. and Rose, Derek C. and Johnston, Travis and Heller, William T. and Karnowski, thomas P. and Potok, Thomas E. and Patton, Robert M. and Perdue, Gabriel and Miller, Jonathan},
abstractNote = {While a large number of deep learning networks have been studied and published that produce outstanding results on natural image datasets, these datasets only make up a fraction of those to which deep learning can be applied. These datasets include text data, audio data, and arrays of sensors that have very different characteristics than natural images. As these “best” networks for natural images have been largely discovered through experimentation and cannot be proven optimal on some theoretical basis, there is no reason to believe that they are the optimal network for these drastically different datasets. Hyperparameter search is thus often a very important process when applying deep learning to a new problem. In this work we present an evolutionary approach to searching the possible space of network hyperparameters and construction that can scale to 18, 000 nodes. This approach is applied to datasets of varying types and characteristics where we demonstrate the ability to rapidly find best hyperparameters in order to enable practitioners to quickly iterate between idea and result.},
doi = {10.1145/3146347.3146355},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2017},
month = {1}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Works referenced in this record:

The <mml:math altimg="si0011.gif" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:mi>MINER</mml:mi><mml:mi mathvariant="normal">ν</mml:mi><mml:mi mathvariant="normal">A</mml:mi></mml:math> data acquisition system and infrastructure
journal, December 2012

  • Perdue, G. N.; Bagby, L.; Baldin, B.
  • Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 694
  • DOI: 10.1016/j.nima.2012.08.024

A convolutional neural network neutrino event classifier
journal, September 2016


ImageNet Large Scale Visual Recognition Challenge
journal, April 2015

  • Russakovsky, Olga; Deng, Jia; Su, Hao
  • International Journal of Computer Vision, Vol. 115, Issue 3
  • DOI: 10.1007/s11263-015-0816-y

Evolving Neural Networks through Augmenting Topologies
journal, June 2002


A survey on deep learning in medical image analysis
journal, December 2017