Optimizing neural networks

Patton, Robert M.; Young, Steven R.; Rose, Derek C.; Karnowski, Thomas P.; Lim, Seung-Hwan; Potok, Thomas E.; Johnston, J. Travis

Title: Optimizing neural networks

Abstract

A system and method design and optimize neural networks. The system and method include a data store that stores a plurality of gene vectors that represent diverse and distinct neural networks and an evaluation queue stored with the plurality of gene vectors. Secondary nodes construct, train, and evaluate the neural network and automatically render a plurality of fitness values asynchronously. A primary node executes a gene amplification on a select plurality of gene vectors, a crossing-over of the amplified gene vectors, and a mutation of the crossing-over gene vectors automatically and asynchronously, which are then transmitted to the evaluation queue. The process continuously repeats itself by processing the gene vectors inserted into the evaluation queue until a fitness level is reached, a network's accuracy level plateaus, a processing time period expires, or when some stopping condition or performance metric is met or exceeded.

Inventors:: Patton, Robert M.; Young, Steven R.; Rose, Derek C.; Karnowski, Thomas P.; Lim, Seung-Hwan; Potok, Thomas E.; Johnston, J. Travis

Publication Date:: Tue Aug 30 00:00:00 EDT 2022

Research Org.:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1925038

Patent Number(s):: 11,429,865

Application Number:: 16/265,252

Assignee:: UT-Battelle, LLC (Oak Ridge, TN)

DOE Contract Number:: AC05-00OR22725

Resource Type:: Patent

Resource Relation:: Patent File Date: 02/01/2019

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Patton, Robert M., Young, Steven R., Rose, Derek C., Karnowski, Thomas P., Lim, Seung-Hwan, Potok, Thomas E., and Johnston, J. Travis. Optimizing neural networks.  United States: N. p., 2022. 
        Web.

Copy to clipboard


                    Patton, Robert M., Young, Steven R., Rose, Derek C., Karnowski, Thomas P., Lim, Seung-Hwan, Potok, Thomas E., & Johnston, J. Travis. Optimizing neural networks.  United States.

Copy to clipboard


                    Patton, Robert M., Young, Steven R., Rose, Derek C., Karnowski, Thomas P., Lim, Seung-Hwan, Potok, Thomas E., and Johnston, J. Travis. 2022.  
        "Optimizing neural networks".  United States.  https://www.osti.gov/servlets/purl/1925038.

Copy to clipboard


                    
@article{osti_1925038,

  title        = {Optimizing neural networks},

  author       = {Patton, Robert M. and Young, Steven R. and Rose, Derek C. and Karnowski, Thomas P. and Lim, Seung-Hwan and Potok, Thomas E. and Johnston, J. Travis},

  abstractNote = {A system and method design and optimize neural networks. The system and method include a data store that stores a plurality of gene vectors that represent diverse and distinct neural networks and an evaluation queue stored with the plurality of gene vectors. Secondary nodes construct, train, and evaluate the neural network and automatically render a plurality of fitness values asynchronously. A primary node executes a gene amplification on a select plurality of gene vectors, a crossing-over of the amplified gene vectors, and a mutation of the crossing-over gene vectors automatically and asynchronously, which are then transmitted to the evaluation queue. The process continuously repeats itself by processing the gene vectors inserted into the evaluation queue until a fitness level is reached, a network's accuracy level plateaus, a processing time period expires, or when some stopping condition or performance metric is met or exceeded.},

  doi          = {},

  url          = {https://www.osti.gov/biblio/1925038},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {Tue Aug 30 00:00:00 EDT 2022},

  month        = {Tue Aug 30 00:00:00 EDT 2022}

}

Copy to clipboard

Patent:

View Patent

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Execution of a genetic algorithm with variable evolutionary weights of topological parameters for neural network generation and training
patent-application, March 2019

Andoni, Sari; Moore, Keith D.; Bonab, Elmira M.
US Patent Application 15/699353; 20190080240
https://doi.org/https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/20190080240

Evolving Deep Networks Using HPC
conference, January 2017

Young, Steven R.; Rose, Derek C.; Johnston, Travis
Proceedings of the Machine Learning on HPC Environments - MLHPC'17
https://doi.org/10.1145/3146347.3146355

Vertex reconstruction of neutrino interactions using deep learning
conference, May 2017

Terwilliger, Adam M.; Perdue, Gabriel N.; Isele, David
2017 International Joint Conference on Neural Networks (IJCNN)
https://doi.org/10.1109/IJCNN.2017.7966131

GPU-based asynchronous particle swarm optimization
conference, July 2011

Mussi, Luca; Nashed, Youssef S. G.; Cagnoni, Stefano
Proceedings of the 13th annual conference on Genetic and evolutionary computation
https://doi.org/10.1145/2001576.2001786

Similar records in OSTI.GOV collections:

Application of Genetic Algorithm in determining optimal backpropagation neural network architecture for ground-level ozone forecasting model

Conference Chang, H; Liu, Q

Backpropagation neural networks have shown better prediction accuracy than linear regression in summertime ground-level ozone forecasting. It is highly desirable to have a distinct, optimal neural network model presenting the distinct properties of each region. Before normalizing values of inputs and output to be used in the neural networks, they may be transformed, such as taking logarithm or their reciprocals, to provide distinct perspectives of studied regions. In this paper, the authors propose a system consisting of a backpropagation neural network and a Genetic Algorithm unit to examine efficiently the effects of form transformation and different number of hidden nodesmore »« less
Combustion control using spiking neural networks

Patent Puente, Brian; Kaul, Brian; Schuman, Catherine; ...

A system that controls a combustion engine stores network vectors in a memory that represent diverse and distinct spiking neural networks. The system decodes the network vectors and trains and evaluates the spiking neural networks. The system duplicates selected network vectors and crosses-over the duplicated network vectors that represent modified spiking neural networks. The system mutates the crossed-over duplicated network vectors by randomly modifying one or more portions of the crossing-over duplicated network vectors. The system meter exhaust gas into an intake manifold when an engine temperature exceeds a threshold, an engine load exceeds a threshold, an engine's rotation-per-minute ratemore »« less
Full Text Available
Efficient I/O for Neural Network Training with Compressed Data

Conference Zhang, Zhao; Huang, Lei; Pauloski, J.; ...

FanStore is a shared object store that enables efficient and scalable neural network training on supercomputers. By providing a global cache layer on node-local burst buffers using a compressed representation, it significantly enhances the processing capability of deep learning (DL) applications on existing hardware. In addition, FanStore allows POSIX-compliant file access to the compressed data in user space. We investigate the tradeoff between runtime overhead and data compression ratio using real-world datasets and applications, and propose a compressor selection algorithm to maximize storage capacity given performance constraints. We consider both asynchronous (i.e., with prefetching) and synchronous I/O strategies, and proposemore »« less
https://doi.org/10.1109/IPDPS47924.2020.00050
New learning and control algorithms for neural networks

Miscellaneous Youn, C

Neural networks offer distributed processing power, error correcting capability and structural simplicity of the basic computing element. Neural networks have been found to be attractive for applications such as associative memory, robotics, image processing, speech understanding and optimization. Neural networks are self-adaptive systems that try to configure themselves to store new information. This dissertation investigates two approaches to improve performance: better learning and supervisory control. A new learning algorithm called the Correlation Continuous Unlearning (CCU) algorithm is presented. It is based on the idea of removing undesirable information that is encountered during the learning period. The control methods proposed inmore »« less
A neural network model to predict lung radiation-induced pneumonitis

Journal Article Shifeng, Chen; Sumin, Zhou; Junan, Zhang; ... - Medical Physics

A feed-forward neural network was investigated to predict the occurrence of lung radiation-induced Grade 2+ pneumonitis. The database consisted of 235 patients with lung cancer treated using radiotherapy, of whom 34 were diagnosed with Grade 2+ pneumonitis at follow-up. The network was constructed using an algorithm that alternately grew and pruned it, starting from the smallest possible network, until a satisfactory solution was found. The weights and biases of the network were computed using the error back-propagation approach. Momentum and variable leaning techniques were used to speed convergence. Using the growing/pruning approach, the network selected features from 66 dose andmore »« less
https://doi.org/10.1118/1.2759601

Similar Records

Title: Optimizing neural networks

Abstract

Citation Formats

Execution of a genetic algorithm with variable evolutionary weights of topological parameters for neural network generation and training patent-application, March 2019

Evolving Deep Networks Using HPC conference, January 2017

Vertex reconstruction of neutrino interactions using deep learning conference, May 2017

GPU-based asynchronous particle swarm optimization conference, July 2011

Execution of a genetic algorithm with variable evolutionary weights of topological parameters for neural network generation and training
patent-application, March 2019

Evolving Deep Networks Using HPC
conference, January 2017

Vertex reconstruction of neutrino interactions using deep learning
conference, May 2017

GPU-based asynchronous particle swarm optimization
conference, July 2011