skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Composability-Centered Convolutional Neural Network Pruning

Abstract

This work studies the composability of the building blocks of structural CNN models (e.g., GoogleLeNet and Residual Networks) in the context of network pruning. We empirically validate that a network composed of pre-trained building blocks (e.g. residual blocks and Inception modules) not only gives a better initial setting for training, but also allows the training process to converge at a significantly higher accuracy in much less time. Based on that insight, we propose a composability-centered design for CNN network pruning. Experiments show that this new scheme shortens the configuration process in CNN network pruning by up to 186.8X for ResNet-50 and up to 30.2X for Inception-V3, and meanwhile, the models it finds that meet the accuracy requirement are significantly more compact than those found by default schemes.

Authors:
 [1];  [1];  [2];  [2]
  1. North Carolina State Univ., Raleigh, NC (United States)
  2. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1427608
Report Number(s):
ORNL/TM-2018/777
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Guan, Hui, Shen, Xipeng, Lim, Seung-Hwan, and Patton, Robert M. Composability-Centered Convolutional Neural Network Pruning. United States: N. p., 2018. Web. doi:10.2172/1427608.
Guan, Hui, Shen, Xipeng, Lim, Seung-Hwan, & Patton, Robert M. Composability-Centered Convolutional Neural Network Pruning. United States. doi:10.2172/1427608.
Guan, Hui, Shen, Xipeng, Lim, Seung-Hwan, and Patton, Robert M. Thu . "Composability-Centered Convolutional Neural Network Pruning". United States. doi:10.2172/1427608. https://www.osti.gov/servlets/purl/1427608.
@article{osti_1427608,
title = {Composability-Centered Convolutional Neural Network Pruning},
author = {Guan, Hui and Shen, Xipeng and Lim, Seung-Hwan and Patton, Robert M.},
abstractNote = {This work studies the composability of the building blocks of structural CNN models (e.g., GoogleLeNet and Residual Networks) in the context of network pruning. We empirically validate that a network composed of pre-trained building blocks (e.g. residual blocks and Inception modules) not only gives a better initial setting for training, but also allows the training process to converge at a significantly higher accuracy in much less time. Based on that insight, we propose a composability-centered design for CNN network pruning. Experiments show that this new scheme shortens the configuration process in CNN network pruning by up to 186.8X for ResNet-50 and up to 30.2X for Inception-V3, and meanwhile, the models it finds that meet the accuracy requirement are significantly more compact than those found by default schemes.},
doi = {10.2172/1427608},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Thu Feb 15 00:00:00 EST 2018},
month = {Thu Feb 15 00:00:00 EST 2018}
}

Technical Report:

Save / Share: