skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Wootz: a compiler-based framework for fast CNN pruning via composability

Abstract

Convolutional Neural Networks (CNN) are widely used for Deep Learning tasks. CNN pruning is an important method to adapt a large CNN model trained on general datasets to fit a more specialized task or a smaller device. The key challenge is on deciding which filters to remove in order to maximize the quality of the pruned networks while satisfying the constraints. It is time-consuming due to the enormous configuration space and the slowness of CNN training.The problem has drawn many efforts from the machine learning field, which try to reduce the set of network configurations to explore. This work tackles the problem distinctively from a programming systems perspective, trying to speed up the evaluations of the remaining configurations through computation reuse via a compiler-based framework. We empirically uncover the existence of composability in the training of a collection of pruned CNN models, and point out the opportunities for computation reuse. We then propose composability-based CNN pruning, and design a compression-based algorithm to efficiently identify the set of CNN layers to pre-train for maximizing their reuse benefits in CNN pruning. We further develop a compiler-based framework named Wootz, which, for an arbitrary CNN, automatically generates code that builds a Teacher-Student schememore » to materialize composability-based pruning. Experiments show that network pruning enabled by Wootz shortens the state-of-art pruning process by up to 186X while producing significantly better pruning results.« less

Authors:
 [1];  [1]; ORCiD logo [2]
  1. North Carolina State University
  2. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1543204
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2019) - Phoenix, Arizona, United States of America - 6/22/2019 8:00:00 AM-6/26/2019 4:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Guan, Hui, Shen, Xipeng, and Lim, Seung-Hwan. Wootz: a compiler-based framework for fast CNN pruning via composability. United States: N. p., 2019. Web. doi:10.1145/3314221.3314652.
Guan, Hui, Shen, Xipeng, & Lim, Seung-Hwan. Wootz: a compiler-based framework for fast CNN pruning via composability. United States. doi:10.1145/3314221.3314652.
Guan, Hui, Shen, Xipeng, and Lim, Seung-Hwan. Sat . "Wootz: a compiler-based framework for fast CNN pruning via composability". United States. doi:10.1145/3314221.3314652. https://www.osti.gov/servlets/purl/1543204.
@article{osti_1543204,
title = {Wootz: a compiler-based framework for fast CNN pruning via composability},
author = {Guan, Hui and Shen, Xipeng and Lim, Seung-Hwan},
abstractNote = {Convolutional Neural Networks (CNN) are widely used for Deep Learning tasks. CNN pruning is an important method to adapt a large CNN model trained on general datasets to fit a more specialized task or a smaller device. The key challenge is on deciding which filters to remove in order to maximize the quality of the pruned networks while satisfying the constraints. It is time-consuming due to the enormous configuration space and the slowness of CNN training.The problem has drawn many efforts from the machine learning field, which try to reduce the set of network configurations to explore. This work tackles the problem distinctively from a programming systems perspective, trying to speed up the evaluations of the remaining configurations through computation reuse via a compiler-based framework. We empirically uncover the existence of composability in the training of a collection of pruned CNN models, and point out the opportunities for computation reuse. We then propose composability-based CNN pruning, and design a compression-based algorithm to efficiently identify the set of CNN layers to pre-train for maximizing their reuse benefits in CNN pruning. We further develop a compiler-based framework named Wootz, which, for an arbitrary CNN, automatically generates code that builds a Teacher-Student scheme to materialize composability-based pruning. Experiments show that network pruning enabled by Wootz shortens the state-of-art pruning process by up to 186X while producing significantly better pruning results.},
doi = {10.1145/3314221.3314652},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2019},
month = {6}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: