Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Weight Quantization for Multi-layer Perceptrons Using Soft Weight Sharing

Summary: Weight Quantization for Multi-layer Perceptrons
Using Soft Weight Sharing
Fatih K¨oksal1
, Ethem Alpaydin1
, and G¨unhan D¨undar2
Department of Computer Engineering
Department of Electrical and Electronics Engineering
Bogazi¸ci University, Istanbul Turkey
Abstract. We propose a novel approach for quantizing the weights of
a multi-layer perceptron (MLP) for efficient VLSI implementation. Our
approach uses soft weight sharing, previously proposed for improved gen-
eralization and considers the weights not as constant numbers but as
random variables drawn from a Gaussian mixture distribution; which in-
cludes as its special cases k-means clustering and uniform quantization.
This approach couples the training of weights for reduced error with their
quantization. Simulations on synthetic and real regression and classifi-
cation data sets compare various quantization schemes and demonstrate
the advantage of the coupled training of distribution parameters.


Source: Alpaydın, Ethem - Department of Computer Engineering, Bogaziçi University


Collections: Computer Technologies and Information Sciences