WavPool: A New Block for Deep Neural Networks

Conference · Wed Jun 14 04:00:00 EDT 2023

OSTI ID:1988456

McDermott, Samuel D.; Voetberg, M.; Nord, Brian

Modern deep neural networks comprise many operational layers, such as dense or convolutional layers, which are often collected into blocks. In this work, we introduce a new, wavelet-transform-based network architecture that we call the multi-resolution perceptron: by adding a pooling layer, we create a new network block, the WavPool. The first step of the multi-resolution perceptron is transforming the data into its multi-resolution decomposition form by convolving the input data with filters of fixed coefficients but increasing size. Following image processing techniques, we are able to make scale and spatial information simultaneously accessible to the network without increasing the size of the data vector. WavPool outperforms a similar multilayer perceptron while using fewer parameters, and outperforms a comparable convolutional neural network by ~ 10% on relative accuracy on CIFAR-10.

• Research Organization: Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
• DOE Contract Number: AC02-07CH11359
• OSTI ID: 1988456
• Report Number(s): FERMILAB-CONF-23-278-CSAID; arXiv:2306.08734; oai:inspirehep.net:2669167
• Country of Publication: United States
• Language: English