A mixed-scale dense convolutional neural network for image analysis

Pelt, Daniël M.; Sethian, James A.

doi:10.1073/pnas.1715832114

Title: A mixed-scale dense convolutional neural network for image analysis

Journal Article · 26 December 2017 · Proceedings of the National Academy of Sciences of the United States of America

DOI: https://doi.org/10.1073/pnas.1715832114 · OSTI ID:1414877

Pelt, Daniël M. ^[1]; Sethian, James A. ^[2]

Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Univ. of California, Berkeley, CA (United States)

We report that deep convolutional neural networks have been successfully applied to many image-processing problems in recent works. Popular network architectures often add additional operations and connections to the standard architecture to enable training deeper networks. To achieve accurate results in practice, a large number of trainable parameters are often required. Here, we introduce a network architecture based on using dilated convolutions to capture features at different image scales and densely connecting all feature maps with each other. The resulting architecture is able to achieve accurate results with relatively few parameters and consists of a single set of operations, making it easier to implement, train, and apply in practice, and automatically adapts to different problems. Lastly, we compare results of the proposed network architecture with popular existing architectures for several segmentation problems, showing that the proposed architecture is able to achieve accurate results with fewer parameters, with a reduced risk of overfitting the training data.

View Journal Article

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Basic Energy Sciences (BES); USDOE Office of Science (SC), Biological and Environmental Research (BER); USDOE

Grant/Contract Number:: AC02-05CH11231; AC03-76SF00098

OSTI ID:: 1414877

Alternate ID(s):: OSTI ID: 1485062

Journal Information:: Proceedings of the National Academy of Sciences of the United States of America, Vol. 115, Issue 2; ISSN 0027-8424

Publisher:: National Academy of Sciences, Washington, DC (United States)Copyright Statement

Country of Publication:: United States

Language:: English

References (19)

Superparsing: Scalable Nonparametric Image Parsing with Superpixels Tighe, Joseph; Lazebnik, Svetlana International Journal of Computer Vision, Vol. 101, Issue 2 https://doi.org/10.1007/s11263-012-0574-z	journal	October 2012
What is the best multi-stage architecture for object recognition? Jarrett, Kevin; Kavukcuoglu, Koray; Ranzato, Marc' Aurelio 2009 IEEE 12th International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2009.5459469	conference	September 2009
Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Valmet: A New Validation Tool for Assessing and Improving 3D Object Segmentation Gerig, Guido; Jomier, Matthieu; Chakos, Miranda Medical Image Computing and Computer-Assisted Intervention – MICCAI 2001 https://doi.org/10.1007/3-540-45468-3_62	book	January 2001
Generalizing the Hough transform to detect arbitrary shapes Ballard, D. H. Pattern Recognition, Vol. 13, Issue 2 https://doi.org/10.1016/0031-3203(81)90009-1	journal	January 1981
ImageNet: A large-scale hierarchical image database Deng, Jia; Dong, Wei; Socher, Richard 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2009 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2009.5206848	conference	June 2009
Radio frequency interference mitigation using deep convolutional neural networks Akeret, J.; Chang, C.; Lucchi, A. Astronomy and Computing, Vol. 18 https://doi.org/10.1016/j.ascom.2017.01.002	journal	January 2017
Caffe: Convolutional Architecture for Fast Feature Embedding Jia, Yangqing; Shelhamer, Evan; Donahue, Jeff Proceedings of the ACM International Conference on Multimedia - MM '14 https://doi.org/10.1145/2647868.2654889	conference	January 2014
U-Net: Convolutional Networks for Biomedical Image Segmentation Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III https://doi.org/10.1007/978-3-319-24574-4_28	book	November 2015
PyCUDA and PyOpenCL: A scripting-based approach to GPU run-time code generation Klöckner, Andreas; Pinto, Nicolas; Lee, Yunsup Parallel Computing, Vol. 38, Issue 3 https://doi.org/10.1016/j.parco.2011.09.001	journal	March 2012
Densely Connected Convolutional Networks Huang, Gao; Liu, Zhuang; Maaten, Laurens van der 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.243	conference	July 2017
Analyzing the Performance of Multilayer Neural Networks for Object Recognition Agrawal, Pulkit; Girshick, Ross; Malik, Jitendra Computer Vision – ECCV 2014 https://doi.org/10.1007/978-3-319-10584-0_22	book	January 2014
Deep Residual Learning for Image Recognition He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.90	conference	June 2016
Fully Convolutional Networks for Semantic Segmentation Shelhamer, Evan; Long, Jonathan; Darrell, Trevor IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, Issue 4 https://doi.org/10.1109/TPAMI.2016.2572683	journal	April 2017
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing 2015 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2015.123	conference	December 2015
Semantic object classes in video: A high-definition ground truth database Brostow, Gabriel J.; Fauqueur, Julien; Cipolla, Roberto Pattern Recognition Letters, Vol. 30, Issue 2 https://doi.org/10.1016/j.patrec.2008.04.005	journal	January 2009
What, Where and How Many? Combining Object Detectors and CRFs Ladický, Ľubor; Sturgess, Paul; Alahari, Karteek Computer Vision – ECCV 2010 https://doi.org/10.1007/978-3-642-15561-1_31	book	January 2010
Automated detection of pulmonary nodules in helical CT images based on an improved template-matching technique IEEE Transactions on Medical Imaging, Vol. 20, Issue 7 https://doi.org/10.1109/42.932744	journal	July 2001
Image-to-Image Translation with Conditional Adversarial Networks Isola, Phillip; Zhu, Jun-Yan; Zhou, Tinghui 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.632	conference	July 2017

Cited By (33)

A deep learning reconstruction framework for X-ray computed tomography with incomplete data Dong, Jianbing; Fu, Jian; He, Zhao PLOS ONE, Vol. 14, Issue 11 https://doi.org/10.1371/journal.pone.0224426	journal	November 2019
Seismic fault detection using an encoder–decoder convolutional neural network with a small training set Li, Shengrong; Yang, Changchun; Sun, Hui Journal of Geophysics and Engineering, Vol. 16, Issue 1 https://doi.org/10.1093/jge/gxy015	journal	February 2019
A Lightweight Solution of Industrial Computed Tomography with Convolutional Neural Network Zhu, Guogang; Fu, Jian arXiv https://doi.org/10.48550/arxiv.2004.06262	preprint	January 2020
Deep learning for cellular image analysis Moen, Erick; Bannon, Dylan; Kudo, Takamasa Nature Methods, Vol. 16, Issue 12 https://doi.org/10.1038/s41592-019-0403-1	journal	May 2019
Choices for long-term hypertensive control in patients after first-ever hemorrhagic stroke: a nationwide cohort study Liu, Chi-Hung; Lin, Yu-Sheng; Chi, Ching-Chi Therapeutic Advances in Neurological Disorders, Vol. 11 https://doi.org/10.1177/1756286418802688	journal	January 2018
Identification of Drug-Disease Associations Using Information of Molecular Structures and Clinical Symptoms via Deep Convolutional Neural Network Li, Zhanchao; Huang, Qixing; Chen, Xingyu Frontiers in Chemistry, Vol. 7 https://doi.org/10.3389/fchem.2019.00924	journal	January 2020
Combining Deep Learning and Active Contours Opens The Way to Robust, Automated Analysis of Brain Cytoarchitectonics Thierbach, Konstantin; Bazin, Pierre-Louis; Back, Walter de Machine Learning in Medical Imaging https://doi.org/10.1007/978-3-030-00919-9_21	book	January 2018
Machine learning and big scientific data Hey, Tony; Butler, Keith; Jackson, Sam Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 378, Issue 2166 https://doi.org/10.1098/rsta.2019.0054	journal	January 2020
A deep learning reconstruction framework for X-ray computed tomography with incomplete data Dong, Jianbing; Fu, Jian; He, Zhao PLOS ONE, Vol. 14, Issue 11 https://doi.org/10.1371/journal.pone.0224426	journal	November 2019
Solar image denoising with convolutional neural networks Díaz Baso, C. J.; de la Cruz Rodríguez, J.; Danilovic, S. Astronomy & Astrophysics, Vol. 629 https://doi.org/10.1051/0004-6361/201936069	journal	September 2019
Combining Deep Learning and Active Contours Opens The Way to Robust, Automated Analysis of Brain Cytoarchitectonics Thierbach, Konstantin; Bazin, Pierre-Louis; Back, Walter De https://doi.org/10.1101/297689	posted_content	August 2018
Interactive volumetric segmentation for textile micro‐tomography data using wavelets and nonlocal means MacNeil, J. Michael L.; Ushizima, Daniela M.; Panerai, Francesco Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 12, Issue 4 https://doi.org/10.1002/sam.11429	journal	May 2019
BS-Nets: An End-to-End Framework For Band Selection of Hyperspectral Image Cai, Yaoming; Liu, Xiaobo; Cai, Zhihua arXiv https://doi.org/10.48550/arxiv.1904.08269	text	January 2019
Deep learning for cellular image analysis Moen, Erick; Bannon, Dylan; Kudo, Takamasa Nature Methods, Vol. 16, Issue 12 https://doi.org/10.1038/s41592-019-0403-1	journal	May 2019
On-the-Fly Machine Learning for Improving Image Resolution in Tomography Hendriksen, Allard A.; Pelt, Daniël M.; Palenstijn, Willem Jan Applied Sciences, Vol. 9, Issue 12 https://doi.org/10.3390/app9122445	journal	June 2019
Towards Understanding Residual and Dilated Dense Neural Networks via Convolutional Sparse Coding Zhang, Zhiyang; Zhang, Shihua arXiv https://doi.org/10.48550/arxiv.1912.02605	text	January 2019
Single Remote Sensing Image Dehazing Using a Prior-Based Dense Attentive Network Gu, Ziqi; Zhan, Zongqian; Yuan, Qiangqiang Remote Sensing, Vol. 11, Issue 24 https://doi.org/10.3390/rs11243008	journal	December 2019
Stem cell motion-tracking by using deep neural networks with multi-output Wang, Yangxu; Mao, Hua; Yi, Zhang Neural Computing and Applications, Vol. 31, Issue 8 https://doi.org/10.1007/s00521-017-3291-2	journal	November 2017
Deep Learning Based Computed Tomography Whys and Wherefores Bazrafkan, Shabab; Van Nieuwenhove, Vincent; Soons, Joris arXiv https://doi.org/10.48550/arxiv.1904.03908	preprint	January 2019
Dense Dilated Convolutions Merging Network for Semantic Mapping of Remote Sensing Images liu, Qinghui; Kampffmeyer, Michael; Jenssen, Robert 2019 Joint Urban Remote Sensing Event (JURSE) https://doi.org/10.1109/jurse.2019.8809046	conference	May 2019
High-Resolution Boundary Refined Convolutional Neural Network for Automatic Agricultural Greenhouses Extraction from GaoFen-2 Satellite Imageries Zhang, Xiaoping; Cheng, Bo; Chen, Jinfen Remote Sensing, Vol. 13, Issue 21 https://doi.org/10.3390/rs13214237	journal	October 2021
Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters Zhu, Wei; Qiu, Qiang; Calderbank, Robert arXiv https://doi.org/10.48550/arxiv.1909.11193	preprint	January 2019
A Computationally Efficient Reconstruction Algorithm for Circular Cone-Beam Computed Tomography Using Shallow Neural Networks Lagerwerf, Marinus J.; Pelt, Daniël M.; Palenstijn, Willem Jan Journal of Imaging, Vol. 6, Issue 12 https://doi.org/10.3390/jimaging6120135	journal	December 2020
Cycloidal CT with CNN-based sinogram completion and in-scan generation of training data Pelt, Daniël M.; Roche i. Morgó, Oriol; Maughan Jones, Charlotte Scientific Reports, Vol. 12, Issue 1 https://doi.org/10.1038/s41598-022-04910-y	journal	January 2022
Using machine learning to predict extreme events in complex systems Qi, Di; Majda, Andrew J. Proceedings of the National Academy of Sciences, Vol. 117, Issue 1 https://doi.org/10.1073/pnas.1917285117	journal	December 2019
On-the-Fly Machine Learning for Improving Image Resolution in Tomography Hendriksen, Allard A.; Pelt, Daniël M.; Palenstijn, Willem Jan Applied Sciences, Vol. 9, Issue 12 https://doi.org/10.3390/app9122445	journal	June 2019
Segmentation of dental cone‐beam CT scans affected by metal artifacts using a mixed‐scale dense convolutional neural network Minnema, Jordi; Eijnatten, Maureen; Hendriksen, Allard A. Medical Physics, Vol. 46, Issue 11 https://doi.org/10.1002/mp.13793	journal	September 2019
Choices for long-term hypertensive control in patients after first-ever hemorrhagic stroke: a nationwide cohort study Liu, Chi-Hung; Lin, Yu-Sheng; Chi, Ching-Chi Therapeutic Advances in Neurological Disorders, Vol. 11 https://doi.org/10.1177/1756286418802688	journal	January 2018
To Recurse or not to Recurse,a Low Dose CT Study Bazrafkan, Shabab; Van Nieuwenhove, Vincent; Sijbers, Jan arXiv https://doi.org/10.48550/arxiv.1910.06565	preprint	January 2019
Automated segmentation of the knee for age assessment in 3D MR images using convolutional neural networks Pröve, Paul-Louis; Jopp-van Well, Eilin; Stanczus, Ben International Journal of Legal Medicine, Vol. 133, Issue 4 https://doi.org/10.1007/s00414-018-1953-y	journal	November 2018
Convolutional neural networks for grazing incidence x-ray scattering patterns: thin film structure identification Liu, Shuai; Melton, Charles N.; Venkatakrishnan, Singanallur MRS Communications, Vol. 9, Issue 02 https://doi.org/10.1557/mrc.2019.26	journal	March 2019
Using machine learning to predict extreme events in complex systems Qi, Di; Majda, Andrew J. Proceedings of the National Academy of Sciences, Vol. 117, Issue 1 https://doi.org/10.1073/pnas.1917285117	journal	December 2019
Deep Neural Network Assisted Iterative Reconstruction Method for Low Dose CT Bazrafkan, Shabab; Van Nieuwenhove, Vincent; Soons, Joris arXiv https://doi.org/10.48550/arxiv.1906.00650	preprint	January 2019