skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Toward Large-Scale Image Segmentation on Summit

Abstract

Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (∼ 102 × 102), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (∼ 104 × 104). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on 256 nodes (1536 GPUs) of the Summit supercomputer. Using a single node of the Summit supercomputer, an early evaluation of a recently released model parallel framework called GPipe is demonstrated to deliver ∼ 2X speedup in executing a U-Net model with an order of magnitude larger number of trainable parameters than reported before. Performance bottlenecks for pipelined training of U-Net models are identified andmore » mitigation strategies to improve the speedups are discussed. Together, these results open up the possibility of combining both approaches into a unified scalable pipelined and data parallel algorithm to efficiently train U-Net models with very large receptive fields on data sets of ultra-large extent images.« less


Citation Formats

Seal, Sudip, Lim, Seung-Hwan, Wang, Dali, Hinkle, Jacob, Lunga, Dalton, and Tsaris, Aristeidis. Toward Large-Scale Image Segmentation on Summit. United States: N. p., 2020. Web.
Seal, Sudip, Lim, Seung-Hwan, Wang, Dali, Hinkle, Jacob, Lunga, Dalton, & Tsaris, Aristeidis. Toward Large-Scale Image Segmentation on Summit. United States.
Seal, Sudip, Lim, Seung-Hwan, Wang, Dali, Hinkle, Jacob, Lunga, Dalton, and Tsaris, Aristeidis. Sat . "Toward Large-Scale Image Segmentation on Summit". United States. https://www.osti.gov/servlets/purl/1665994.
@article{osti_1665994,
title = {Toward Large-Scale Image Segmentation on Summit},
author = {Seal, Sudip and Lim, Seung-Hwan and Wang, Dali and Hinkle, Jacob and Lunga, Dalton and Tsaris, Aristeidis},
abstractNote = {Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (∼ 102 × 102), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (∼ 104 × 104). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on 256 nodes (1536 GPUs) of the Summit supercomputer. Using a single node of the Summit supercomputer, an early evaluation of a recently released model parallel framework called GPipe is demonstrated to deliver ∼ 2X speedup in executing a U-Net model with an order of magnitude larger number of trainable parameters than reported before. Performance bottlenecks for pipelined training of U-Net models are identified and mitigation strategies to improve the speedups are discussed. Together, these results open up the possibility of combining both approaches into a unified scalable pipelined and data parallel algorithm to efficiently train U-Net models with very large receptive fields on data sets of ultra-large extent images.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2020},
month = {8}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: