Scalable and accurate multi-GPU-based image reconstruction of large-scale ptychography data

Journal Article · Tue Mar 29 00:00:00 EDT 2022 · Scientific Reports

DOI:https://doi.org/10.1038/s41598-022-09430-3· OSTI ID:1860447

Yu, Xiaodong; Nikitin, Viktor; Ching, Daniel J.; Aslan, Selin; Gürsoy, Doğa; Biçer, Tekin

Abstract While the advances in synchrotron light sources, together with the development of focusing optics and detectors, allow nanoscale ptychographic imaging of materials and biological specimens, the corresponding experiments can yield terabyte-scale volumes of data that can impose a heavy burden on the computing platform. Although graphics processing units (GPUs) provide high performance for such large-scale ptychography datasets, a single GPU is typically insufficient for analysis and reconstruction. Several works have considered leveraging multiple GPUs to accelerate the ptychographic reconstruction. However, most of these works utilize only the Message Passing Interface to handle the communications between GPUs. This approach poses inefficiency for a hardware configuration that has multiple GPUs in a single node, especially while reconstructing a single large projection, since it provides no optimizations to handle the heterogeneous GPU interconnections containing both low-speed (e.g., PCIe) and high-speed links (e.g., NVLink). In this paper, we provide an optimized intranode multi-GPU implementation that can efficiently solve large-scale ptychographic reconstruction problems. We focus on the maximum likelihood reconstruction problem using a conjugate gradient (CG) method for the solution and propose a novel hybrid parallelization model to address the performance bottlenecks in the CG solver. Accordingly, we have developed a tool, called PtyGer ( Pty chographic G PU(multipl e )-based r econstruction), implementing our hybrid parallelization model design. A comprehensive evaluation verifies that PtyGer can fully preserve the original algorithm’s accuracy while achieving outstanding intranode GPU scalability.

• Research Organization: Argonne National Laboratory (ANL), Argonne, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Basic Energy Sciences (BES); USDOE National Nuclear Security Administration (NNSA); US Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA)
• Grant/Contract Number: AC02-06CH11357; 89233218CNA000001; D2019-1903270004
• OSTI ID: 1860447
• Alternate ID(s): OSTI ID: 1901717
• Journal Information: Scientific Reports, Journal Name: Scientific Reports Vol. 12 Journal Issue: 1; ISSN 2045-2322
• Publisher: Nature Publishing Group
• Country of Publication: United Kingdom
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING