Accelerating Advanced Light Source Science Through Multi-Facility HPC Workflows

Conference · Sun Nov 16 00:00:00 EST 2025

DOI:https://doi.org/10.1145/3731599.3767590· OSTI ID:3005978

Abramov, David; Welborn, Samuel; Chard, Ryan; Chawla, Kuldeep; Chong, Xiaoya; Clark, Elizabeth; Enders, Bjoern; Hexemer, Alexander; Jed, Jason; Koepp, Wiebke; Krishnan, Harinarayan; De Leon, Seij; Parkinson, Dilworth; Perlmutter, David; Sriramoju, Raja Vyshnavi; Uram, Thomas; Yang, Lee Lisheng; McReynolds, Dylan

Synchrotron light sources support a wide array of techniques to investigate materials, often producing complex, high-volume data that challenge traditional workflows. At the Advanced Light Source (ALS), we developed infrastructure to move microtomography data over ESnet to ALCF and NERSC, where CPU- and GPU-based algorithms generate 3D reconstructed volumes of experimental samples. We employ two data movement and reconstruction models: real-time processing as data streams directly to NERSC compute nodes, and automated file transfer to NERSC and ALCF file systems. The streaming pipeline provides users with feedback in under ten seconds, while the file-based workflow produces high-quality reconstructions suitable for deeper analysis in 20-30 minutes. This infrastructure enables users to utilize HPC resources without direct access to backend systems. We plan to extend this architecture to more endstations, supporting our beamline scientists and users.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: US Department of Energy; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-ASCR); USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22), Scientific User Facilities Division (SC-22.3 )
• DOE Contract Number: AC02-05CH11231
• OSTI ID: 3005978
• Resource Type: Conference paper
• Conference Information: Proceedings of 2025 Workshops of the International Conference on High Performance Computing Network Storage and Analysis Sc 2025 Workshops
• Country of Publication: United States
• Language: English