Demonstrating Cross-Facility Data Processing at Scale with Laue Microdiffraction
In February and April 2023 live, at-scale data processing demonstrations were conducted between the Advanced Photon Source (APS), a synchrotron light source, and the Argonne Leadership Computing Facility (ALCF). These tests were run as part of a novel beamline technique: coded aperture laue micro-diffraction. This technique requires a significant amount of compute to decode appeture patterns embedded in the detector stream. An autonomous system was able to send data to ALCF during an experiment, utilize 50 nodes of the Polaris supercomputer to process 6-12 hour scans, and return the data back to the APS within 12-15 minutes behind the detector. With scan points arriving every 72 seconds, the system kept up with the beamline, potentially enabling in-experiment analysis. The data processing system utilizes Globus infrastructure and an on-demand queue to dynamically acquire nodes on Polaris. The underlying reconstruction algorithms were parallelized via MPI and accelerated with custom CUDA kernels.
- Research Organization:
- Argonne National Laboratory (ANL), Argonne, IL (United States)
- Sponsoring Organization:
- USDOE Office of Science - Office of Basic Energy Sciences - Scientific User Facilities Division; USDOE Office of Science - Office of Advanced Scientific Computing Research (ASCR)
- DOE Contract Number:
- AC02-06CH11357
- OSTI ID:
- 2246619
- Resource Relation:
- Conference: 5th Annual Workshop on Large-scale Experiment-in-the-Loop Computing in conjunction with SC23 The International Conference for High Performance Computing, Networking, Storage and Analysis - (Denver, CO, US, 11/12/23-11/17/23), 11/12/23 - 11/12/23, Denver, CO, US
- Country of Publication:
- United States
- Language:
- English
Similar Records
Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science
Empowering Scientific Discovery Through Computing at the Advanced Photon Source