Deployment and Evaluation of SciStream on OLCF's Advanced Computing Ecosystem (ACE)
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
The growing demand for real-time analysis, experimental steering, and decision-making in scientific workflows has created a need for tightly coupled integrations between experimental facilities and high-performance computing (HPC) systems. The Department of Energy’s Integrated Research Infrastructure (IRI) initiative highlights data streaming as a key capability for enabling memory-to-memory data transfers, bypassing the limitations of traditional store-and-forward models. SciStream is a toolkit developed by researchers at Argonne National Laboratory (ANL) to support such streaming by addressing cross-domain security, delegated authentication, and application transparency. We deployed and evaluated SciStream on the Oak Ridge Leadership Computing Facility’s (OLCF) Advanced Computing Ecosystem (ACE) infrastructure, leveraging the Olivine OpenShift cluster and its high-bandwidth Data Streaming Nodes (DSNs) as gateway nodes. Our evaluation included synthetic streaming workloads derived from IRI science workflows, a streaming simulator, and integration with RabbitMQ to handle low-level messaging. This report documents the deployment process, performance evaluation, and challenges encountered, along with opportunities for future improvements.
- Research Organization:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
- Sponsoring Organization:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- DOE Contract Number:
- AC05-00OR22725
- OSTI ID:
- 3006496
- Report Number(s):
- ORNL/TM--2025/4190
- Country of Publication:
- United States
- Language:
- English
Similar Records
OLCF’s Advanced Computing Ecosystem (ACE): FY25 Update for Ongoing Efforts
OLCF’s Advanced Computing Ecosystem (ACE): FY24 Efforts for the DOE Integrated Research Infrastructure (IRI) Program
Empowering Scientific Innovation Through An Integrated Research Infrastructure: The Role of the Advanced Computing Ecosystem
Technical Report
·
Sun Nov 30 19:00:00 EST 2025
·
OSTI ID:3006499
OLCF’s Advanced Computing Ecosystem (ACE): FY24 Efforts for the DOE Integrated Research Infrastructure (IRI) Program
Technical Report
·
Fri Nov 01 00:00:00 EDT 2024
·
OSTI ID:2477506
Empowering Scientific Innovation Through An Integrated Research Infrastructure: The Role of the Advanced Computing Ecosystem
Conference
·
Thu Oct 31 20:00:00 EDT 2024
·
OSTI ID:2538306