Utilizing Distributed Heterogeneous Computing with PanDA in ATLAS

Journal Article · Mon May 06 00:00:00 EDT 2024 · EPJ Web of Conferences (Online)

DOI:https://doi.org/10.1051/epjconf/202429504053· OSTI ID:2448346

Maeno, Tadashi ^[1]; Alekseev, Aleksandr ^[2]; Barreiro Megino, Fernando Harald ^[2]; De, Kaushik ^[2]; Guan, Wen ^[1]; Karavakis, Edward ^[1]; Klimentov, Alexei ^[1]; Korchuganova, Tatiana ^[3]; Lin, FaHui ^[2]; Nilsson, Paul ^[1]; Wenaus, Torre ^[1]; Yang, Zhaoyu ^[1]; Zhao, Xin ^[1]

---

1. Brookhaven National Laboratory (BNL), Upton, NY (United States)
2. Univ. of Texas, Arlington, TX (United States)
3. Univ. of Pittsburgh, PA (United States)

In recent years, advanced and complex analysis workflows have gained increasing importance in the ATLAS experiment at CERN, one of the large scientific experiments at LHC. Support for such workflows has allowed users to exploit remote computing resources and service providers distributed worldwide, overcoming limitations on local resources and services. The spectrum of computing options keeps increasing across the Worldwide LHC Computing Grid (WLCG), volunteer computing, high-performance computing, commercial clouds, and emerging service levels like Platform-as-a-Service (PaaS), Container-as-a-Service (CaaS) and Function-as-a-Service (FaaS), each one providing new advantages and constraints. Users can significantly benefit from these providers, but at the same time, it is cumbersome to deal with multiple providers, even in a single analysis workflow with fine-grained requirements coming from their applications’ nature and characteristics. In this paper, we will first highlight issues in geographically-distributed heterogeneous computing, such as the insulation of users from the complexities of dealing with remote providers, smart workload routing, complex resource provisioning, seamless execution of advanced workflows, workflow description, pseudointeractive analysis, and integration of PaaS, CaaS, and FaaS providers. We will also outline solutions developed in ATLAS with the Production and Distributed Analysis (PanDA) system and future challenges for LHC Run4.

• Research Organization: Brookhaven National Laboratory (BNL), Upton, NY (United States)
• Sponsoring Organization: USDOE Office of Science (SC), High Energy Physics (HEP)
• Grant/Contract Number: SC0012704
• OSTI ID: 2448346
• Report Number(s): BNL--226155-2024-JAAM
• Journal Information: EPJ Web of Conferences (Online), Journal Name: EPJ Web of Conferences (Online) Vol. 295; ISSN 2100-014X
• Publisher: EDP Sciences
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING