Accelerating Science Impact through Big Data Workflow Management and Supercomputing

Journal Article · Tue Feb 09 00:00:00 EST 2016 · EPJ Web of Conferences

DOI:https://doi.org/10.1051/epjconf/201610801003· OSTI ID:1567415

De, K. ^[1]; Klimentov, A. ^[2]; Maeno, T. ^[3]; Mashinistov, R. ^[4]; Nilsson, P. ^[3]; Oleynik, D. ^[5]; Panitkin, S. ^[3]; Ryabinkin, E. ^[4]; Wenaus, T. ^[3]

---

1. Univ. of Texas, Arlington, TX (United States). Dept. of Physics
2. Brookhaven National Lab. (BNL), Upton, NY (United States). Dept. of Physics; National Research Centre Kurchatov Inst., Moscow (Russia)
3. Brookhaven National Lab. (BNL), Upton, NY (United States). Dept. of Physics
4. National Research Centre Kurchatov Inst., Moscow (Russia)
5. Univ. of Texas, Arlington, TX (United States). Dept. of Physics; Joint Inst. for Nuclear Research (JINR), Dubna (Russia). Lab. of Information Technologies

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. ATLAS, one of the largest collaborations ever assembled in the the history of science, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. To manage the workflow for all data processing on hundreds of data centers the PanDA (Production and Distributed Analysis)Workload Management System is used. An ambitious program to expand PanDA to all available computing resources, including opportunistic use of commercial and academic clouds and Leadership Computing Facilities (LCF), is realizing within BigPanDA and megaPanDA projects. These projects are now exploring how PanDA might be used for managing computing jobs that run on supercomputers including OLCF’s Titan and NRC-KI HPC2. The main idea is to reuse, as much as possible, existing components of the PanDA system that are already deployed on the LHC Grid for analysis of physics data. The next generation of PanDA will allow many data-intensive sciences employing a variety of computing platforms to benefit from ATLAS experience and proven tools in highly scalable processing.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
• Sponsoring Organization: Russian Ministry of Science and Education; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
• Grant/Contract Number: AC02-06CH11357; AC02-98CH10886
• OSTI ID: 1567415
• Journal Information: EPJ Web of Conferences, Journal Name: EPJ Web of Conferences Vol. 108; ISSN 2100-014X
• Publisher: EDP Sciences
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
Physics