Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

HPC resources for CMS offline computing: An integration and scalability challenge for the Submission Infrastructure

Conference · · EPJ Web Conf.
 [1];  [2];  [3];  [4];  [4];  [4];  [3];  [5]
  1. Madrid, CIEMAT; PIC, Bellaterra
  2. UC, San Diego
  3. CERN
  4. Fermilab
  5. NCP, Islamabad
+ Show Author Affiliations
The computing resource needs of LHC experiments are expected to continue growing significantly during the Run 3 and into the HL-LHC era. The landscape of available resources will also evolve, as High Performance Computing (HPC) and Cloud resources will provide a comparable, or even dominant, fraction of the total compute capacity. The future years present a challenge for the experiments’ resource provisioning models, both in terms of scalability and increasing complexity. The CMS Submission Infrastructure (SI) provisions computing resources for CMS workflows. This infrastructure is built on a set of federated HTCondor pools, currently aggregating 400k CPU cores distributed worldwide and supporting the simultaneous execution of over 200k computing tasks. Incorporating HPC resources into CMS computing represents firstly an integration challenge, as HPC centers are much more diverse compared to Grid sites. Secondly, evolving the present SI, dimensioned to harness the current CMS computing capacity, to reach the resource scales required for the HLLHC phase, while maintaining global flexibility and efficiency, will represent an additional challenge for the SI. To preventively address future potential scalability limits, the SI team regularly runs tests to explore the maximum reach of our infrastructure. In this note, the integration of HPC resources into CMS offline computing is summarized, the potential concerns for the SI derived from the increased scale of operations are described, and the most recent results of scalability test on the CMS SI are reported.
Research Organization:
Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
Sponsoring Organization:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
Contributing Organization:
CMS
DOE Contract Number:
AC02-07CH11359
OSTI ID:
2467532
Report Number(s):
FERMILAB-CONF-24-0622-CMS; arXiv:2405.14631; oai:inspirehep.net:2784625
Conference Information:
Journal Name: EPJ Web Conf. Journal Volume: 295
Country of Publication:
United States
Language:
English

References (11)

Effective Dynamic Integration and Utilization of Heterogenous Compute Resources journal January 2020
HPC resource integration into CMS Computing via HEPCloud journal January 2019
HEPCloud, an Elastic Hybrid HEP Facility using an Intelligent Decision Support System journal January 2019
How much higher can HTCondor fly? journal December 2015
Extending the distributed computing infrastructure of the CMS experiment with HPC resources journal February 2023
Sharing server nodes for storage and computer journal January 2019
Using the glideinWMS System as a Common Resource Provisioning Layer in CMS journal December 2015
The CMS experiment at the CERN LHC journal August 2008
Computing for the Large Hadron Collider journal November 2011
Evolution of the CMS Global Submission Infrastructure for the HL-LHC Era journal January 2020
Reaching new peaks for the future of the CMS HTCondor Global Pool journal January 2021

Similar Records

Related Subjects