skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Exploring GlideinWMS and HTCondor scalability frontiers for an expanding CMS Global Pool

Abstract

The CMS Submission Infrastructure Global Pool, built on Glidein-WMS andHTCondor, is a worldwide distributed dynamic pool responsible for the allocation of resources for all CMS computing workloads. Matching the continuously increasing demand for computing resources by CMS requires the anticipated assessment of its scalability limitations. In addition, the Global Plmust be able to expand in a more heterogeneous environment, in terms of resource provisioning (combining Grid, HPC and Cloud) and workload submissi.A dedicated testbed has been set up to simulate such conditions with the purpose of finding potential bottlenecks in the software or its configuration. This report provides a thorough description of the various scalabilitydimensions in size and complexity that are being explored for the future Global Pool, along with the analysis and solutions to the limitations proposed with the support of the GlideinWMS and HTCondor developer teams.

Authors:
ORCiD logo [1]; ORCiD logo [2];  [3]; ORCiD logo [4];  [5];  [6]; ORCiD logo [7];  [6]; ORCiD logo [8];  [8];  [6]
  1. Research Centre for Energy, Environment and Technology (CIEMAT), Madrid (Spain); Spain Port d’Informació Cientifica (PIC), Barcelona (Spain)
  2. Univ. of Nebraska, Lincoln, NE (United States)
  3. Benemérita Univ. Autónoma de Puebla, Puebla (México)
  4. Univ. of Notre Dame, IN (United States)
  5. Univ. of Sofia, Sofia (Bulgaria)
  6. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
  7. Univ. of Malaya, Kuala Lumpur (Malaysia)
  8. Univ. of California, San Diego, CA (United States)
Publication Date:
Research Org.:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP)
Contributing Org.:
CMS Collaboration
OSTI Identifier:
1574964
Report Number(s):
FERMILAB-CONF-19-553-SCD
Journal ID: ISSN 2100-014X; oai:inspirehep.net:1760907; TRN: US2001152
Grant/Contract Number:  
AC02-07CH11359
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
EPJ Web of Conferences
Additional Journal Information:
Journal Volume: 214; Conference: 23. International Conference on Computing in High Energy and Nuclear Physics, Sofia (Bulgaria), 9-13 Jul 2018; Journal ID: ISSN 2100-014X
Publisher:
EDP Sciences
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Pérez-Calero Yzquierdo, Antonio, Bockelman, Brian Paul, Davila Foyo, Diego, Hurtado Anampa, Kenyi, Ivanov, Todor Trendafilov, Khan, Farrukh Aftab, Kotobi, Amjad, Larson, Krista, Letts, James, Mascheroni, Marco, and Mason, David. Exploring GlideinWMS and HTCondor scalability frontiers for an expanding CMS Global Pool. United States: N. p., 2019. Web. doi:10.1051/epjconf/201921403002.
Pérez-Calero Yzquierdo, Antonio, Bockelman, Brian Paul, Davila Foyo, Diego, Hurtado Anampa, Kenyi, Ivanov, Todor Trendafilov, Khan, Farrukh Aftab, Kotobi, Amjad, Larson, Krista, Letts, James, Mascheroni, Marco, & Mason, David. Exploring GlideinWMS and HTCondor scalability frontiers for an expanding CMS Global Pool. United States. https://doi.org/10.1051/epjconf/201921403002
Pérez-Calero Yzquierdo, Antonio, Bockelman, Brian Paul, Davila Foyo, Diego, Hurtado Anampa, Kenyi, Ivanov, Todor Trendafilov, Khan, Farrukh Aftab, Kotobi, Amjad, Larson, Krista, Letts, James, Mascheroni, Marco, and Mason, David. 2019. "Exploring GlideinWMS and HTCondor scalability frontiers for an expanding CMS Global Pool". United States. https://doi.org/10.1051/epjconf/201921403002. https://www.osti.gov/servlets/purl/1574964.
@article{osti_1574964,
title = {Exploring GlideinWMS and HTCondor scalability frontiers for an expanding CMS Global Pool},
author = {Pérez-Calero Yzquierdo, Antonio and Bockelman, Brian Paul and Davila Foyo, Diego and Hurtado Anampa, Kenyi and Ivanov, Todor Trendafilov and Khan, Farrukh Aftab and Kotobi, Amjad and Larson, Krista and Letts, James and Mascheroni, Marco and Mason, David},
abstractNote = {The CMS Submission Infrastructure Global Pool, built on Glidein-WMS andHTCondor, is a worldwide distributed dynamic pool responsible for the allocation of resources for all CMS computing workloads. Matching the continuously increasing demand for computing resources by CMS requires the anticipated assessment of its scalability limitations. In addition, the Global Plmust be able to expand in a more heterogeneous environment, in terms of resource provisioning (combining Grid, HPC and Cloud) and workload submissi.A dedicated testbed has been set up to simulate such conditions with the purpose of finding potential bottlenecks in the software or its configuration. This report provides a thorough description of the various scalabilitydimensions in size and complexity that are being explored for the future Global Pool, along with the analysis and solutions to the limitations proposed with the support of the GlideinWMS and HTCondor developer teams.},
doi = {10.1051/epjconf/201921403002},
url = {https://www.osti.gov/biblio/1574964}, journal = {EPJ Web of Conferences},
issn = {2100-014X},
number = ,
volume = 214,
place = {United States},
year = {2019},
month = {9}
}