skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Evolution of CMS Workload Management Towards Multicore Job Support

Abstract

The successful exploitation of multicore processor architectures is a key element of the LHC distributed computing system in the coming era of the LHC Run 2. High-pileup complex-collision events represent a challenge for the traditional sequential programming in terms of memory and processing time budget. The CMS data production and processing framework is introducing the parallel execution of the reconstruction and simulation algorithms to overcome these limitations. CMS plans to execute multicore jobs while still supporting singlecore processing for other tasks difficult to parallelize, such as user analysis. The CMS strategy for job management thus aims at integrating single and multicore job scheduling across the Grid. This is accomplished by employing multicore pilots with internal dynamic partitioning of the allocated resources, capable of running payloads of various core counts simultaneously. An extensive test programme has been conducted to enable multicore scheduling with the various local batch systems available at CMS sites, with the focus on the Tier-0 and Tier-1s, responsible during 2015 of the prompt data reconstruction. Scale tests have been run to analyse the performance of this scheduling strategy and ensure an efficient use of the distributed resources. This paper presents the evolution of the CMS job management andmore » resource provisioning systems in order to support this hybrid scheduling model, as well as its deployment and performance tests, which will enable CMS to transition to a multicore production model for the second LHC run.« less

Authors:
 [1];  [1];  [2];  [3];  [4];  [4];  [3];  [4]
  1. Madrid, CIEMAT
  2. Quaid-i-Azam U.
  3. UC, San Diego
  4. Fermilab
Publication Date:
Research Org.:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP)
OSTI Identifier:
1247510
Report Number(s):
FERMILAB-CONF-15-607-CD
1413965
DOE Contract Number:  
AC02-07CH11359
Resource Type:
Conference
Journal Name:
J.Phys.Conf.Ser.
Additional Journal Information:
Journal Volume: 664; Journal Issue: 6; Conference: 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa, Japan, 04/13-04/17/2015
Country of Publication:
United States
Language:
English

Citation Formats

Perez-Calero Yzquierdo, A., Hernández, J. M., Khan, F. A., Letts, J., Majewski, K., Rodrigues, A. M., McCrea, A., and Vaandering, E. Evolution of CMS Workload Management Towards Multicore Job Support. United States: N. p., 2015. Web. doi:10.1088/1742-6596/664/6/062046.
Perez-Calero Yzquierdo, A., Hernández, J. M., Khan, F. A., Letts, J., Majewski, K., Rodrigues, A. M., McCrea, A., & Vaandering, E. Evolution of CMS Workload Management Towards Multicore Job Support. United States. https://doi.org/10.1088/1742-6596/664/6/062046
Perez-Calero Yzquierdo, A., Hernández, J. M., Khan, F. A., Letts, J., Majewski, K., Rodrigues, A. M., McCrea, A., and Vaandering, E. 2015. "Evolution of CMS Workload Management Towards Multicore Job Support". United States. https://doi.org/10.1088/1742-6596/664/6/062046. https://www.osti.gov/servlets/purl/1247510.
@article{osti_1247510,
title = {Evolution of CMS Workload Management Towards Multicore Job Support},
author = {Perez-Calero Yzquierdo, A. and Hernández, J. M. and Khan, F. A. and Letts, J. and Majewski, K. and Rodrigues, A. M. and McCrea, A. and Vaandering, E.},
abstractNote = {The successful exploitation of multicore processor architectures is a key element of the LHC distributed computing system in the coming era of the LHC Run 2. High-pileup complex-collision events represent a challenge for the traditional sequential programming in terms of memory and processing time budget. The CMS data production and processing framework is introducing the parallel execution of the reconstruction and simulation algorithms to overcome these limitations. CMS plans to execute multicore jobs while still supporting singlecore processing for other tasks difficult to parallelize, such as user analysis. The CMS strategy for job management thus aims at integrating single and multicore job scheduling across the Grid. This is accomplished by employing multicore pilots with internal dynamic partitioning of the allocated resources, capable of running payloads of various core counts simultaneously. An extensive test programme has been conducted to enable multicore scheduling with the various local batch systems available at CMS sites, with the focus on the Tier-0 and Tier-1s, responsible during 2015 of the prompt data reconstruction. Scale tests have been run to analyse the performance of this scheduling strategy and ensure an efficient use of the distributed resources. This paper presents the evolution of the CMS job management and resource provisioning systems in order to support this hybrid scheduling model, as well as its deployment and performance tests, which will enable CMS to transition to a multicore production model for the second LHC run.},
doi = {10.1088/1742-6596/664/6/062046},
url = {https://www.osti.gov/biblio/1247510}, journal = {J.Phys.Conf.Ser.},
number = 6,
volume = 664,
place = {United States},
year = {Wed Dec 23 00:00:00 EST 2015},
month = {Wed Dec 23 00:00:00 EST 2015}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: