skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Multi-core processing and scheduling performance in CMS

Abstract

Commodity hardware is going many-core. We might soon not be able to satisfy the job memory needs per core in the current single-core processing model in High Energy Physics. In addition, an ever increasing number of independent and incoherent jobs running on the same physical hardware not sharing resources might significantly affect processing performance. It will be essential to effectively utilize the multi-core architecture. CMS has incorporated support for multi-core processing in the event processing framework and the workload management system. Multi-core processing jobs share common data in memory, such us the code libraries, detector geometry and conditions data, resulting in a much lower memory usage than standard single-core independent jobs. Exploiting this new processing model requires a new model in computing resource allocation, departing from the standard single-core allocation for a job. The experiment job management system needs to have control over a larger quantum of resource since multi-core aware jobs require the scheduling of multiples cores simultaneously. CMS is exploring the approach of using whole nodes as unit in the workload management system where all cores of a node are allocated to a multi-core job. Whole-node scheduling allows for optimization of the data/workflow management (e.g. I/O caching, localmore » merging) but efficient utilization of all scheduled cores is challenging. Dedicated whole-node queues have been setup at all Tier-1 centers for exploring multi-core processing workflows in CMS. We present the evaluation of the performance scheduling and executing multi-core workflows in whole-node queues compared to the standard single-core processing workflows.« less

Authors:
 [1];  [2];  [2]
  1. Madrid, CIEMAT
  2. Fermilab
Publication Date:
Research Org.:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP)
OSTI Identifier:
1405107
Report Number(s):
FERMILAB-CONF-12-822-PPD
1211532
DOE Contract Number:  
AC02-07CH11359
Resource Type:
Conference
Journal Name:
J.Phys.Conf.Ser.
Additional Journal Information:
Journal Volume: 396; Conference: 19th International Conference on Computing in High Energy and Nuclear Physics, New York, USA, 05/21-05/25/2012
Country of Publication:
United States
Language:
English

Citation Formats

Hernandez, J. M., Evans, D., and Foulkes, S. Multi-core processing and scheduling performance in CMS. United States: N. p., 2012. Web. doi:10.1088/1742-6596/396/3/032055.
Hernandez, J. M., Evans, D., & Foulkes, S. Multi-core processing and scheduling performance in CMS. United States. https://doi.org/10.1088/1742-6596/396/3/032055
Hernandez, J. M., Evans, D., and Foulkes, S. 2012. "Multi-core processing and scheduling performance in CMS". United States. https://doi.org/10.1088/1742-6596/396/3/032055. https://www.osti.gov/servlets/purl/1405107.
@article{osti_1405107,
title = {Multi-core processing and scheduling performance in CMS},
author = {Hernandez, J. M. and Evans, D. and Foulkes, S.},
abstractNote = {Commodity hardware is going many-core. We might soon not be able to satisfy the job memory needs per core in the current single-core processing model in High Energy Physics. In addition, an ever increasing number of independent and incoherent jobs running on the same physical hardware not sharing resources might significantly affect processing performance. It will be essential to effectively utilize the multi-core architecture. CMS has incorporated support for multi-core processing in the event processing framework and the workload management system. Multi-core processing jobs share common data in memory, such us the code libraries, detector geometry and conditions data, resulting in a much lower memory usage than standard single-core independent jobs. Exploiting this new processing model requires a new model in computing resource allocation, departing from the standard single-core allocation for a job. The experiment job management system needs to have control over a larger quantum of resource since multi-core aware jobs require the scheduling of multiples cores simultaneously. CMS is exploring the approach of using whole nodes as unit in the workload management system where all cores of a node are allocated to a multi-core job. Whole-node scheduling allows for optimization of the data/workflow management (e.g. I/O caching, local merging) but efficient utilization of all scheduled cores is challenging. Dedicated whole-node queues have been setup at all Tier-1 centers for exploring multi-core processing workflows in CMS. We present the evaluation of the performance scheduling and executing multi-core workflows in whole-node queues compared to the standard single-core processing workflows.},
doi = {10.1088/1742-6596/396/3/032055},
url = {https://www.osti.gov/biblio/1405107}, journal = {J.Phys.Conf.Ser.},
number = ,
volume = 396,
place = {United States},
year = {Sun Jan 01 00:00:00 EST 2012},
month = {Sun Jan 01 00:00:00 EST 2012}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: