Concurrent Conditions Access across Validity Intervals in CMSSW
Abstract
The CMS software system, known as CMSSW, has a generalized conditions, calibration, and geometry data products system called the EventSetup. The EventSetup caches results of reading or calculating data products based on the 'interval of validity', IOV, which is based on the time period for which that data product is appropriate. With the original single threaded CMSSW framework, updating only on an IOV boundary meant we only required memory for a single data product of a given type at any time during the program execution. In 2016 CMS transitioned to using a multithreaded framework as a way to save on memory during processing. This was accomplished by amortizing the memory cost of EventSetup data products across multiple concurrent events. To initially accomplish that goal required synchronizing event processing across IOV boundaries, thereby decreasing the scalability of the system. In this presentation we will explain how we used 'limited concurrent task queues' to allow co ncurrent IOVs while still being able to limit the memory utilized. In addition, we will present performance measurements for both threading scalability of event throughtput and memory utilization.
- Authors:
-
- Fermilab
- Publication Date:
- Research Org.:
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
- OSTI Identifier:
- 1574835
- Report Number(s):
- FERMILAB-POSTER-19-129-CMS-SCD
oai:inspirehep.net:1764649
- DOE Contract Number:
- AC02-07CH11359
- Resource Type:
- Conference
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Dagenhart, W.David, and Jones, Christopher D. Concurrent Conditions Access across Validity Intervals in CMSSW. United States: N. p., 2019.
Web.
Dagenhart, W.David, & Jones, Christopher D. Concurrent Conditions Access across Validity Intervals in CMSSW. United States.
Dagenhart, W.David, and Jones, Christopher D. Tue .
"Concurrent Conditions Access across Validity Intervals in CMSSW". United States. https://www.osti.gov/servlets/purl/1574835.
@article{osti_1574835,
title = {Concurrent Conditions Access across Validity Intervals in CMSSW},
author = {Dagenhart, W.David and Jones, Christopher D.},
abstractNote = {The CMS software system, known as CMSSW, has a generalized conditions, calibration, and geometry data products system called the EventSetup. The EventSetup caches results of reading or calculating data products based on the 'interval of validity', IOV, which is based on the time period for which that data product is appropriate. With the original single threaded CMSSW framework, updating only on an IOV boundary meant we only required memory for a single data product of a given type at any time during the program execution. In 2016 CMS transitioned to using a multithreaded framework as a way to save on memory during processing. This was accomplished by amortizing the memory cost of EventSetup data products across multiple concurrent events. To initially accomplish that goal required synchronizing event processing across IOV boundaries, thereby decreasing the scalability of the system. In this presentation we will explain how we used 'limited concurrent task queues' to allow co ncurrent IOVs while still being able to limit the memory utilized. In addition, we will present performance measurements for both threading scalability of event throughtput and memory utilization.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2019},
month = {1}
}