Concurrent Conditions Access across Validity Intervals in CMSSW
- Fermilab
The CMS software system, known as CMSSW, has a generalized conditions, calibration, and geometry data products system called the EventSetup. The EventSetup caches results of reading or calculating data products based on the 'interval of validity', IOV, which is based on the time period for which that data product is appropriate. With the original single threaded CMSSW framework, updating only on an IOV boundary meant we only required memory for a single data product of a given type at any time during the program execution. In 2016 CMS transitioned to using a multithreaded framework as a way to save on memory during processing. This was accomplished by amortizing the memory cost of EventSetup data products across multiple concurrent events. To initially accomplish that goal required synchronizing event processing across IOV boundaries, thereby decreasing the scalability of the system. In this presentation we will explain how we used 'limited concurrent task queues' to allow co ncurrent IOVs while still being able to limit the memory utilized. In addition, we will present performance measurements for both threading scalability of event throughtput and memory utilization.
- Research Organization:
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), High Energy Physics (HEP)
- DOE Contract Number:
- AC02-07CH11359
- OSTI ID:
- 1574835
- Report Number(s):
- FERMILAB-POSTER-19-129-CMS-SCD; oai:inspirehep.net:1764649
- Country of Publication:
- United States
- Language:
- English
Similar Records
CMSSW Scaling Limits on Many-Core Machines
Implementing Concurrent Non-Event Transitionsin CMS