Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Improving the Scheduling Efficiency of a Global Multi-Core HTCondor Pool in CMS

Conference · · EPJ Web Conf.
 [1];  [2];  [3];  [4];  [5];  [6];  [5];  [7];  [7];  [5];  [8]
  1. Nebraska U.
  2. Puebla U., Mexico
  3. Notre Dame U.
  4. Sofiya U.
  5. Fermilab
  6. Malaya U.
  7. UC, San Diego
  8. Madrid, CIEMAT; PIC, Bellaterra
+ Show Author Affiliations
Scheduling multi-core workflows in a global HTCondor pool is a multi-dimensional problem whose solution depends on the requirements of the job payloads, the characteristics of available resources, and the boundary conditions such as fair share and prioritization imposed on the job matching to resources. Within the context of a dedicated task force, CMS has increased significantly the scheduling efficiency of workflows in reusable multi-core pilots by various improvements to the limitations of the GlideinWMS pilots, accuracy of resource requests, efficiency and speed of the HTCondor infrastructure, and job matching algorithms.
Research Organization:
Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
Sponsoring Organization:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
DOE Contract Number:
AC02-07CH11359;
OSTI ID:
1854894
Report Number(s):
FERMILAB-CONF-19-846-SCD; oai:inspirehep.net:1760953
Resource Type:
Conference paper
Conference Information:
Journal Name: EPJ Web Conf. Journal Volume: 214
Country of Publication:
United States
Language:
English

References (4)

Distributed computing in practice: the Condor experience
  • Thain, Douglas; Tannenbaum, Todd; Livny, Miron
  • Concurrency and Computation: Practice and Experience, Vol. 17, Issue 2-4, p. 323-356 https://doi.org/10.1002/cpe.938
journal January 2005
HEPCloud, a New Paradigm for HEP Facilities: CMS Amazon Web Services Investigation journal September 2017
Connecting Restricted, High-Availability, or Low-Latency Resources to a Seamless Global Pool for CMS journal October 2017
The Pilot Way to Grid Resources Using glideinWMS conference March 2009

Similar Records

Multi-core processing and scheduling performance in CMS
Conference · 2011 · J.Phys.Conf.Ser. · OSTI ID:1405107
CMS readiness for multi-core workload scheduling
Journal Article · 2017 · Journal of Physics. Conference Series · OSTI ID:1420916
Evolution of the pilot infrastructure of CMS: towards a single glideinWMS pool
Conference · 2013 · J.Phys.Conf.Ser. · OSTI ID:1295710

Related Subjects