Scalability and interoperability within glideinWMS

Conference · Fri Jan 01 04:00:00 EST 2010 · J.Phys.Conf.Ser.219:062036,2010

DOI:https://doi.org/10.1088/1742-6596/219/6/062036· OSTI ID:986995

Bradley, D; Sfiligoi, I; Padhi, S; Frey, J; Tannenbaum, T

Physicists have access to thousands of CPUs in grid federations such as OSG and EGEE. With the start-up of the LHC, it is essential for individuals or groups of users to wrap together available resources from multiple sites across multiple grids under a higher user-controlled layer in order to provide a homogeneous pool of available resources. One such system is glideinWMS, which is based on the Condor batch system. A general discussion of glideinWMS can be found elsewhere. Here, we focus on recent advances in extending its reach: scalability and integration of heterogeneous compute elements. We demonstrate that the new developments exceed the design goal of over 10,000 simultaneous running jobs under a single Condor schedd, using strong security protocols across global networks, and sustaining a steady-state job completion rate of a few Hz. We also show interoperability across heterogeneous computing elements achieved using client-side methods. We discuss this technique and the challenges in direct access to NorduGrid and CREAM compute elements, in addition to Globus based systems.

• Research Organization: Fermi National Accelerator Laboratory (FNAL), Batavia, IL
• Sponsoring Organization: USDOE
• DOE Contract Number: AC02-07CH11359
• OSTI ID: 986995
• Report Number(s): FERMILAB-CONF-10-258-CD
• Conference Information: Journal Name: J.Phys.Conf.Ser.219:062036,2010
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
Computing
DESIGN
NUCLEAR PHYSICS
SECURITY
START-UP