Coscheduling in Clusters: Is It a Viable Alternative?
As clusters are widely accepted as cost-effective infrastructures for many scientific and commercial applications, improving the deliverable performance and reducing the energy consumption of such systems has become a pressing issue. In this paper, we exploit the feasibility of achieving these objectives through efficiently scheduling the communicating processes of parallel applications. In this context, we conduct an in-depth evaluation of a broad spectrum of scheduling alternatives for clusters. These include the widely used batch scheduling, local scheduling, gang scheduling, all prior communication-driven coscheduling algorithms, and a newly proposed HYBRID coscheduling algorithm. In order to provide ease of implementation and portability across many cluster platforms, we propose a generic framework for deploying any coscheduling algorithm. We have implemented four prior coscheduling algorithms (Dynamic Coscheduling (DCS), Spin Block (SB), Periodic Boost (PB), and Co-ordinated Coscheduling (CC)) and the HYBRID coscheduling using this framework on a 16-node, Myrinet connected Linux cluster that uses GM as the communication layer. In addition, we use PBS as the batch scheduler and a previously proposed gang scheduler (SCore) to analyze all classes of scheduling techniques. Performance and energy measurements using several NAS and LLNL benchmarks on the Linux cluster provide several interesting conclusions. First, although batch scheduling is currently used in most clusters, all blocking-based coscheduling techniques such as SB, CC and HYBRID and the gang scheduling can provide much better performance even in a dedicated cluster platform. Under high system load, these coscheduling schemes can provide orders of magnitude reduction in average response time and much better performance-energy behavior compared to the PBS scheme. Second, in contrast to some of the prior studies, we observe that blocking-based schemes like SB and HYBRID can provide better performance than spin-based techniques like PB on a Linux platform. Third, the proposed HYBRID scheduling provides the best performance-energy behavior and can be implemented on any cluster with little effort. All these results suggest that blocking-based coscheduling techniques are viable candidates to be used instead of batching scheme for significant performance-energy benefits.
- Research Organization:
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- W-7405-ENG-48
- OSTI ID:
- 15013790
- Report Number(s):
- UCRL-CONF-200869; TRN: US200803%%846
- Resource Relation:
- Conference: Presented at: ACM SIGMETRICS - Performance 2004, New York, NY, United States, Jun 12 - Jun 16, 2004
- Country of Publication:
- United States
- Language:
- English
Similar Records
Adaptive Parallel Job Scheduling with Flexible CoScheduling
Coscheduling Technique for Symmetric Multiprocessor Clusters