Advance Resource Provisioning in Bulk Data Scheduling
Today?s scientific and business applications generate mas- sive data sets that need to be transferred to remote sites for sharing, processing, and long term storage. Because of increasing data volumes and enhancement in current net- work technology that provide on-demand high-speed data access between collaborating institutions, data handling and scheduling problems have reached a new scale. In this paper, we present a new data scheduling model with ad- vance resource provisioning, in which data movement operations are defined with earliest start and latest comple- tion times. We analyze time-dependent resource assign- ment problem, and propose a new methodology to improve the current systems by allowing researchers and higher-level meta-schedulers to use data-placement as-a-service, so they can plan ahead and submit transfer requests in advance. In general, scheduling with time and resource conflicts is NP-hard. We introduce an efficient algorithm to organize multiple requests on the fly, while satisfying users? time and resource constraints. We successfully tested our algorithm in a simple benchmark simulator that we have developed, and demonstrated its performance with initial test results.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC)
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 1165206
- Report Number(s):
- LBNL-6364E
- Resource Relation:
- Conference: IEEE 27th International Conference on Advanced Information Networking and Applications (AINA), Barcelona, Spain Spain , March 25-March 28 2013
- Country of Publication:
- United States
- Language:
- English
Similar Records
A New Approach in Advance Network Reservation and Provisioning for High-Performance Scientific Data Transfers
An Online Scheduling Algorithm with Advance Reservation for Large-Scale Data Transfers