skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Advance Resource Provisioning in Bulk Data Scheduling

Abstract

Today?s scientific and business applications generate mas- sive data sets that need to be transferred to remote sites for sharing, processing, and long term storage. Because of increasing data volumes and enhancement in current net- work technology that provide on-demand high-speed data access between collaborating institutions, data handling and scheduling problems have reached a new scale. In this paper, we present a new data scheduling model with ad- vance resource provisioning, in which data movement operations are defined with earliest start and latest comple- tion times. We analyze time-dependent resource assign- ment problem, and propose a new methodology to improve the current systems by allowing researchers and higher-level meta-schedulers to use data-placement as-a-service, so they can plan ahead and submit transfer requests in advance. In general, scheduling with time and resource conflicts is NP-hard. We introduce an efficient algorithm to organize multiple requests on the fly, while satisfying users? time and resource constraints. We successfully tested our algorithm in a simple benchmark simulator that we have developed, and demonstrated its performance with initial test results.

Authors:
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1165206
Report Number(s):
LBNL-6364E
DOE Contract Number:  
DE-AC02-05CH11231
Resource Type:
Conference
Resource Relation:
Conference: IEEE 27th International Conference on Advanced Information Networking and Applications (AINA), Barcelona, Spain Spain , March 25-March 28 2013
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Gale- Shapley algorithm, scheduling with constraints, bulk data move- ment, time-dependent graphs, network virtualization

Citation Formats

Balman, Mehmet. Advance Resource Provisioning in Bulk Data Scheduling. United States: N. p., 2012. Web.
Balman, Mehmet. Advance Resource Provisioning in Bulk Data Scheduling. United States.
Balman, Mehmet. Mon . "Advance Resource Provisioning in Bulk Data Scheduling". United States. https://www.osti.gov/servlets/purl/1165206.
@article{osti_1165206,
title = {Advance Resource Provisioning in Bulk Data Scheduling},
author = {Balman, Mehmet},
abstractNote = {Today?s scientific and business applications generate mas- sive data sets that need to be transferred to remote sites for sharing, processing, and long term storage. Because of increasing data volumes and enhancement in current net- work technology that provide on-demand high-speed data access between collaborating institutions, data handling and scheduling problems have reached a new scale. In this paper, we present a new data scheduling model with ad- vance resource provisioning, in which data movement operations are defined with earliest start and latest comple- tion times. We analyze time-dependent resource assign- ment problem, and propose a new methodology to improve the current systems by allowing researchers and higher-level meta-schedulers to use data-placement as-a-service, so they can plan ahead and submit transfer requests in advance. In general, scheduling with time and resource conflicts is NP-hard. We introduce an efficient algorithm to organize multiple requests on the fly, while satisfying users? time and resource constraints. We successfully tested our algorithm in a simple benchmark simulator that we have developed, and demonstrated its performance with initial test results.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2012},
month = {10}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: