skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Orchestrating Bulk Data Movement in Grid Environments

Abstract

Data Grids provide a convenient environment for researchers to manage and access massively distributed bulk data by addressing several system and transfer challenges inherent to these environments. This work addresses issues involved in the efficient selection and access of replicated data in Grid environments in the context of the Globus Toolkit{trademark}, building middleware that (1) selects datasets in highly replicated environments, enabling efficient scheduling of data transfer requests; (2) predicts transfer times of bulk wide-area data transfers using extensive statistical analysis; and (3) co-allocates bulk data transfer requests, enabling parallel downloads from mirrored sites. These efforts have demonstrated a decentralized data scheduling architecture, a set of forecasting tools that predict bandwidth availability within 15% error and co-allocation architecture, and heuristics that expedites data downloads by up to 2 times.

Authors:
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
885937
Report Number(s):
ORNL/TM-2004/121
TRN: US200617%%469
DOE Contract Number:  
DE-AC05-00OR22725
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; ARCHITECTURE; AVAILABILITY; FORECASTING

Citation Formats

Vazhkudai, SS. Orchestrating Bulk Data Movement in Grid Environments. United States: N. p., 2005. Web. doi:10.2172/885937.
Vazhkudai, SS. Orchestrating Bulk Data Movement in Grid Environments. United States. doi:10.2172/885937.
Vazhkudai, SS. Tue . "Orchestrating Bulk Data Movement in Grid Environments". United States. doi:10.2172/885937. https://www.osti.gov/servlets/purl/885937.
@article{osti_885937,
title = {Orchestrating Bulk Data Movement in Grid Environments},
author = {Vazhkudai, SS},
abstractNote = {Data Grids provide a convenient environment for researchers to manage and access massively distributed bulk data by addressing several system and transfer challenges inherent to these environments. This work addresses issues involved in the efficient selection and access of replicated data in Grid environments in the context of the Globus Toolkit{trademark}, building middleware that (1) selects datasets in highly replicated environments, enabling efficient scheduling of data transfer requests; (2) predicts transfer times of bulk wide-area data transfers using extensive statistical analysis; and (3) co-allocates bulk data transfer requests, enabling parallel downloads from mirrored sites. These efforts have demonstrated a decentralized data scheduling architecture, a set of forecasting tools that predict bandwidth availability within 15% error and co-allocation architecture, and heuristics that expedites data downloads by up to 2 times.},
doi = {10.2172/885937},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2005},
month = {1}
}

Technical Report:

Save / Share: