skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Enabling distributed petascale science.

Abstract

Petascale science is an end-to-end endeavour, involving not only the creation of massive datasets at supercomputers or experimental facilities, but the subsequent analysis of that data by a user community that may be distributed across many laboratories and universities. The new SciDAC Center for Enabling Distributed Petascale Science (CEDPS) is developing tools to support this end-to-end process. These tools include data placement services for the reliable, high-performance, secure, and policy-driven placement of data within a distributed science environment; tools and techniques for the construction, operation, and provisioning of scalable science services; and tools for the detection and diagnosis of failures in end-to-end data placement and distributed application hosting configurations. In each area, we build on a strong base of existing technology and have made useful progress in the first year of the project. For example, we have recently achieved order-of-magnitude improvements in transfer times (for lots of small files) and implemented asynchronous data staging capabilities; demonstrated dynamic deployment of complex application stacks for the STAR experiment; and designed and deployed end-to-end troubleshooting services. We look forward to working with SciDAC application and technology projects to realize the promise of petascale science.

Authors:
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more »; ; ; ; « less
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
973002
Report Number(s):
ANL/MCS/CP-59797
TRN: US201005%%399
DOE Contract Number:  
DE-AC02-06CH11357
Resource Type:
Conference
Journal Name:
J. Phys.: Conf. Ser.
Additional Journal Information:
Journal Volume: 78; Journal Issue: 2007; Conference: Scientific Discovery through Advanced Computing (SciDAC 2007); Jun. 24, 2007 - Jun. 26, 2007; Boston, MA
Country of Publication:
United States
Language:
ENGLISH
Subject:
97 MATHEMATICAL METHODS AND COMPUTING; 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; SUPERCOMPUTERS; SIZE; DATA ANALYSIS; PARALLEL PROCESSING; DATA TRANSMISSION; PERFORMANCE

Citation Formats

Baranovski, A, Bharathi, S, Bresnahan, J, chervenak, A, Foster, I, Fraser, D, Freeman, T, Gunter, D, Jackson, K, Keahey, K, Kesselman, C, Konerding, D E, Leroy, N, Link, M, Livny, M, Miller, N, Miller, R, Oleynik, G, Pearlman, L, Schopf, J M, Schuler, R, Tierney, B, Mathematics and Computer Science, FNL,, Univ. of Southern California, Univ. of Chicago, LBNL,, and Univ. of Wisconsin. Enabling distributed petascale science.. United States: N. p., 2007. Web. doi:10.1088/1742-6596/78/1/012020.
Baranovski, A, Bharathi, S, Bresnahan, J, chervenak, A, Foster, I, Fraser, D, Freeman, T, Gunter, D, Jackson, K, Keahey, K, Kesselman, C, Konerding, D E, Leroy, N, Link, M, Livny, M, Miller, N, Miller, R, Oleynik, G, Pearlman, L, Schopf, J M, Schuler, R, Tierney, B, Mathematics and Computer Science, FNL,, Univ. of Southern California, Univ. of Chicago, LBNL,, & Univ. of Wisconsin. Enabling distributed petascale science.. United States. https://doi.org/10.1088/1742-6596/78/1/012020
Baranovski, A, Bharathi, S, Bresnahan, J, chervenak, A, Foster, I, Fraser, D, Freeman, T, Gunter, D, Jackson, K, Keahey, K, Kesselman, C, Konerding, D E, Leroy, N, Link, M, Livny, M, Miller, N, Miller, R, Oleynik, G, Pearlman, L, Schopf, J M, Schuler, R, Tierney, B, Mathematics and Computer Science, FNL,, Univ. of Southern California, Univ. of Chicago, LBNL,, and Univ. of Wisconsin. 2007. "Enabling distributed petascale science.". United States. https://doi.org/10.1088/1742-6596/78/1/012020.
@article{osti_973002,
title = {Enabling distributed petascale science.},
author = {Baranovski, A and Bharathi, S and Bresnahan, J and chervenak, A and Foster, I and Fraser, D and Freeman, T and Gunter, D and Jackson, K and Keahey, K and Kesselman, C and Konerding, D E and Leroy, N and Link, M and Livny, M and Miller, N and Miller, R and Oleynik, G and Pearlman, L and Schopf, J M and Schuler, R and Tierney, B and Mathematics and Computer Science and FNL, and Univ. of Southern California and Univ. of Chicago and LBNL, and Univ. of Wisconsin},
abstractNote = {Petascale science is an end-to-end endeavour, involving not only the creation of massive datasets at supercomputers or experimental facilities, but the subsequent analysis of that data by a user community that may be distributed across many laboratories and universities. The new SciDAC Center for Enabling Distributed Petascale Science (CEDPS) is developing tools to support this end-to-end process. These tools include data placement services for the reliable, high-performance, secure, and policy-driven placement of data within a distributed science environment; tools and techniques for the construction, operation, and provisioning of scalable science services; and tools for the detection and diagnosis of failures in end-to-end data placement and distributed application hosting configurations. In each area, we build on a strong base of existing technology and have made useful progress in the first year of the project. For example, we have recently achieved order-of-magnitude improvements in transfer times (for lots of small files) and implemented asynchronous data staging capabilities; demonstrated dynamic deployment of complex application stacks for the STAR experiment; and designed and deployed end-to-end troubleshooting services. We look forward to working with SciDAC application and technology projects to realize the promise of petascale science.},
doi = {10.1088/1742-6596/78/1/012020},
url = {https://www.osti.gov/biblio/973002}, journal = {J. Phys.: Conf. Ser.},
number = 2007,
volume = 78,
place = {United States},
year = {Mon Jan 01 00:00:00 EST 2007},
month = {Mon Jan 01 00:00:00 EST 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: