skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: ZOID : I/O-forwarding infrasatructure for petascale architectures.

Abstract

The ZeptoOS project is developing an open-source alternative to the proprietary software stacks available on contemporary massively parallel architectures. The aim is to enable computer science research on these architectures, enhance community collaboration, and foster innovation. In this paper, we introduce a component of ZeptoOS called ZOID - an I/O-forwarding infrastructure for architectures such as IBM Blue Gene that decouple file and socket I/O from the compute nodes, shipping those functions to dedicated I/O nodes. Through the use of optimized network protocols and data paths, as well as a multithreaded daemon running on I/O nodes, ZOID provides greater performance than does the stock infrastructure. We present a set of benchmark results that highlight the improvements. Our infrastructure also offers vastly improved flexibility, allowing users to forward data using custom-designed application interfaces, through an easy-to-use plug-in mechanism. This capability is used for real-time telescope data transfers, extensively discussed in the paper. Plug-in-specific threads implement prefetching of data obtained over sockets from an input cluster and merge results from individual compute nodes before sending them out, significantly reducing required network bandwidth. This approach allows a ZOID version of the application to handle a larger number of subbands per I/O node, or evenmore » to bypass the input cluster altogether, plugging the input from remote receiver stations directly into the I/O nodes. Using the resources more efficiently can result in considerable savings.« less

Authors:
; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
963856
Report Number(s):
ANL/MCS/CP-59999
TRN: US200918%%250
DOE Contract Number:  
DE-AC02-06CH11357
Resource Type:
Conference
Resource Relation:
Conference: 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP '08); Feb. 20, 2007 - Feb. 23, 2007; Salt Lake City, UT
Country of Publication:
United States
Language:
ENGLISH
Subject:
59 BASIC BIOLOGICAL SCIENCES; BENCHMARKS; COMPUTERS; FLEXIBILITY; GENES; PERFORMANCE; PLUGGING; PROGRAMMING; TELESCOPES

Citation Formats

Iskra, K, Romein, J W, Yoshii, K, Beckman, P, Mathematics and Computer Science, and Netherlands Foundation for Research in Astronomy. ZOID : I/O-forwarding infrasatructure for petascale architectures.. United States: N. p., 2008. Web.
Iskra, K, Romein, J W, Yoshii, K, Beckman, P, Mathematics and Computer Science, & Netherlands Foundation for Research in Astronomy. ZOID : I/O-forwarding infrasatructure for petascale architectures.. United States.
Iskra, K, Romein, J W, Yoshii, K, Beckman, P, Mathematics and Computer Science, and Netherlands Foundation for Research in Astronomy. Tue . "ZOID : I/O-forwarding infrasatructure for petascale architectures.". United States.
@article{osti_963856,
title = {ZOID : I/O-forwarding infrasatructure for petascale architectures.},
author = {Iskra, K and Romein, J W and Yoshii, K and Beckman, P and Mathematics and Computer Science and Netherlands Foundation for Research in Astronomy},
abstractNote = {The ZeptoOS project is developing an open-source alternative to the proprietary software stacks available on contemporary massively parallel architectures. The aim is to enable computer science research on these architectures, enhance community collaboration, and foster innovation. In this paper, we introduce a component of ZeptoOS called ZOID - an I/O-forwarding infrastructure for architectures such as IBM Blue Gene that decouple file and socket I/O from the compute nodes, shipping those functions to dedicated I/O nodes. Through the use of optimized network protocols and data paths, as well as a multithreaded daemon running on I/O nodes, ZOID provides greater performance than does the stock infrastructure. We present a set of benchmark results that highlight the improvements. Our infrastructure also offers vastly improved flexibility, allowing users to forward data using custom-designed application interfaces, through an easy-to-use plug-in mechanism. This capability is used for real-time telescope data transfers, extensively discussed in the paper. Plug-in-specific threads implement prefetching of data obtained over sockets from an input cluster and merge results from individual compute nodes before sending them out, significantly reducing required network bandwidth. This approach allows a ZOID version of the application to handle a larger number of subbands per I/O node, or even to bypass the input cluster altogether, plugging the input from remote receiver stations directly into the I/O nodes. Using the resources more efficiently can result in considerable savings.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2008},
month = {1}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: