Petabyte class storage at Jefferson Lab (CEBAF)
Conference
·
OSTI ID:283608
By 1997, this facility will collect over 1 Terabyte of raw information/day accelerator operation from three concurrently operating experimental halls. With post-processing, it means that about 250 TB raw and formatted experimental data will be generated each year. By the year 2000, a total of one Petabyte will be stored on-line. Critical to the program is the networking and computational capability to collect, store, retrieve, and reconstruct data on this scale. Design criteria include support of a raw data stream of 10-12 MB/second from Experimental Hall B, which will operate the CEBAF Large Acceptance Spectrometer. Keeping up with this data stream implies design strategies that provide storage guarantees during accelerator operation, minimize the number of times data is buffered, allow seamless access to specific data sets for the researcher, synchronize data retrievals with scheduling of postprocessing calculations on the data reconstruction CPU farms, as well as support the site capability for data reconstruction and reduction at same overall rate that new data is being collected. Current implementation uses state of the art StorageTek Redwood tape drives and robotics library integrated with the Open Storage Manager Hierarchical Storage Management software (Computer Associates, International), the use of Fibre Channel RAID disks dual-ported between Sun Microsystems SMP servers, and a network-based interface to a 10,000 SPECint92 data processing CPU farm. Issues of efficiency, scaleability, and manageability will become critical to meet the year 2000 requirements for a Petabyte of near-line storage interfaced to over 30,000 SPECint92 of data processing power.
- Research Organization:
- Thomas Jefferson National Accelerator Facility, Newport News, VA (United States)
- Sponsoring Organization:
- USDOE, Washington, DC (United States)
- DOE Contract Number:
- AC05-84ER40150
- OSTI ID:
- 283608
- Report Number(s):
- DOE/ER/40150--371; CONF-9609219--2; ON: DE96013057
- Country of Publication:
- United States
- Language:
- English
Similar Records
Lessons Learned from Managing a Petabyte
Data management for batch systems
Monitoring a petabyte scale storage system
Technical Report
·
Wed Jan 19 23:00:00 EST 2005
·
OSTI ID:839755
Data management for batch systems
Conference
·
Mon Mar 31 23:00:00 EST 1997
·
OSTI ID:756240
Monitoring a petabyte scale storage system
Conference
·
Tue Nov 30 23:00:00 EST 2004
·
OSTI ID:15016947