Petabyte class storage at Jefferson Lab (CEBAF)
By 1997, this facility will collect over 1 Terabyte of raw information/day accelerator operation from three concurrently operating experimental halls. With post-processing, it means that about 250 TB raw and formatted experimental data will be generated each year. By the year 2000, a total of one Petabyte will be stored on-line. Critical to the program is the networking and computational capability to collect, store, retrieve, and reconstruct data on this scale. Design criteria include support of a raw data stream of 10-12 MB/second from Experimental Hall B, which will operate the CEBAF Large Acceptance Spectrometer. Keeping up with this data stream implies design strategies that provide storage guarantees during accelerator operation, minimize the number of times data is buffered, allow seamless access to specific data sets for the researcher, synchronize data retrievals with scheduling of postprocessing calculations on the data reconstruction CPU farms, as well as support the site capability for data reconstruction and reduction at same overall rate that new data is being collected. Current implementation uses state of the art StorageTek Redwood tape drives and robotics library integrated with the Open Storage Manager Hierarchical Storage Management software (Computer Associates, International), the use of Fibre Channel RAID disks dual-ported between Sun Microsystems SMP servers, and a network-based interface to a 10,000 SPECint92 data processing CPU farm. Issues of efficiency, scaleability, and manageability will become critical to meet the year 2000 requirements for a Petabyte of near-line storage interfaced to over 30,000 SPECint92 of data processing power.
- Research Organization:
- Thomas Jefferson National Accelerator Facility (TJNAF), Newport News, VA (United States)
- Sponsoring Organization:
- USDOE, Washington, DC (United States)
- DOE Contract Number:
- AC05-84ER40150
- OSTI ID:
- 283608
- Report Number(s):
- DOE/ER/40150-371; CONF-9609219-2; ON: DE96013057
- Resource Relation:
- Conference: 5. NASA Goddard Space Flight Center conference on mass storage systems and technologies, College Park, MD (United States), 17-19 Sep 1996; Other Information: PBD: [1996]
- Country of Publication:
- United States
- Language:
- English
Similar Records
Lessons Learned from Managing a Petabyte
Distributed data access in the sequential access model at the D0 experiment at Fermilab