Grid Data Access on Widely Distributed Worker Nodes Using Scalla and SRM
Facing the reality of storage economics, NP experiments such as RHIC/STAR have been engaged in a shift of the analysis model, and now heavily rely on using cheap disks attached to processing nodes, as such a model is extremely beneficial over expensive centralized storage. Additionally, exploiting storage aggregates with enhanced distributed computing capabilities such as dynamic space allocation (lifetime of spaces), file management on shared storages (lifetime of files, pinning file), storage policies or a uniform access to heterogeneous storage solutions is not an easy task. The Xrootd/Scalla system allows for storage aggregation. We will present an overview of the largest deployment of Scalla (Structured Cluster Architecture for Low Latency Access) in the world spanning over 1000 CPUs co-sharing the 350 TB Storage Elements and the experience on how to make such a model work in the RHIC/STAR standard analysis framework. We will explain the key features and approach on how to make access to mass storage (HPSS) possible in such a large deployment context. Furthermore, we will give an overview of a fully 'gridified' solution using the plug-and-play features of Scalla architecture, replacing standard storage access with grid middleware SRM (Storage Resource Manager) components designed for space management and will compare the solution with the standard Scalla approach in use in STAR for the past 2 years. Integration details, future plans and status of development will be explained in the area of best transfer strategy between multiple-choice data pools and best placement with respect of load balancing and interoperability with other SRM aware tools or implementations.
- Research Organization:
- SLAC National Accelerator Lab., Menlo Park, CA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC02-76SF00515
- OSTI ID:
- 1029161
- Report Number(s):
- SLAC-PUB-14709; TRN: US1105638
- Journal Information:
- J.Phys.Conf.Ser.119:072019,2008, Conference: Prepared for International Conference on Computing in High Energy and Nuclear Physics (CHEP 07), Victoria, BC, Canada, 2-7 Sep 2007
- Country of Publication:
- United States
- Language:
- English
Similar Records
Storage resource manager
National Geothermal Data System: Transforming the Discovery, Access, and Analytics of Data for Geothermal Exploration