skip to main content

DOE PAGESDOE PAGES

Title: dCache on Steroids - Delegated Storage Solutions

For over a decade, dCache.org has delivered a robust software used at more than 80 Universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments as well as many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from a SoC based all-in-one Raspberry-Pi up to hundreds of nodes in a multipetabyte installation. Due to lack of managed storage at the time, dCache implemented data placement, replication and data integrity directly. Today, many alternatives are available: S3, GlusterFS, CEPH and others. While such solutions position themselves as scalable storage systems, they cannot be used by many scientific communities out of the box. The absence of community-accepted authentication and authorization mechanisms, the use of product specific protocols and the lack of namespace are some of the reasons that prevent wide-scale adoption of these alternatives. Most of these limitations are already solved by dCache. By delegating low-level storage management functionality to the above-mentioned new systems and providing the missing layer through dCache, we provide a solution which combines the benefits of both worlds - industry standard storage building blocks with the accessmore » protocols and authentication required by scientific communities. In this paper, we focus on CEPH, a popular software for clustered storage that supports file, block and object interfaces. CEPH is often used in modern computing centers, for example as a backend to OpenStack services. We will show prototypes of dCache running with a CEPH backend and discuss the benefits and limitations of such an approach. As a result, we will also outline the roadmap for supporting ‘delegated storage’ within the dCache releases.« less
Authors:
 [1] ;  [1] ;  [1] ;  [2] ;  [1] ; ORCiD logo [3] ;  [1] ;  [3] ;  [1] ;  [1]
  1. Deutsches Elektronen-Synchrotron (DESY), Hamburg (Germany)
  2. Nordic e-Infrastructure Collaboration (NeIC), Copenhagen (Denmark)
  3. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Publication Date:
Report Number(s):
FERMILAB-CONF-17-647-CD
Journal ID: ISSN 1742-6588; 1638513; TRN: US1801508
Grant/Contract Number:
AC02-07CH11359
Type:
Accepted Manuscript
Journal Name:
Journal of Physics. Conference Series
Additional Journal Information:
Journal Volume: 898; Journal Issue: 6; Journal ID: ISSN 1742-6588
Publisher:
IOP Publishing
Research Org:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
Country of Publication:
United States
Language:
English
Subject:
96 KNOWLEDGE MANAGEMENT AND PRESERVATION
OSTI Identifier:
1420913

Mkrtchyan, Tigran, Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Rossi, A., Sahakyan, M., and Starek, J.. dCache on Steroids - Delegated Storage Solutions. United States: N. p., Web. doi:10.1088/1742-6596/898/6/062021.
Mkrtchyan, Tigran, Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Rossi, A., Sahakyan, M., & Starek, J.. dCache on Steroids - Delegated Storage Solutions. United States. doi:10.1088/1742-6596/898/6/062021.
Mkrtchyan, Tigran, Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Rossi, A., Sahakyan, M., and Starek, J.. 2017. "dCache on Steroids - Delegated Storage Solutions". United States. doi:10.1088/1742-6596/898/6/062021. https://www.osti.gov/servlets/purl/1420913.
@article{osti_1420913,
title = {dCache on Steroids - Delegated Storage Solutions},
author = {Mkrtchyan, Tigran and Adeyemi, F. and Ashish, A. and Behrmann, G. and Fuhrmann, P. and Litvintsev, D. and Millar, P. and Rossi, A. and Sahakyan, M. and Starek, J.},
abstractNote = {For over a decade, dCache.org has delivered a robust software used at more than 80 Universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments as well as many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from a SoC based all-in-one Raspberry-Pi up to hundreds of nodes in a multipetabyte installation. Due to lack of managed storage at the time, dCache implemented data placement, replication and data integrity directly. Today, many alternatives are available: S3, GlusterFS, CEPH and others. While such solutions position themselves as scalable storage systems, they cannot be used by many scientific communities out of the box. The absence of community-accepted authentication and authorization mechanisms, the use of product specific protocols and the lack of namespace are some of the reasons that prevent wide-scale adoption of these alternatives. Most of these limitations are already solved by dCache. By delegating low-level storage management functionality to the above-mentioned new systems and providing the missing layer through dCache, we provide a solution which combines the benefits of both worlds - industry standard storage building blocks with the access protocols and authentication required by scientific communities. In this paper, we focus on CEPH, a popular software for clustered storage that supports file, block and object interfaces. CEPH is often used in modern computing centers, for example as a backend to OpenStack services. We will show prototypes of dCache running with a CEPH backend and discuss the benefits and limitations of such an approach. As a result, we will also outline the roadmap for supporting ‘delegated storage’ within the dCache releases.},
doi = {10.1088/1742-6596/898/6/062021},
journal = {Journal of Physics. Conference Series},
number = 6,
volume = 898,
place = {United States},
year = {2017},
month = {11}
}