skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: dCache on Steroids - Delegated Storage Solutions

Abstract

For over a decade, dCache.org has delivered a robust software used at more than 80 Universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments as well as many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from a SoC based all-in-one Raspberry-Pi up to hundreds of nodes in a multipetabyte installation. Due to lack of managed storage at the time, dCache implemented data placement, replication and data integrity directly. Today, many alternatives are available: S3, GlusterFS, CEPH and others. While such solutions position themselves as scalable storage systems, they cannot be used by many scientific communities out of the box. The absence of community-accepted authentication and authorization mechanisms, the use of product specific protocols and the lack of namespace are some of the reasons that prevent wide-scale adoption of these alternatives. Most of these limitations are already solved by dCache. By delegating low-level storage management functionality to the above-mentioned new systems and providing the missing layer through dCache, we provide a solution which combines the benefits of both worlds - industry standard storage building blocks with the accessmore » protocols and authentication required by scientific communities. In this paper, we focus on CEPH, a popular software for clustered storage that supports file, block and object interfaces. CEPH is often used in modern computing centers, for example as a backend to OpenStack services. We will show prototypes of dCache running with a CEPH backend and discuss the benefits and limitations of such an approach. As a result, we will also outline the roadmap for supporting ‘delegated storage’ within the dCache releases.« less

Authors:
 [1];  [1];  [1];  [2];  [1]; ORCiD logo [3];  [1];  [3];  [1];  [1]
  1. Deutsches Elektronen-Synchrotron (DESY), Hamburg (Germany)
  2. Nordic e-Infrastructure Collaboration (NeIC), Copenhagen (Denmark)
  3. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Publication Date:
Research Org.:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
OSTI Identifier:
1420913
Report Number(s):
FERMILAB-CONF-17-647-CD
Journal ID: ISSN 1742-6588; 1638513; TRN: US1801508
Grant/Contract Number:
AC02-07CH11359
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Journal of Physics. Conference Series
Additional Journal Information:
Journal Volume: 898; Journal Issue: 6; Journal ID: ISSN 1742-6588
Publisher:
IOP Publishing
Country of Publication:
United States
Language:
English
Subject:
96 KNOWLEDGE MANAGEMENT AND PRESERVATION

Citation Formats

Mkrtchyan, Tigran, Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Rossi, A., Sahakyan, M., and Starek, J. dCache on Steroids - Delegated Storage Solutions. United States: N. p., 2017. Web. doi:10.1088/1742-6596/898/6/062021.
Mkrtchyan, Tigran, Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Rossi, A., Sahakyan, M., & Starek, J. dCache on Steroids - Delegated Storage Solutions. United States. doi:10.1088/1742-6596/898/6/062021.
Mkrtchyan, Tigran, Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Rossi, A., Sahakyan, M., and Starek, J. Thu . "dCache on Steroids - Delegated Storage Solutions". United States. doi:10.1088/1742-6596/898/6/062021. https://www.osti.gov/servlets/purl/1420913.
@article{osti_1420913,
title = {dCache on Steroids - Delegated Storage Solutions},
author = {Mkrtchyan, Tigran and Adeyemi, F. and Ashish, A. and Behrmann, G. and Fuhrmann, P. and Litvintsev, D. and Millar, P. and Rossi, A. and Sahakyan, M. and Starek, J.},
abstractNote = {For over a decade, dCache.org has delivered a robust software used at more than 80 Universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments as well as many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from a SoC based all-in-one Raspberry-Pi up to hundreds of nodes in a multipetabyte installation. Due to lack of managed storage at the time, dCache implemented data placement, replication and data integrity directly. Today, many alternatives are available: S3, GlusterFS, CEPH and others. While such solutions position themselves as scalable storage systems, they cannot be used by many scientific communities out of the box. The absence of community-accepted authentication and authorization mechanisms, the use of product specific protocols and the lack of namespace are some of the reasons that prevent wide-scale adoption of these alternatives. Most of these limitations are already solved by dCache. By delegating low-level storage management functionality to the above-mentioned new systems and providing the missing layer through dCache, we provide a solution which combines the benefits of both worlds - industry standard storage building blocks with the access protocols and authentication required by scientific communities. In this paper, we focus on CEPH, a popular software for clustered storage that supports file, block and object interfaces. CEPH is often used in modern computing centers, for example as a backend to OpenStack services. We will show prototypes of dCache running with a CEPH backend and discuss the benefits and limitations of such an approach. As a result, we will also outline the roadmap for supporting ‘delegated storage’ within the dCache releases.},
doi = {10.1088/1742-6596/898/6/062021},
journal = {Journal of Physics. Conference Series},
number = 6,
volume = 898,
place = {United States},
year = {Thu Nov 23 00:00:00 EST 2017},
month = {Thu Nov 23 00:00:00 EST 2017}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share: