Data Resilience in the dCache Storage System

Rossi, A. L.; Adeyemi, F.; Ashish, A.; Behrmann, G.; Fuhrmann, P.; Litvintsev, D.; Millar, P.; Mkrtchyan, T.; Mohiuddin, A.; Sahakyan, M.; Starek, J.; Yasar, S.

doi:10.1088/1742-6596/898/6/062024

Title: Data Resilience in the dCache Storage System

Full Record
Other Related Research

Abstract

In this study we discuss design, implementation considerations, and performance of a new Resilience Service in the dCache storage system responsible for file availability and durability functionality.

Authors:

Rossi, A. L. ^[1]; Adeyemi, F. ^[2]; Ashish, A. ^[2]; Behrmann, G. ^[3]; Fuhrmann, P. ^[2]; Litvintsev, D. ^[1]; Millar, P. ^[2]; Mkrtchyan, T. ^[2]; Mohiuddin, A. ^[2]; Sahakyan, M. ^[2]; Starek, J. ^[2]; Yasar, S. ^[2]

Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Deutsches Elektronen-Synchrotron (DESY), Hamburg (Germany)
NORDUnet, Copenhagen (Denmark)

Publication Date:: Thu Nov 23 00:00:00 EST 2017

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1420912

Report Number(s):: FERMILAB-CONF-16-764-CD
Journal ID: ISSN 1742-6588; 1638515; TRN: US1801507

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Physics. Conference Series

Additional Journal Information:: Journal Volume: 898; Journal Issue: 6; Journal ID: ISSN 1742-6588

Publisher:: IOP Publishing

Country of Publication:: United States

Language:: English

Subject:: 96 KNOWLEDGE MANAGEMENT AND PRESERVATION

Citation Formats


                    Rossi, A. L., Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Mkrtchyan, T., Mohiuddin, A., Sahakyan, M., Starek, J., and Yasar, S. Data Resilience in the dCache Storage System.  United States: N. p., 2017. 
Web.  doi:10.1088/1742-6596/898/6/062024.

Copy to clipboard


                    Rossi, A. L., Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Mkrtchyan, T., Mohiuddin, A., Sahakyan, M., Starek, J., & Yasar, S. Data Resilience in the dCache Storage System.  United States.  https://doi.org/10.1088/1742-6596/898/6/062024

Copy to clipboard


                    Rossi, A. L., Adeyemi, F., Ashish, A., Behrmann, G., Fuhrmann, P., Litvintsev, D., Millar, P., Mkrtchyan, T., Mohiuddin, A., Sahakyan, M., Starek, J., and Yasar, S. Thu .  
"Data Resilience in the dCache Storage System".  United States.  https://doi.org/10.1088/1742-6596/898/6/062024.  https://www.osti.gov/servlets/purl/1420912.

Copy to clipboard


                    
@article{osti_1420912,

  title        = {Data Resilience in the dCache Storage System},

  author       = {Rossi, A. L. and Adeyemi, F. and Ashish, A. and Behrmann, G. and Fuhrmann, P. and Litvintsev, D. and Millar, P. and Mkrtchyan, T. and Mohiuddin, A. and Sahakyan, M. and Starek, J. and Yasar, S.},

  abstractNote = {In this study we discuss design, implementation considerations, and performance of a new Resilience Service in the dCache storage system responsible for file availability and durability functionality.},

  doi          = {10.1088/1742-6596/898/6/062024},

  journal      = {Journal of Physics. Conference Series},

  number       = 6,

  volume       = 898,

  place        = {United States},

  year         = {Thu Nov 23 00:00:00 EST 2017},

  month        = {Thu Nov 23 00:00:00 EST 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1088/1742-6596/898/6/062024

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

dCache: from Resilience to Quality of Service

Journal Article Rossi, Albert ; Chitrapu, Krishnaveni ; Garonne, Vincent ; ... - EPJ Web of Conferences (Online)

A major goal of future dCache development will be to allow users to define file Quality of Service (QoS) in a more flexible way than currently available. This will mean implementing what might be called a QoS rule engine responsible for registering and managing time-bound QoS transitions for files or storage units. In anticipation of this extension to existing dCache capabilities, the Resilience service, which maintains on-disk replica state, needs to undergo both structural modification and generalization. This paper describes ongoing work to transform Resilience into the new architecture which will eventually support a more broadly defined file QoS.
https://doi.org/10.1051/epjconf/202125102043

Full Text Available
dCache on Steroids - Delegated Storage Solutions

Journal Article Mkrtchyan, Tigran ; Adeyemi, F. ; Ashish, A. ; ... - Journal of Physics. Conference Series

For over a decade, dCache.org has delivered a robust software used at more than 80 Universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments as well as many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from a SoC based all-in-one Raspberry-Pi up to hundreds of nodes in a multipetabyte installation. Due to lack of managed storage at the time, dCache implemented data placement, replication and data integrity directly. Today, many alternatives are available: S3, GlusterFS, CEPH andmore »« less
https://doi.org/10.1088/1742-6596/898/6/062021

Full Text Available
dCache - Joining the noWORM Storage Club

Conference Mkrtchyan, Tigran ; Adeyemi, Olufemi ; Fuhrmann, Patrick ; ... - EPJ Web Conf.

For over a decade, dCache.ORG has provided robust software, called dCache, that is used at more than 80 universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments and many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from all-in-one Raspberry-Pi up to hundreds of nodes in multi-petabyte infrastructures. The life cycle of scientific data is well defined - collected, processed, archived and finally deleted, when it’s not needed anymore. Moreover, during all those stages the data is nevermore »« less
https://doi.org/10.1051/epjconf/201921404048

Full Text Available
Quota management in dCache or making a perfectly normal file system normal

Conference Litvintsev, Dmitry ; Krishnaveni, Chitrapu ; Meyer, Svenja ; ...

dCache (https://dcache.org) is a highly scalable storage system providing location-independent access to data. The data are stored across multiple data servers as complete files presented to the end-user via a single-rooted namespace. From its inception, dCache has been designed as a caching disk buffer to a tertiary tape storage system with the assumption that the latter has virtually unlimited capacity. dCache can also be configured as a disk-only storage system with no tape backend. Owing to the idea that a tape resource is infinite, or purely physically limited by budget considerations, the system has never provided for any restrictions onmore »« less
Full Text Available
Horizontally scaling dChache SRM with the Terracotta platform

Conference Perelmutov, T ; Crawford, M ; Moibenko, A ; ... - Submitted to J.Phys.Conf.Ser.

The dCache disk caching file system has been chosen by a majority of LHC experiments Tier 1 centers for their data storage needs. It is also deployed at many Tier 2 centers. The Storage Resource Manager (SRM) is a standardized grid storage interface and a single point of remote entry into dCache, and hence is a critical component. SRM must scale to increasing transaction rates and remain resilient against changing usage patterns. The initial implementation of the SRM service in dCache suffered from an inability to support clustered deployment, and its performance was limited by the hardware of a singlemore »« less
Full Text Available

Similar Records