dCache: from Resilience to Quality of Service

Rossi, Albert; Chitrapu, Krishnaveni; Garonne, Vincent; Litvintsev, Dmitry; Meyer, Svenja; Millar, Paul; Mkrtchyan, Tigran; Morschel, Lea; Sahakyan, Marina

doi:10.1051/epjconf/202125102043

Title: dCache: from Resilience to Quality of Service

Abstract

A major goal of future dCache development will be to allow users to define file Quality of Service (QoS) in a more flexible way than currently available. This will mean implementing what might be called a QoS rule engine responsible for registering and managing time-bound QoS transitions for files or storage units. In anticipation of this extension to existing dCache capabilities, the Resilience service, which maintains on-disk replica state, needs to undergo both structural modification and generalization. This paper describes ongoing work to transform Resilience into the new architecture which will eventually support a more broadly defined file QoS.

Authors:

Rossi, Albert ^[1]; Chitrapu, Krishnaveni ^[2]; Garonne, Vincent ^[3]; Litvintsev, Dmitry ^[1]; Meyer, Svenja ^[4]; Millar, Paul ^[4]; Mkrtchyan, Tigran ^[4]; Morschel, Lea ^[4]; Sahakyan, Marina ^[4]

Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Linköping University (Sweden)
Univ. of Oslo (Norway)
Deutsches Elektronen-Synchrotron (DESY), Hamburg (Germany)

Publication Date:: Mon Aug 23 00:00:00 EDT 2021

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1831934

Report Number(s):: FERMILAB-CONF-21-212-SCD
Journal ID: ISSN 2100-014X; oai:inspirehep.net:1914520; TRN: US2216678

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Accepted Manuscript

Journal Name:: EPJ Web of Conferences (Online)

Additional Journal Information:: Journal Name: EPJ Web of Conferences (Online); Journal Volume: 251; Journal ID: ISSN 2100-014X

Publisher:: EDP Sciences

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Rossi, Albert, Chitrapu, Krishnaveni, Garonne, Vincent, Litvintsev, Dmitry, Meyer, Svenja, Millar, Paul, Mkrtchyan, Tigran, Morschel, Lea, and Sahakyan, Marina. dCache: from Resilience to Quality of Service.  United States: N. p., 2021. 
Web.  doi:10.1051/epjconf/202125102043.

Copy to clipboard


                    Rossi, Albert, Chitrapu, Krishnaveni, Garonne, Vincent, Litvintsev, Dmitry, Meyer, Svenja, Millar, Paul, Mkrtchyan, Tigran, Morschel, Lea, & Sahakyan, Marina. dCache: from Resilience to Quality of Service.  United States.  https://doi.org/10.1051/epjconf/202125102043

Copy to clipboard


                    Rossi, Albert, Chitrapu, Krishnaveni, Garonne, Vincent, Litvintsev, Dmitry, Meyer, Svenja, Millar, Paul, Mkrtchyan, Tigran, Morschel, Lea, and Sahakyan, Marina. Mon .  
"dCache: from Resilience to Quality of Service".  United States.  https://doi.org/10.1051/epjconf/202125102043.  https://www.osti.gov/servlets/purl/1831934.

Copy to clipboard


                    
@article{osti_1831934,

  title        = {dCache: from Resilience to Quality of Service},

  author       = {Rossi, Albert and Chitrapu, Krishnaveni and Garonne, Vincent and Litvintsev, Dmitry and Meyer, Svenja and Millar, Paul and Mkrtchyan, Tigran and Morschel, Lea and Sahakyan, Marina},

  abstractNote = {A major goal of future dCache development will be to allow users to define file Quality of Service (QoS) in a more flexible way than currently available. This will mean implementing what might be called a QoS rule engine responsible for registering and managing time-bound QoS transitions for files or storage units. In anticipation of this extension to existing dCache capabilities, the Resilience service, which maintains on-disk replica state, needs to undergo both structural modification and generalization. This paper describes ongoing work to transform Resilience into the new architecture which will eventually support a more broadly defined file QoS.},

  doi          = {10.1051/epjconf/202125102043},

  journal      = {EPJ Web of Conferences (Online)},

  number       = ,

  volume       = 251,

  place        = {United States},

  year         = {Mon Aug 23 00:00:00 EDT 2021},

  month        = {Mon Aug 23 00:00:00 EDT 2021}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1051/epjconf/202125102043

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Data Resilience in the dCache Storage System
journal, October 2017

Rossi, A. L.; Adeyemi, F.; Ashish, A.
Journal of Physics: Conference Series, Vol. 898
DOI: 10.1088/1742-6596/898/6/062024

Storage quality-of-service in cloud-based scientific environments: a standardization approach
journal, October 2017

Millar, Paul; Fuhrmann, Patrick; Hardt, Marcus
Journal of Physics: Conference Series, Vol. 898
DOI: 10.1088/1742-6596/898/6/062043

Similar Records in DOE PAGES and OSTI.GOV collections:

dCache - Joining the noWORM Storage Club

Conference Mkrtchyan, Tigran ; Adeyemi, Olufemi ; Fuhrmann, Patrick ; ... - EPJ Web Conf.

For over a decade, dCache.ORG has provided robust software, called dCache, that is used at more than 80 universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments and many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from all-in-one Raspberry-Pi up to hundreds of nodes in multi-petabyte infrastructures. The life cycle of scientific data is well defined - collected, processed, archived and finally deleted, when it’s not needed anymore. Moreover, during all those stages the data is nevermore »« less
https://doi.org/10.1051/epjconf/201921404048

Full Text Available
dCache – Efficient Message Encoding For Inter-Service Communication in dCache: Evaluation of Existing Serialization Protocols as a Replacement for Java Object Serialization

Conference Morschel, Lea ; Adeyemi, Olufemi ; Garonne, Vincent ; ... - EPJ Web Conf.

As a well established, large-scale distributed storage system, dCache is used to manage and serve huge amounts of data collected by high energy physics, astrophysics and photon science experiments. Based on a microservices-like architecture, dCache is built as a modular distributed system, where each component provides a different core functionality. These services communicate by passing serialized messages to each other, a core behavior whose performance properties can consequently affect the entire system.This paper compares and evaluates different data serialization protocols in computer science with the objective of replacing and improving upon Java Object Serialization (JOS), which has increasingly presented itselfmore »« less
https://doi.org/10.1051/epjconf/202024505017

Full Text Available
dCache on Steroids - Delegated Storage Solutions

Journal Article Mkrtchyan, Tigran ; Adeyemi, F. ; Ashish, A. ; ... - Journal of Physics. Conference Series

For over a decade, dCache.org has delivered a robust software used at more than 80 Universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments as well as many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from a SoC based all-in-one Raspberry-Pi up to hundreds of nodes in a multipetabyte installation. Due to lack of managed storage at the time, dCache implemented data placement, replication and data integrity directly. Today, many alternatives are available: S3, GlusterFS, CEPH andmore »« less
https://doi.org/10.1088/1742-6596/898/6/062021

Full Text Available
Data Resilience in the dCache Storage System

Journal Article Rossi, A. L. ; Adeyemi, F. ; Ashish, A. ; ... - Journal of Physics. Conference Series

In this study we discuss design, implementation considerations, and performance of a new Resilience Service in the dCache storage system responsible for file availability and durability functionality.
https://doi.org/10.1088/1742-6596/898/6/062024

Full Text Available
A quality of service negotiation procedure for distributed multimedia presentational applications

Conference Hafid, A ; Bochmann, G V ; Kerherve, B

Most of current approaches in designing and implementing distributed multimedia (MM) presentational applications, e.g. news-on-demand, have concentrated on the performance of the continuous media file servers in terms of seek time overhead, and real-time disk scheduling; particularly the QoS negotiation mechanisms they provide are used in a rather static manner that is, these mechanisms are restricted to the evaluation of the capacity of certain system components, e.g. file server a priori known to support a specific quality of service (QoS). In contrast to those approaches, we propose a general QoS negotiation framework that supports the dynamic choice of a configurationmore »« less

Similar Records

Title: dCache: from Resilience to Quality of Service

Abstract

Citation Formats

Data Resilience in the dCache Storage System journal, October 2017

Storage quality-of-service in cloud-based scientific environments: a standardization approach journal, October 2017

Data Resilience in the dCache Storage System
journal, October 2017

Storage quality-of-service in cloud-based scientific environments: a standardization approach
journal, October 2017