skip to main content

DOE PAGESDOE PAGES

Title: Integration of end-user Cloud storage for CMS analysis

End-user Cloud storage is increasing rapidly in popularity in research communities thanks to the collaboration capabilities it offers, namely synchronisation and sharing. CERN IT has implemented a model of such storage named, CERNBox, integrated with the CERN AuthN and AuthZ services. To exploit the use of the end-user Cloud storage for the distributed data analysis activity, the CMS experiment has started the integration of CERNBox as a Grid resource. This will allow CMS users to make use of their own storage in the Cloud for their analysis activities as well as to benefit from synchronisation and sharing capabilities to achieve results faster and more effectively. It will provide an integration model of Cloud storages in the Grid, which is implemented and commissioned over the world’s largest computing Grid infrastructure, Worldwide LHC Computing Grid (WLCG). In this paper, we present the integration strategy and infrastructure changes needed in order to transparently integrate end-user Cloud storage with the CMS distributed computing model. We describe the new challenges faced in data management between Grid and Cloud and how they were addressed, along with details of the support for Cloud storage recently introduced into the WLCG data movement middleware, FTS3. Finally, the commissioning experiencemore » of CERNBox for the distributed data analysis activity is also presented.« less
Authors:
 [1] ;  [1] ;  [1] ;  [2] ;  [3] ;  [4] ;  [1] ;  [5] ;  [1] ;  [1] ;  [5] ;  [6] ;  [5]
  1. European Organization for Nuclear Research, Geneva (Switzerland)
  2. California Inst. of Technology (CalTech), Pasadena, CA (United States)
  3. Univ. and INFN Perugia, Perugia (Italy)
  4. CIEMAT, Madrid (Spain)
  5. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
  6. Univ. of California, San Diego, La Jolla, CA (United States)
Publication Date:
Report Number(s):
FERMILAB-PUB-17-650
Journal ID: ISSN 0167-739X; 1653645; TRN: US1801487
Grant/Contract Number:
AC02-07CH11359
Type:
Accepted Manuscript
Journal Name:
Future Generations Computer Systems
Additional Journal Information:
Journal Volume: 78; Journal Issue: P3; Journal ID: ISSN 0167-739X
Publisher:
Elsevier
Research Org:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Data transfer protocols; Cloud storage; Distributed data management; Distributed data analysis
OSTI Identifier:
1420408