Integration of end-user Cloud storage for CMS analysis

Journal Article · Fri May 19 00:00:00 EDT 2017 · Future Generations Computer Systems

DOI:https://doi.org/10.1016/j.future.2017.04.021· OSTI ID:1420408

Riahi, Hassen ^[1]; Aimar, Alberto ^[1]; Ayllon, Alejandro Alvarez ^[1]; Balcas, Justas ^[2]; Ciangottini, Diego ^[3]; Hernandez, Jose M. ^[4]; Keeble, Oliver ^[1]; Magini, Nicolo ^[5]; Manzi, Andrea ^[1]; Mascetti, Luca ^[1]; Mascheroni, Marco ^[5]; Tanasijczuk, Andres Jorge ^[6]; Vaandering, Eric Wayne ^[5]

---

1. European Organization for Nuclear Research, Geneva (Switzerland)
2. California Inst. of Technology (CalTech), Pasadena, CA (United States)
3. Univ. and INFN Perugia, Perugia (Italy)
4. CIEMAT, Madrid (Spain)
5. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
6. Univ. of California, San Diego, La Jolla, CA (United States)

End-user Cloud storage is increasing rapidly in popularity in research communities thanks to the collaboration capabilities it offers, namely synchronisation and sharing. CERN IT has implemented a model of such storage named, CERNBox, integrated with the CERN AuthN and AuthZ services. To exploit the use of the end-user Cloud storage for the distributed data analysis activity, the CMS experiment has started the integration of CERNBox as a Grid resource. This will allow CMS users to make use of their own storage in the Cloud for their analysis activities as well as to benefit from synchronisation and sharing capabilities to achieve results faster and more effectively. It will provide an integration model of Cloud storages in the Grid, which is implemented and commissioned over the world’s largest computing Grid infrastructure, Worldwide LHC Computing Grid (WLCG). In this paper, we present the integration strategy and infrastructure changes needed in order to transparently integrate end-user Cloud storage with the CMS distributed computing model. We describe the new challenges faced in data management between Grid and Cloud and how they were addressed, along with details of the support for Cloud storage recently introduced into the WLCG data movement middleware, FTS3. Finally, the commissioning experience of CERNBox for the distributed data analysis activity is also presented.

• Research Organization: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), High Energy Physics (HEP)
• Grant/Contract Number: AC02-07CH11359
• OSTI ID: 1420408
• Report Number(s): FERMILAB-PUB-17-650; 1653645; TRN: US1801487
• Journal Information: Future Generations Computer Systems, Vol. 78, Issue P3; ISSN 0167-739X
• Publisher: Elsevier
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
Data transfer protocols
Cloud storage
Distributed data management
Distributed data analysis