skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Experience in using commercial clouds in CMS

Abstract

Historically high energy physics computing has been performed on large purposebuilt computing systems. In the beginning there were single site computing facilities, which evolved into the Worldwide LHC Computing Grid (WLCG) used today. The vast majority of the WLCG resources are used for LHC computing and the resources are scheduled to be continuously used throughout the year. In the last several years there has been an explosion in capacity and capability of commercial and academic computing clouds. Cloud resources are highly virtualized and intended to be able to be flexibly deployed for a variety of computing tasks. There is a growing interest amongst the cloud providers to demonstrate the capability to perform large scale scientific computing. In this presentation we will discuss results from the CMS experiment using the Fermilab HEPCloud Facility, which utilized both local Fermilab resources and Amazon Web Services (AWS). The goal was to work with AWS through a matching grant to demonstrate a sustained scale approximately equal to half of the worldwide processing resources available to CMS. We will discuss the planning and technical challenges involved in organizing the most IO intensive CMS workflows on a large-scale set of virtualized resource provisioned by the Fermilab HEPCloud.more » We will describe the data handling and data management challenges. Also, we will discuss the economic issues and cost and operational efficiency comparison to our dedicated resources. At the end we will consider the changes in the working model of HEP computing in a domain with the availability of large scale resources scheduled at peak times.« less

Authors:
 [1];  [2];  [1];  [1];  [1];  [3];  [1];  [1];  [1];  [1];  [1];  [1];  [1];  [1];  [1];  [1]
  1. Fermilab
  2. Nebraska U.
  3. CERN
Publication Date:
Research Org.:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
Contributing Org.:
CMS
OSTI Identifier:
1402473
Report Number(s):
FERMILAB-CONF-17-405-CD
1630322
DOE Contract Number:  
AC02-07CH11359
Resource Type:
Conference
Country of Publication:
United States
Language:
English

Citation Formats

Bauerdick, L., Bockelman, B., Dykstra, D., Fuess, S., Garzoglio, G., Girone, M., Gutsche, O., Holzman, B., Hugnagel, D., Kim, H., Kennedy, R., Mason, D., Spentzouris, P., Timm, S., Tiradani, A., and Vaandering, E. Experience in using commercial clouds in CMS. United States: N. p., 2017. Web.
Bauerdick, L., Bockelman, B., Dykstra, D., Fuess, S., Garzoglio, G., Girone, M., Gutsche, O., Holzman, B., Hugnagel, D., Kim, H., Kennedy, R., Mason, D., Spentzouris, P., Timm, S., Tiradani, A., & Vaandering, E. Experience in using commercial clouds in CMS. United States.
Bauerdick, L., Bockelman, B., Dykstra, D., Fuess, S., Garzoglio, G., Girone, M., Gutsche, O., Holzman, B., Hugnagel, D., Kim, H., Kennedy, R., Mason, D., Spentzouris, P., Timm, S., Tiradani, A., and Vaandering, E. Tue . "Experience in using commercial clouds in CMS". United States. doi:. https://www.osti.gov/servlets/purl/1402473.
@article{osti_1402473,
title = {Experience in using commercial clouds in CMS},
author = {Bauerdick, L. and Bockelman, B. and Dykstra, D. and Fuess, S. and Garzoglio, G. and Girone, M. and Gutsche, O. and Holzman, B. and Hugnagel, D. and Kim, H. and Kennedy, R. and Mason, D. and Spentzouris, P. and Timm, S. and Tiradani, A. and Vaandering, E.},
abstractNote = {Historically high energy physics computing has been performed on large purposebuilt computing systems. In the beginning there were single site computing facilities, which evolved into the Worldwide LHC Computing Grid (WLCG) used today. The vast majority of the WLCG resources are used for LHC computing and the resources are scheduled to be continuously used throughout the year. In the last several years there has been an explosion in capacity and capability of commercial and academic computing clouds. Cloud resources are highly virtualized and intended to be able to be flexibly deployed for a variety of computing tasks. There is a growing interest amongst the cloud providers to demonstrate the capability to perform large scale scientific computing. In this presentation we will discuss results from the CMS experiment using the Fermilab HEPCloud Facility, which utilized both local Fermilab resources and Amazon Web Services (AWS). The goal was to work with AWS through a matching grant to demonstrate a sustained scale approximately equal to half of the worldwide processing resources available to CMS. We will discuss the planning and technical challenges involved in organizing the most IO intensive CMS workflows on a large-scale set of virtualized resource provisioned by the Fermilab HEPCloud. We will describe the data handling and data management challenges. Also, we will discuss the economic issues and cost and operational efficiency comparison to our dedicated resources. At the end we will consider the changes in the working model of HEP computing in a domain with the availability of large scale resources scheduled at peak times.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Oct 03 00:00:00 EDT 2017},
month = {Tue Oct 03 00:00:00 EDT 2017}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: