skip to main content

DOE PAGESDOE PAGES

This content will become publicly available on September 29, 2018

Title: HEPCloud, a New Paradigm for HEP Facilities: CMS Amazon Web Services Investigation

Historically, high energy physics computing has been performed on large purpose-built computing systems. These began as single-site compute facilities, but have evolved into the distributed computing grids used today. Recently, there has been an exponential increase in the capacity and capability of commercial clouds. Cloud resources are highly virtualized and intended to be able to be flexibly deployed for a variety of computing tasks. There is a growing interest among the cloud providers to demonstrate the capability to perform large-scale scientific computing. In this paper, we discuss results from the CMS experiment using the Fermilab HEPCloud facility, which utilized both local Fermilab resources and virtual machines in the Amazon Web Services Elastic Compute Cloud. We discuss the planning, technical challenges, and lessons learned involved in performing physics workflows on a large-scale set of virtualized resources. Additionally, we will discuss the economics and operational efficiencies when executing workflows both in the cloud and on dedicated resources.
Authors:
ORCiD logo [1] ; ORCiD logo [1] ; ORCiD logo [2] ; ORCiD logo [1] ;  [3] ;  [1] ;  [1] ;  [4] ; ORCiD logo [1] ; ORCiD logo [1] ;  [1] ;  [1] ;  [1] ;  [1] ;  [1] ;  [1] ; ORCiD logo [1] ; ORCiD logo [1]
  1. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
  2. Univ. of Nebraska, Lincoln, NE (United States)
  3. Simons Foundation, New York, NY (United States)
  4. European Organization for Nuclear Research (CERN), Geneva (Switzerland)
Publication Date:
Report Number(s):
arXiv:1710.00100; FERMILAB-PUB-17-092-CD
Journal ID: ISSN 2510-2036; 1628463; TRN: US1801245
Grant/Contract Number:
AC02-07CH11359
Type:
Accepted Manuscript
Journal Name:
Computing and Software for Big Science
Additional Journal Information:
Journal Volume: 1; Journal Issue: 1; Journal ID: ISSN 2510-2036
Publisher:
Springer
Research Org:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS; High energy physics; Computing; Cloud; Amazon web services
OSTI Identifier:
1418149