skip to main content

Title: Cloud services for the Fermilab scientific stakeholders

As part of the Fermilab/KISTI cooperative research project, Fermilab has successfully run an experimental simulation workflow at scale on a federation of Amazon Web Services (AWS), FermiCloud, and local FermiGrid resources. We used the CernVM-FS (CVMFS) file system to deliver the application software. We established Squid caching servers in AWS as well, using the Shoal system to let each individual virtual machine find the closest squid server. We also developed an automatic virtual machine conversion system so that we could transition virtual machines made on FermiCloud to Amazon Web Services. We used this system to successfully run a cosmic ray simulation of the NOvA detector at Fermilab, making use of both AWS spot pricing and network bandwidth discounts to minimize the cost. On FermiCloud we also were able to run the workflow at the scale of 1000 virtual machines, using a private network routable inside of Fermilab. As a result, we present in detail the technological improvements that were used to make this work a reality.
 [1] ;  [1] ;  [1] ;  [1] ;  [1] ;  [1] ;  [1] ;  [1] ;  [2] ;  [3] ;  [3]
  1. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
  2. Korea Institute of Science and Technology Information, Daejeon (South Korea)
  3. Illinois Inst. of Technology, Chicago, IL (United States)
Publication Date:
OSTI Identifier:
Report Number(s):
Journal ID: ISSN 1742-6588; 1407404
Grant/Contract Number:
Accepted Manuscript
Journal Name:
Journal of Physics. Conference Series
Additional Journal Information:
Journal Volume: 664; Journal Issue: 2; Conference: 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa (Japan), 13-17 Apr 2015; Journal ID: ISSN 1742-6588
IOP Publishing
Research Org:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
Country of Publication:
United States