Cloud services for the Fermilab scientific stakeholders

Journal Article · Wed Dec 23 00:00:00 EST 2015 · Journal of Physics. Conference Series

DOI:https://doi.org/10.1088/1742-6596/664/2/022039· OSTI ID:1250521

Timm, S. ^[1]; Garzoglio, G. ^[1]; Mhashilkar, P. ^[1]; Boyd, J. ^[1]; Bernabeu, G. ^[1]; Sharma, N. ^[1]; Peregonow, N. ^[1]; Kim, H. ^[1]; Noh, S. ^[2]; Palur, S. ^[3]; Raicu, I. ^[3]

---

1. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
2. Korea Institute of Science and Technology Information, Daejeon (South Korea)
3. Illinois Inst. of Technology, Chicago, IL (United States)

As part of the Fermilab/KISTI cooperative research project, Fermilab has successfully run an experimental simulation workflow at scale on a federation of Amazon Web Services (AWS), FermiCloud, and local FermiGrid resources. We used the CernVM-FS (CVMFS) file system to deliver the application software. We established Squid caching servers in AWS as well, using the Shoal system to let each individual virtual machine find the closest squid server. We also developed an automatic virtual machine conversion system so that we could transition virtual machines made on FermiCloud to Amazon Web Services. We used this system to successfully run a cosmic ray simulation of the NOvA detector at Fermilab, making use of both AWS spot pricing and network bandwidth discounts to minimize the cost. On FermiCloud we also were able to run the workflow at the scale of 1000 virtual machines, using a private network routable inside of Fermilab. As a result, we present in detail the technological improvements that were used to make this work a reality.

• Research Organization: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), High Energy Physics (HEP)
• Grant/Contract Number: AC02-07CH11359
• OSTI ID: 1250521
• Report Number(s): FERMILAB-CONF-15-194-CD; 1407404
• Journal Information: Journal of Physics. Conference Series, Vol. 664, Issue 2; Conference: 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa (Japan), 13-17 Apr 2015; ISSN 1742-6588
• Publisher: IOP Publishing
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING