Cloud services for the Fermilab scientific stakeholders

Timm, S.; Garzoglio, G.; Mhashilkar, P.; Boyd, J.; Bernabeu, G.; Sharma, N.; Peregonow, N.; Kim, H.; Noh, S.; Palur, S.; Raicu, I.

doi:10.1088/1742-6596/664/2/022039

Title: Cloud services for the Fermilab scientific stakeholders

Full Record
Other Related Research

Abstract

As part of the Fermilab/KISTI cooperative research project, Fermilab has successfully run an experimental simulation workflow at scale on a federation of Amazon Web Services (AWS), FermiCloud, and local FermiGrid resources. We used the CernVM-FS (CVMFS) file system to deliver the application software. We established Squid caching servers in AWS as well, using the Shoal system to let each individual virtual machine find the closest squid server. We also developed an automatic virtual machine conversion system so that we could transition virtual machines made on FermiCloud to Amazon Web Services. We used this system to successfully run a cosmic ray simulation of the NOvA detector at Fermilab, making use of both AWS spot pricing and network bandwidth discounts to minimize the cost. On FermiCloud we also were able to run the workflow at the scale of 1000 virtual machines, using a private network routable inside of Fermilab. As a result, we present in detail the technological improvements that were used to make this work a reality.

Authors:

Timm, S. ^[1]; Garzoglio, G. ^[1]; Mhashilkar, P. ^[1]; Boyd, J. ^[1]; Bernabeu, G. ^[1]; Sharma, N. ^[1]; Peregonow, N. ^[1]; Kim, H. ^[1]; Noh, S. ^[2]; Palur, S. ^[3]; Raicu, I. ^[3]

Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Korea Institute of Science and Technology Information, Daejeon (South Korea)
Illinois Inst. of Technology, Chicago, IL (United States)

Publication Date:: Wed Dec 23 00:00:00 EST 2015

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1250521

Report Number(s):: FERMILAB-CONF-15-194-CD
Journal ID: ISSN 1742-6588; 1407404

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Physics. Conference Series

Additional Journal Information:: Journal Volume: 664; Journal Issue: 2; Conference: 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa (Japan), 13-17 Apr 2015; Journal ID: ISSN 1742-6588

Publisher:: IOP Publishing

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Timm, S., Garzoglio, G., Mhashilkar, P., Boyd, J., Bernabeu, G., Sharma, N., Peregonow, N., Kim, H., Noh, S., Palur, S., and Raicu, I. Cloud services for the Fermilab scientific stakeholders.  United States: N. p., 2015. 
Web.  doi:10.1088/1742-6596/664/2/022039.

Copy to clipboard


                    Timm, S., Garzoglio, G., Mhashilkar, P., Boyd, J., Bernabeu, G., Sharma, N., Peregonow, N., Kim, H., Noh, S., Palur, S., & Raicu, I. Cloud services for the Fermilab scientific stakeholders.  United States.  https://doi.org/10.1088/1742-6596/664/2/022039

Copy to clipboard


                    Timm, S., Garzoglio, G., Mhashilkar, P., Boyd, J., Bernabeu, G., Sharma, N., Peregonow, N., Kim, H., Noh, S., Palur, S., and Raicu, I. Wed .  
"Cloud services for the Fermilab scientific stakeholders".  United States.  https://doi.org/10.1088/1742-6596/664/2/022039.  https://www.osti.gov/servlets/purl/1250521.

Copy to clipboard


                    
@article{osti_1250521,

  title        = {Cloud services for the Fermilab scientific stakeholders},

  author       = {Timm, S. and Garzoglio, G. and Mhashilkar, P. and Boyd, J. and Bernabeu, G. and Sharma, N. and Peregonow, N. and Kim, H. and Noh, S. and Palur, S. and Raicu, I.},

  abstractNote = {As part of the Fermilab/KISTI cooperative research project, Fermilab has successfully run an experimental simulation workflow at scale on a federation of Amazon Web Services (AWS), FermiCloud, and local FermiGrid resources. We used the CernVM-FS (CVMFS) file system to deliver the application software. We established Squid caching servers in AWS as well, using the Shoal system to let each individual virtual machine find the closest squid server. We also developed an automatic virtual machine conversion system so that we could transition virtual machines made on FermiCloud to Amazon Web Services. We used this system to successfully run a cosmic ray simulation of the NOvA detector at Fermilab, making use of both AWS spot pricing and network bandwidth discounts to minimize the cost. On FermiCloud we also were able to run the workflow at the scale of 1000 virtual machines, using a private network routable inside of Fermilab. As a result, we present in detail the technological improvements that were used to make this work a reality.},

  doi          = {10.1088/1742-6596/664/2/022039},

  journal      = {Journal of Physics. Conference Series},

  number       = 2,

  volume       = 664,

  place        = {United States},

  year         = {Wed Dec 23 00:00:00 EST 2015},

  month        = {Wed Dec 23 00:00:00 EST 2015}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1088/1742-6596/664/2/022039

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 2 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Virtual machine provisioning, code management, and data movement design for the Fermilab HEPCloud Facility

Journal Article Timm, S. ; Cooper, G. ; Fuess, S. ; ... - Journal of Physics. Conference Series

The Fermilab HEPCloud Facility Project has as its goal to extend the current Fermilab facility interface to provide transparent access to disparate resources including commercial and community clouds, grid federations, and HPC centers. This facility enables experiments to perform the full spectrum of computing tasks, including data-intensive simulation and reconstruction. We have evaluated the use of the commercial cloud to provide elasticity to respond to peaks of demand without overprovisioning local resources. Full scale data-intensive workflows have been successfully completed on Amazon Web Services for two High Energy Physics Experiments, CMS and NOνA, at the scale of 58000 simultaneous cores.more »« less
https://doi.org/10.1088/1742-6596/898/5/052041

Full Text Available
Web Proxy Auto Discovery for the WLCG

Journal Article Dykstra, D. ; Blomer, J. ; Blumenfeld, B. ; ... - Journal of Physics. Conference Series

All four of the LHC experiments depend on web proxies (that is, squids) at each grid site to support software distribution by the CernVM FileSystem (CVMFS). CMS and ATLAS also use web proxies for conditions data distributed through the Frontier Distributed Database caching system. ATLAS & CMS each have their own methods for their grid jobs to find out which web proxies to use for Frontier at each site, and CVMFS has a third method. Those diverse methods limit usability and flexibility, particularly for opportunistic use cases, where an experiment’s jobs are run at sites that do not primarily supportmore »« less
https://doi.org/10.1088/1742-6596/898/5/052043

Full Text Available
Engineering the CernVM-Filesystem as a High Bandwidth Distributed Filesystem for Auxiliary Physics Data

Conference Dykstra, D. ; Bockelman, B. ; Blomer, J. ; ... - J.Phys.Conf.Ser.

A common use pattern in the computing models of particle physics experiments is running many distributed applications that read from a shared set of data files. We refer to this data is auxiliary data, to distinguish it from (a) event data from the detector (which tends to be different for every job), and (b) conditions data about the detector (which tends to be the same for each job in a batch of jobs). Relatively speaking, conditions data also tends to be relatively small per job where both event data and auxiliary data are larger per job. Unlike event data, auxiliarymore »« less
https://doi.org/10.1088/1742-6596/664/4/042012

Full Text Available
Virtual Facility at Fermilab: Infrastructure and Services Expand to Public Clouds

Journal Article Timm, Steve ; Garzoglio, Gabriele ; Cooper, Glenn ; ... - PoS Proceedings of Science

In preparation for its new Virtual Facility Project, Fermilab has launched a program of work to determine the requirements for running a computation facility on-site, in public clouds, or a combination of both. This program builds on the work we have done to successfully run experimental workflows of 1000-VM scale both on an on-site private cloud and on Amazon AWS. To do this at scale we deployed dynamically launched and discovered caching services on the cloud. We are now testing the deployment of more complicated services on Amazon AWS using native load balancing and auto scaling features they provide. Themore »« less
Full Text Available
Grids, virtualization, and clouds at Fermilab

Journal Article Timm, S. ; Chadwick, K. ; Garzoglio, G. ; ... - Journal of Physics. Conference Series

Fermilab supports a scientific program that includes experiments and scientists located across the globe. To better serve this community, in 2004, the (then) Computing Division undertook the strategy of placing all of the High Throughput Computing (HTC) resources in a Campus Grid known as FermiGrid, supported by common shared services. In 2007, the FermiGrid Services group deployed a service infrastructure that utilized Xen virtualization, LVS network routing and MySQL circular replication to deliver highly available services that offered significant performance, reliability and serviceability improvements. This deployment was further enhanced through the deployment of a distributed redundant network core architecture andmore »« less
Cited by 2
https://doi.org/10.1088/1742-6596/513/3/032037

Full Text Available

Similar Records