Diversity in computing technologies and strategies for dynamic resource allocation

Garzoglio, G.; Gutsche, O.

doi:10.1088/1742-6596/664/1/012001

Title: Diversity in computing technologies and strategies for dynamic resource allocation

Full Record
Other Related Research

Abstract

Here, High Energy Physics (HEP) is a very data intensive and trivially parallelizable science discipline. HEP is probing nature at increasingly finer details requiring ever increasing computational resources to process and analyze experimental data. In this paper, we discuss how HEP provisioned resources so far using Grid technologies, how HEP is starting to include new resource providers like commercial Clouds and HPC installations, and how HEP is transparently provisioning resources at these diverse providers.

Authors:

Garzoglio, G. ^[1]; Gutsche, O. ^[1]

Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Publication Date:: Wed Dec 23 00:00:00 EST 2015

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1250770

Report Number(s):: FERMILAB-CONF-15-443-CD
Journal ID: ISSN 1742-6588; 1413172

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Physics. Conference Series

Additional Journal Information:: Journal Volume: 664; Journal Issue: 1; Conference: 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa (Japan), 13-17 Apr 2015; Journal ID: ISSN 1742-6588

Publisher:: IOP Publishing

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Garzoglio, G., and Gutsche, O. Diversity in computing technologies and strategies for dynamic resource allocation.  United States: N. p., 2015. 
Web.  doi:10.1088/1742-6596/664/1/012001.

Copy to clipboard


                    Garzoglio, G., & Gutsche, O. Diversity in computing technologies and strategies for dynamic resource allocation.  United States.  https://doi.org/10.1088/1742-6596/664/1/012001

Copy to clipboard


                    Garzoglio, G., and Gutsche, O. Wed .  
"Diversity in computing technologies and strategies for dynamic resource allocation".  United States.  https://doi.org/10.1088/1742-6596/664/1/012001.  https://www.osti.gov/servlets/purl/1250770.

Copy to clipboard


                    
@article{osti_1250770,

  title        = {Diversity in computing technologies and strategies for dynamic resource allocation},

  author       = {Garzoglio, G. and Gutsche, O.},

  abstractNote = {Here, High Energy Physics (HEP) is a very data intensive and trivially parallelizable science discipline. HEP is probing nature at increasingly finer details requiring ever increasing computational resources to process and analyze experimental data. In this paper, we discuss how HEP provisioned resources so far using Grid technologies, how HEP is starting to include new resource providers like commercial Clouds and HPC installations, and how HEP is transparently provisioning resources at these diverse providers.},

  doi          = {10.1088/1742-6596/664/1/012001},

  journal      = {Journal of Physics. Conference Series},

  number       = 1,

  volume       = 664,

  place        = {United States},

  year         = {Wed Dec 23 00:00:00 EST 2015},

  month        = {Wed Dec 23 00:00:00 EST 2015}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1088/1742-6596/664/1/012001

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

The HARNESS Workbench: Unified and Adaptive Access to Diverse HPC Platforms

Technical Report Sunderam, Vaidy S

The primary goal of the Harness WorkBench (HWB) project is to investigate innovative software environments that will help enhance the overall productivity of applications science on diverse HPC platforms. Two complementary frameworks were designed: one, a virtualized command toolkit for application building, deployment, and execution, that provides a common view across diverse HPC systems, in particular the DOE leadership computing platforms (Cray, IBM, SGI, and clusters); and two, a unified runtime environment that consolidates access to runtime services via an adaptive framework for execution-time and post processing activities. A prototype of the first was developed based on the concept ofmore »« less
https://doi.org/10.2172/1036700

Full Text Available
Advances in Grid Computing for the FabrIc for Frontier Experiments Project at Fermialb

Conference Herner, K. ; Alba Hernandex, A. F. ; Bhat, S. ; ...

The FabrIc for Frontier Experiments (FIFE) project is a major initiative within the Fermilab Scientic Computing Division charged with leading the computing model for Fermilab experiments. Work within the FIFE project creates close collaboration between experimenters and computing professionals to serve high-energy physics experiments of diering size, scope, and physics area. The FIFE project has worked to develop common tools for job submission, certicate management, software and reference data distribution through CVMFS repositories, robust data transfer, job monitoring, and databases for project tracking. Since the projects inception the experiments under the FIFE umbrella have signicantly matured, and present an increasinglymore »« less
Full Text Available
Virtual machine provisioning, code management, and data movement design for the Fermilab HEPCloud Facility

Journal Article Timm, S. ; Cooper, G. ; Fuess, S. ; ... - Journal of Physics. Conference Series

The Fermilab HEPCloud Facility Project has as its goal to extend the current Fermilab facility interface to provide transparent access to disparate resources including commercial and community clouds, grid federations, and HPC centers. This facility enables experiments to perform the full spectrum of computing tasks, including data-intensive simulation and reconstruction. We have evaluated the use of the commercial cloud to provide elasticity to respond to peaks of demand without overprovisioning local resources. Full scale data-intensive workflows have been successfully completed on Amazon Web Services for two High Energy Physics Experiments, CMS and NOνA, at the scale of 58000 simultaneous cores.more »« less
https://doi.org/10.1088/1742-6596/898/5/052041

Full Text Available
The OSG Open Facility: an on-ramp for opportunistic scientific computing

Journal Article Jayatilaka, B. ; Levshina, T. ; Sehgal, C. ; ... - Journal of Physics. Conference Series

The Open Science Grid (OSG) is a large, robust computing grid that started primarily as a collection of sites associated with large HEP experiments such as ATLAS, CDF, CMS, and DZero, but has evolved in recent years to a much larger user and resource platform. In addition to meeting the US LHC community’s computational needs, the OSG continues to be one of the largest providers of distributed high-throughput computing (DHTC) to researchers from a wide variety of disciplines via the OSG Open Facility. The Open Facility consists of OSG resources that are available opportunistically to users other than resource ownersmore »« less
https://doi.org/10.1088/1742-6596/898/8/082048

Full Text Available
Using the GlideinWMS System as a Common Resource Provisioning Layer in CMS

Conference Balcas, J. ; Belforte, S. ; Bockelman, B. ; ... - J.Phys.Conf.Ser.

CMS will require access to more than 125k processor cores for the beginning of Run 2 in 2015 to carry out its ambitious physics program with more and higher complexity events. During Run1 these resources were predominantly provided by a mix of grid sites and local batch resources. During the long shut down cloud infrastructures, diverse opportunistic resources and HPC supercomputing centers were made available to CMS, which further complicated the operations of the submission infrastructure. In this presentation we will discuss the CMS effort to adopt and deploy the glideinWMS system as a common resource provisioning layer to grid,more »« less
https://doi.org/10.1088/1742-6596/664/6/062031

Full Text Available

Similar Records