skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Understanding the Performance and Potential of Cloud Computing for Scientific Applications

Abstract

In this paper, commercial clouds bring a great opportunity to the scientific computing area. Scientific applications usually require significant resources, however not all scientists have access to sufficient high-end computing systems, may of which can be found in the Top500 list. Cloud Computing has gained the attention of scientists as a competitive resource to run HPC applications at a potentially lower cost. But as a different infrastructure, it is unclear whether clouds are capable of running scientific applications with a reasonable performance per money spent. This work studies the performance of public clouds and places this performance in context to price. We evaluate the raw performance of different services of AWS cloud in terms of the basic resources, such as compute, memory, network and I/O. We also evaluate the performance of the scientific applications running in the cloud. This paper aims to assess the ability of the cloud to perform well, as well as to evaluate the cost of the cloud running scientific applications. We developed a full set of metrics and conducted a comprehensive performance evlauation over the Amazon cloud. We evaluated EC2, S3, EBS and DynamoDB among the many Amazon AWS services. We evaluated the memory sub-system performancemore » with CacheBench, the network performance with iperf, processor and network performance with the HPL benchmark application, and shared storage with NFS and PVFS in addition to S3. We also evaluated a real scientific computing application through the Swift parallel scripting system at scale. Armed with both detailed benchmarks to gauge expected performance and a detailed monetary cost analysis, we expect this paper will be a recipe cookbook for scientists to help them decide where to deploy and run their scientific applications between public clouds, private clouds, or hybrid clouds.« less

Authors:
 [1];  [1];  [1];  [1];  [2];  [3];  [1];  [4];  [4];  [1]
  1. Illinois Institute of Technology, Chicago, IL (United States)
  2. Univ. of Electronic Science and Technology of China, Chengdu (China)
  3. Argonne National Lab. (ANL), Lemont, IL (United States)
  4. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States); Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
OSTI Identifier:
1339973
Report Number(s):
FERMILAB-PUB-14-466-CD
Journal ID: ISSN 2168-7161; 1509983
Grant/Contract Number:  
AC02-07CH11359
Resource Type:
Accepted Manuscript
Journal Name:
IEEE Transactions on Cloud Computing
Additional Journal Information:
Journal Volume: 5; Journal Issue: 2; Journal ID: ISSN 2168-7161
Publisher:
IEEE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; cloud computing; Amazon AWS; performance; cloud costs; scientific computing

Citation Formats

Sadooghi, Iman, Martin, Jesus Hernandez, Li, Tonglin, Brandstatter, Kevin, Zhao, Yong, Maheshwari, Ketan, Pais Pitta de Lacerda Ruivo, Tiago, Timm, Steven, Garzoglio, Gabriele, and Raicu, Ioan. Understanding the Performance and Potential of Cloud Computing for Scientific Applications. United States: N. p., 2015. Web. doi:10.1109/TCC.2015.2404821.
Sadooghi, Iman, Martin, Jesus Hernandez, Li, Tonglin, Brandstatter, Kevin, Zhao, Yong, Maheshwari, Ketan, Pais Pitta de Lacerda Ruivo, Tiago, Timm, Steven, Garzoglio, Gabriele, & Raicu, Ioan. Understanding the Performance and Potential of Cloud Computing for Scientific Applications. United States. doi:10.1109/TCC.2015.2404821.
Sadooghi, Iman, Martin, Jesus Hernandez, Li, Tonglin, Brandstatter, Kevin, Zhao, Yong, Maheshwari, Ketan, Pais Pitta de Lacerda Ruivo, Tiago, Timm, Steven, Garzoglio, Gabriele, and Raicu, Ioan. Thu . "Understanding the Performance and Potential of Cloud Computing for Scientific Applications". United States. doi:10.1109/TCC.2015.2404821. https://www.osti.gov/servlets/purl/1339973.
@article{osti_1339973,
title = {Understanding the Performance and Potential of Cloud Computing for Scientific Applications},
author = {Sadooghi, Iman and Martin, Jesus Hernandez and Li, Tonglin and Brandstatter, Kevin and Zhao, Yong and Maheshwari, Ketan and Pais Pitta de Lacerda Ruivo, Tiago and Timm, Steven and Garzoglio, Gabriele and Raicu, Ioan},
abstractNote = {In this paper, commercial clouds bring a great opportunity to the scientific computing area. Scientific applications usually require significant resources, however not all scientists have access to sufficient high-end computing systems, may of which can be found in the Top500 list. Cloud Computing has gained the attention of scientists as a competitive resource to run HPC applications at a potentially lower cost. But as a different infrastructure, it is unclear whether clouds are capable of running scientific applications with a reasonable performance per money spent. This work studies the performance of public clouds and places this performance in context to price. We evaluate the raw performance of different services of AWS cloud in terms of the basic resources, such as compute, memory, network and I/O. We also evaluate the performance of the scientific applications running in the cloud. This paper aims to assess the ability of the cloud to perform well, as well as to evaluate the cost of the cloud running scientific applications. We developed a full set of metrics and conducted a comprehensive performance evlauation over the Amazon cloud. We evaluated EC2, S3, EBS and DynamoDB among the many Amazon AWS services. We evaluated the memory sub-system performance with CacheBench, the network performance with iperf, processor and network performance with the HPL benchmark application, and shared storage with NFS and PVFS in addition to S3. We also evaluated a real scientific computing application through the Swift parallel scripting system at scale. Armed with both detailed benchmarks to gauge expected performance and a detailed monetary cost analysis, we expect this paper will be a recipe cookbook for scientists to help them decide where to deploy and run their scientific applications between public clouds, private clouds, or hybrid clouds.},
doi = {10.1109/TCC.2015.2404821},
journal = {IEEE Transactions on Cloud Computing},
number = 2,
volume = 5,
place = {United States},
year = {2015},
month = {2}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 8 works
Citation information provided by
Web of Science

Save / Share: