I/O Performance of Virtualized Cloud Environments
The scientific community is exploring the suitability of cloud infrastructure to handle High Performance Computing (HPC) applications. The goal of Magellan, a project funded through DOE ASCR, is to investigate the potential role of cloud computing to address the computing needs of the Department of Energy?s Office of Science, especially for mid-range computing and data-intensive applications which are not served through existing DOE centers today. Prior work has shown that applications with significant communication orI/O tend to perform poorly in virtualized cloud environments. However, there is a limited understanding of the I/O characteristics in virtualized cloud environments. This paper will present our results in benchmarking the I/O performance over different cloud and HPC platforms to identify the major bottlenecks in existing infrastructure. We compare the I/O performance using IOR benchmark on two cloud platforms - Amazon and Magellan. We analyze the performance of different storage options available, different instance types in multiple availability zones. Finally, we perform large-scale tests in order to analyze the variability in the I/O patterns over time and region. Our results highlight the overhead and variability in I/O performance on both public and private cloud solutions. Our results will help applications decide between the different storage options enabling applications to make effective choices.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- Computing Sciences Directorate
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 1093574
- Report Number(s):
- LBNL-5432E
- Resource Relation:
- Conference: SC'11 High Performance Computing Networking, Storage and Analysis, Seattle, WA, November 12 - 18, 2011
- Country of Publication:
- United States
- Language:
- English
Similar Records
Distributed Logical Analytic Domains (DLADs) for Distributed HPC Security
Development of the Breakthrough Vorcat Technology for Cloud-Based Complex Energy Simulations