skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Scheduling Multilevel Deadline-Constrained Scientific Workflows on Clouds Based on Cost Optimization

Abstract

This paper presents a cost optimization model for scheduling scientific workflows on IaaS clouds such as Amazon EC2 or RackSpace. We assume multiple IaaS clouds with heterogeneous virtual machine instances, with limited number of instances per cloud and hourly billing. Input and output data are stored on a cloud object store such as Amazon S3. Applications are scientific workflows modeled as DAGs as in the Pegasus Workflow Management System. We assume that tasks in the workflows are grouped into levels of identical tasks. Our model is specified using mathematical programming languages (AMPL and CMPL) and allows us to minimize the cost of workflow execution under deadline constraints. We present results obtained using our model and the benchmark workflows representing real scientific applications in a variety of domains. The data used for evaluation come from the synthetic workflows and from general purpose cloud benchmarks, as well as from the data measured in our own experiments with Montage, an astronomical application, executed on Amazon EC2 cloud. We indicate how this model can be used for scenarios that require resource planning for scientific workflows and their ensembles.

Authors:
 [1];  [1];  [2];  [3];  [4]
  1. Department of Computer Science, AGH University of Science and Technology, Aleja Mickiewicza 30, 30-059 Kraków, Poland
  2. Department of Computer Science, AGH University of Science and Technology, Aleja Mickiewicza 30, 30-059 Kraków, Poland, ACC CYFRONET AGH, Ulica Nawojki 11, 30-950 Kraków, Poland
  3. USC Information Sciences Institute, 4676 Admiralty Way, Marina del Rey, CA 90292, USA
  4. Center for Research Computing, University of Notre Dame, Notre Dame, IN 46556, USA
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1228415
Grant/Contract Number:  
ER26110
Resource Type:
Published Article
Journal Name:
Scientific Programming
Additional Journal Information:
Journal Name: Scientific Programming Journal Volume: 2015; Journal ID: ISSN 1058-9244
Publisher:
Hindawi Publishing Corporation
Country of Publication:
Egypt
Language:
English

Citation Formats

Malawski, Maciej, Figiela, Kamil, Bubak, Marian, Deelman, Ewa, and Nabrzyski, Jarek. Scheduling Multilevel Deadline-Constrained Scientific Workflows on Clouds Based on Cost Optimization. Egypt: N. p., 2015. Web. doi:10.1155/2015/680271.
Malawski, Maciej, Figiela, Kamil, Bubak, Marian, Deelman, Ewa, & Nabrzyski, Jarek. Scheduling Multilevel Deadline-Constrained Scientific Workflows on Clouds Based on Cost Optimization. Egypt. doi:10.1155/2015/680271.
Malawski, Maciej, Figiela, Kamil, Bubak, Marian, Deelman, Ewa, and Nabrzyski, Jarek. Thu . "Scheduling Multilevel Deadline-Constrained Scientific Workflows on Clouds Based on Cost Optimization". Egypt. doi:10.1155/2015/680271.
@article{osti_1228415,
title = {Scheduling Multilevel Deadline-Constrained Scientific Workflows on Clouds Based on Cost Optimization},
author = {Malawski, Maciej and Figiela, Kamil and Bubak, Marian and Deelman, Ewa and Nabrzyski, Jarek},
abstractNote = {This paper presents a cost optimization model for scheduling scientific workflows on IaaS clouds such as Amazon EC2 or RackSpace. We assume multiple IaaS clouds with heterogeneous virtual machine instances, with limited number of instances per cloud and hourly billing. Input and output data are stored on a cloud object store such as Amazon S3. Applications are scientific workflows modeled as DAGs as in the Pegasus Workflow Management System. We assume that tasks in the workflows are grouped into levels of identical tasks. Our model is specified using mathematical programming languages (AMPL and CMPL) and allows us to minimize the cost of workflow execution under deadline constraints. We present results obtained using our model and the benchmark workflows representing real scientific applications in a variety of domains. The data used for evaluation come from the synthetic workflows and from general purpose cloud benchmarks, as well as from the data measured in our own experiments with Montage, an astronomical application, executed on Amazon EC2 cloud. We indicate how this model can be used for scenarios that require resource planning for scientific workflows and their ensembles.},
doi = {10.1155/2015/680271},
journal = {Scientific Programming},
number = ,
volume = 2015,
place = {Egypt},
year = {2015},
month = {1}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.1155/2015/680271

Save / Share: