skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Stochastic Programming Approach for Resource Selection under Demand Uncertainty

Abstract

Cost-efficient selection and scheduling of a subset of geographically distributed resources to meet the demands of a scientific workflow is a challenging problem. The problem is exacerbated by uncertainties in demand and availability of resources. In this paper, we present a stochastic optimization based framework for robust decision making in the selection of distributed resources over a planning horizon under demand uncertainty. We present a novel two-stage stochastic programming model for resource selection, and implement an L-shaped decomposition algorithm to solve this model. A Sample Average Approximation algorithm is integrated to enable stochastic optimization to solve problems with a large number of scenarios. Using the metric of stochastic solution, we demonstrate up to 30% cost reduction relative to solutions without explicit consideration of demand uncertainty for a 24-month problem. We also demonstrate up to 54% cost reduction relative to a previously developed solution for a 36-month problem. We further argue that the composition of resources selected is superior to solutions computed without explicit consideration of uncertainties. Given the importance of resource selection and scheduling of complex scientific workflows, especially in the context of commercial cloud computing, we believe that our novel stochastic programming framework will benefit many researchers as wellmore » as users of distributed computing resources.« less

Authors:
 [1]; ORCiD logo [2]; ORCiD logo [2];  [1];  [3];  [2];  [2]
  1. Mississippi State University
  2. BATTELLE (PACIFIC NW LAB)
  3. Universidad Autonoma Metropolitana
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1526307
Report Number(s):
PNNL-SA-130071
Journal ID: ISSN 0302--9743
DOE Contract Number:  
AC05-76RL01830
Resource Type:
Conference
Resource Relation:
Journal Volume: 11332; Conference: 22nd International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 2018), May 25, 2018, Vancouver, BC, Lecture Notes in Computer Science
Country of Publication:
Switzerland
Language:
English
Subject:
scientific workflows, task scheduling, optimization algorithms, Stochastic Programming, Uncertainties

Citation Formats

Bhuiyan, Tanveer H., Halappanavar, Mahantesh, Friese, Ryan D., Medal, Hugh, De La Torre, Luis, Visweswara Sathanur, Arun, and Tallent, Nathan R. Stochastic Programming Approach for Resource Selection under Demand Uncertainty. Switzerland: N. p., 2018. Web. doi:10.1007/978-3-030-10632-4_6.
Bhuiyan, Tanveer H., Halappanavar, Mahantesh, Friese, Ryan D., Medal, Hugh, De La Torre, Luis, Visweswara Sathanur, Arun, & Tallent, Nathan R. Stochastic Programming Approach for Resource Selection under Demand Uncertainty. Switzerland. doi:10.1007/978-3-030-10632-4_6.
Bhuiyan, Tanveer H., Halappanavar, Mahantesh, Friese, Ryan D., Medal, Hugh, De La Torre, Luis, Visweswara Sathanur, Arun, and Tallent, Nathan R. Fri . "Stochastic Programming Approach for Resource Selection under Demand Uncertainty". Switzerland. doi:10.1007/978-3-030-10632-4_6.
@article{osti_1526307,
title = {Stochastic Programming Approach for Resource Selection under Demand Uncertainty},
author = {Bhuiyan, Tanveer H. and Halappanavar, Mahantesh and Friese, Ryan D. and Medal, Hugh and De La Torre, Luis and Visweswara Sathanur, Arun and Tallent, Nathan R.},
abstractNote = {Cost-efficient selection and scheduling of a subset of geographically distributed resources to meet the demands of a scientific workflow is a challenging problem. The problem is exacerbated by uncertainties in demand and availability of resources. In this paper, we present a stochastic optimization based framework for robust decision making in the selection of distributed resources over a planning horizon under demand uncertainty. We present a novel two-stage stochastic programming model for resource selection, and implement an L-shaped decomposition algorithm to solve this model. A Sample Average Approximation algorithm is integrated to enable stochastic optimization to solve problems with a large number of scenarios. Using the metric of stochastic solution, we demonstrate up to 30% cost reduction relative to solutions without explicit consideration of demand uncertainty for a 24-month problem. We also demonstrate up to 54% cost reduction relative to a previously developed solution for a 36-month problem. We further argue that the composition of resources selected is superior to solutions computed without explicit consideration of uncertainties. Given the importance of resource selection and scheduling of complex scientific workflows, especially in the context of commercial cloud computing, we believe that our novel stochastic programming framework will benefit many researchers as well as users of distributed computing resources.},
doi = {10.1007/978-3-030-10632-4_6},
journal = {},
issn = {0302--9743},
number = ,
volume = 11332,
place = {Switzerland},
year = {2018},
month = {5}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: