Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Packing the Most Onto Your Cloud Ashraf Aboulnaga Ziyu Wang Zi Ye Zhang

Summary: Packing the Most Onto Your Cloud
Ashraf Aboulnaga Ziyu Wang Zi Ye Zhang
David R. Cheriton School of Computer Science
University of Waterloo
Waterloo, Ontario, Canada
{ashraf, z8wang, zy2zhang}@cs.uwaterloo.ca
Parallel dataflow programming frameworks such as Map-Reduce
are increasingly being used for large scale data analysis on com-
puting clouds. It is therefore becoming important to automatically
optimize the performance of these frameworks. In this paper, we
deal with one particular optimization problem, namely scheduling
sets of Map-Reduce jobs on a cluster of machines. We present
a scheduler that takes job characteristics into account and finds
a schedule that minimizes the total completion time of the set of
jobs. Our scheduler decides on the number of machines to assign
to each job, and it tries to pack as many jobs on the machines as
the machine resources can support. To enable flexible assignment
of jobs onto machines, we run the Map-Reduce jobs in virtual ma-
chines. Our scheduling problem is formulated as a constrained op-


Source: Aboulnaga, Ashraf - School of Computer Science, University of Waterloo


Collections: Computer Technologies and Information Sciences