Improving load balance with flexibly assignable tasks
In many applications of parallel computing, distribution ofthe data unambiguously implies distribution of work among processors. Butthere are exceptions where some tasks can be assigned to one of severalprocessors without altering the total volume of communication. In thispaper, we study the problem of exploiting this flexibility in assignmentof tasks to improve load balance. We first model the problem in terms ofnetwork flow and use combinatorial techniques for its solution. Ourparametric search algorithms use maximum flow algorithms for probing on acandidate optimal solution value. We describe two algorithms to solve theassignment problem with \logW_T and vbar P vbar probe calls, w here W_Tand vbar P vbar, respectively, denote the total workload and number ofproce ssors. We also define augmenting paths and cuts for this problem,and show that anyalgorithm based on augmenting paths can be used to findan optimal solution for the task assignment problem. We then consideracontinuous version of the problem, and formulate it as a linearlyconstrained optimization problem, i.e., \min\|Ax\|_\infty,\; {\rms.t.}\;Bx=d. To avoid solving an intractable \infty-norm optimization problem,we show that in this case minimizing the 2-norm is sufficient to minimizethe \infty-norm, which reduces the problem to the well-studiedlinearly-constrained least squares problem. The continuous version of theproblem has the advantage of being easily amenable to parallelization.Our experiments with molecular dynamics and overlapped domaindecomposition applications proved the effectiveness of our methods withsignificant improvements in load balance. We also discuss how ourtechniques can be enhanced for heterogeneous systems.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- USDOE Director. Office of Science. Office of Computationaland Technology Research. Mathematical Information and ComputatiuonalSciences Division, Applied Mathematical Sciences, Sandia Corporation.Lockheed-Martin Company Contract DE-AC94-AL85000
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 860216
- Report Number(s):
- LBNL-53758; R&D Project: 365968; BnR: YN0100000
- Journal Information:
- IEEE Transactions on Parallel and DistributedSystems, Vol. 16, Issue 10; Related Information: Journal Publication Date: 10/2005
- Country of Publication:
- United States
- Language:
- English
Similar Records
LEISURE: Load-Balanced Network-Wide Traffic Measurement and Monitor Placement
Parallel shortest augmenting path algorithm for the assignment problem. Technical report