Models for parallel computation on distributed-memory multiprocessors
The most severe shortcoming of distributed-memory multiprocessors is the communication overhead. Due to the distributed nature of such systems, data must be moved explicit among the processors, including the host, so as to carry out a computing job. Efficient algorithm design becomes crucial in fully utilizing the computing power of the multiprocessors. In this paper a model for parallel computation is proposed, which views a computation as consisting of a hierarchy of multiple logical pipelines. The multiprocessor is organized as pipelines to perform the computation, which takes advantage of overlapped communication and computation in the pipelines to reduce the communication overhead in the system. An analytic model is introduced to characterize the effects of partitioning and pipelining on the resulting performance. Through such analyses, important design parameters can be determined and efficient algorithms will result. Three examples are discussed to illustrate the application of the models. Performance analysis through the analytic model shows an almost linear speedup using overlapped and pipelined operations. 16 refs., 13 figs., 1 tab.
- Research Organization:
- Michigan State Univ., East Lansing (USA); Argonne National Lab., IL (USA)
- DOE Contract Number:
- W-31109-ENG-38
- OSTI ID:
- 6888853
- Report Number(s):
- CONF-880856-1; ON: DE88010078
- Country of Publication:
- United States
- Language:
- English
Similar Records
Large-grain pipelining on distributed-memory multiprocessors
Pipelined data parallel algorithms: Concept and modeling