Summary: Open Problems for Latency Hiding
in Networks of Workstations
Matthew Andrews \Lambda Tom Leighton y P. Takis Metaxas z Lisa Zhang x
Most papers describing algorithms for parallel or distributed computation assume a model of com
putation in which all the edges have unit delay. Such a model is nice to work with and it is realistic
for some parallel machines, but not for most. In reality, there are often substantial delays associated
with some or all of the links. These delays can be caused by long wires, links that are realized by
paths that go through one or more intermediate switches, wires that are required to go offchip or
offboard, and/or by the method which is used to prepare a packet for entry into the network. Link
delays are an even greater concern for distributed machines and networks of workstations (NOWs).
This is because some latencies can be very high (due to the fact that some processors can be far
apart physically) and also because the variation among latencies can be high (since some processors
may be very close or even part of the same tightlycoupled parallel machine).
Several methods have been devised in an attempt to deal with communication latencies. The
simplest method is to slow down the computation to the point where the latency is accommodated.
This approach is most commonly used at circuit level, where the clock speed is set to be slow
enough so that all of the data has time to reach its destination before the next step begins. This
means that the circuit needs to be slowed down to accommodate the highest latency. Such an
approach is clearly less than desirable in the context of a NOW with highlatency links.