Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Indexing data-oriented overlay networks using belief propagation

Summary: 1
Indexing data-oriented overlay networks using
belief propagation
Danny Bickson, Danny Dolev, and Yair Weiss
Karl Aberer+, Manfred Hauswirth+
The Hebrew University of Jerusalem, Israel (HUJI)
+ ´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Switzerland
In this paper we discuss the problem of data-oriented partitioning in large-scale overlay networks, as required
by peer-to-peer databases or by peer-to-peer information retrieval. The goal is to partition a large set of nodes into
k partitions with the additional requirement of meeting certain load-balancing constraints without global knowledge
of the network's parameters, i.e., the desired number of partitions and the partition distribution function are not
known in advance and change dynamically as the network evolves. This key problem in large-scale decentralized
systems has so far received only very limited attention. The novel contributions described in the following are (1)
the definition of a distributed algorithm for local estimation of the partitioning distribution function, which does not
preclude the network's topology, and (2) a distributed method for performing the actual partitioning. As additional
advantages, the algorithms do not require global knowledge and works highly parallel which provides a low latency
of the partitioning process. Both algorithms are based on the max-product belief propagation algorithm and give exact
results on trees, and sufficiently accurate approximations on graphs containing cycles. We show the accuracy of the
proposed algorithms in terms of the number of nodes per partition and the good load balancing of partitions in the


Source: Aberer, Karl - Faculté Informatique et Communications, Ecole Polytechnique Fédérale de Lausanne


Collections: Computer Technologies and Information Sciences