Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

The Generalized Distributive Law and Free Energy Minimization Srinivas M. Aji Robert J. McEliece

Summary: The Generalized Distributive Law and Free Energy Minimization
Srinivas M. Aji Robert J. McEliece
Rainfinity, Inc. Department of Electrical Engineering
87 N. Raymond Ave. Suite 200 California Institute of Technology
Pasadena, CA 91103 Pasadena, CA 91125
saji@rainfinity.com rjm@systems.caltech.edu
In an important recent paper, Yedidia, Freeman, and Weiss [7] showed that there is
a close connection between the belief propagation algorithm for probabilistic infer-
ence and the Bethe-Kikuchi approximation to the variational free energy in statistical
physics. In this paper, we will recast the YFW results in the context of the "generalized
distributive law" [1] formulation of belief propagation. Our main result is that if the
GDL is applied to junction graph, the fixed points of the algorithm are in one-to-one
correspondence with the stationary points of a certain Bethe-Kikuchi free energy. If the
junction graph has no cycles, the BK free energy is convex and has a unique stationary
point, which is a global minimum. On the other hand, if the junction graph has cycles,
the main result at least shows that the GDL is trying to do something sensible.
1. Introduction.
The goals of this paper are twofold: first, to obtain a better understanding of iterative,
belief propagation (BP)-like solutions to the general probabilistic inference (PI) prob-


Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology
McEliece, Robert J. - Department of Electrical Engineering, California Institute of Technology


Collections: Computer Technologies and Information Sciences; Engineering