 
Summary: The Generalized Distributive Law and Free Energy Minimization
Srinivas M. Aji Robert J. McEliece
Rainfinity, Inc. Department of Electrical Engineering
87 N. Raymond Ave. Suite 200 California Institute of Technology
Pasadena, CA 91103 Pasadena, CA 91125
saji@rainfinity.com rjm@systems.caltech.edu
Abstract.
In an important recent paper, Yedidia, Freeman, and Weiss [7] showed that there is
a close connection between the belief propagation algorithm for probabilistic infer
ence and the BetheKikuchi approximation to the variational free energy in statistical
physics. In this paper, we will recast the YFW results in the context of the "generalized
distributive law" [1] formulation of belief propagation. Our main result is that if the
GDL is applied to junction graph, the fixed points of the algorithm are in onetoone
correspondence with the stationary points of a certain BetheKikuchi free energy. If the
junction graph has no cycles, the BK free energy is convex and has a unique stationary
point, which is a global minimum. On the other hand, if the junction graph has cycles,
the main result at least shows that the GDL is trying to do something sensible.
1. Introduction.
The goals of this paper are twofold: first, to obtain a better understanding of iterative,
belief propagation (BP)like solutions to the general probabilistic inference (PI) prob
