skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: LOOP CALCULUS AND BELIEF PROPAGATION FOR Q-ARY ALPHABET: LOOP TOWER

Abstract

Loop calculus introduced in [1], [2] constitutes a new theoretical tool that explicitly expresses symbol Maximum-A-Posteriori (MAP) solution of a general statistical inference problem via a solution of the Belief Propagation (BP) equations. This finding brought a new significance to the BP concept, which in the past was thought of as just a loop-free approximation. In this paper they continue a discussion of the Loop Calculus, partitioning the results into three Sections. In Section 1 they introduce a new formulation of the Loop Calculus in terms of a set of transformations (gauges) that keeping the partition function of the problem invariant. The full expression contains two terms referred to as the 'ground state' and 'excited states' contributions. The BP equations are interpreted as a special (BP) gauge fixing condition that emerges as a special orthogonality constraint between the ground state and excited states, which also selects loop contributions as the only surviving ones among the excited states. In Section 2 they demonstrate how the invariant interpretation of the Loop Calculus, introduced in Section 1, allows a natural extension to the case of a general q-ary alphabet, this is achieved via a loop tower sequential construction. The ground level in themore » tower is exactly equivalent to assigning one color (out of q available) to the 'ground state' and considering all 'excited' states colored in the remaining (q-1) colors, according to the loop calculus rule. Sequentially, the second level in the tower corresponds to selecting a loop from the previous step, colored in (q-1) colors, and repeating the same ground vs excited states splitting procedure into one and (q-2) colors respectively. The construction proceeds till the full (q-1)-levels deep loop tower (and the corresponding contributions to the partition function) are established. In Section 3 they discuss an ultimate relation between the loop calculus and the Bethe-Free energy variational approach of [3].« less

Authors:
 [1];  [1]
  1. Los Alamos National Laboratory
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
OSTI Identifier:
1000500
Report Number(s):
LA-UR-07-0149
TRN: US201101%%635
DOE Contract Number:
AC52-06NA25396
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
99; COLOR; CONSTRUCTION; EXCITED STATES; GROUND LEVEL; GROUND STATES; PARTITION FUNCTIONS; TRANSFORMATIONS

Citation Formats

CHERTKOV, MICHAEL, and CHERNYAK, VLADIMIR. LOOP CALCULUS AND BELIEF PROPAGATION FOR Q-ARY ALPHABET: LOOP TOWER. United States: N. p., 2007. Web. doi:10.2172/1000500.
CHERTKOV, MICHAEL, & CHERNYAK, VLADIMIR. LOOP CALCULUS AND BELIEF PROPAGATION FOR Q-ARY ALPHABET: LOOP TOWER. United States. doi:10.2172/1000500.
CHERTKOV, MICHAEL, and CHERNYAK, VLADIMIR. Wed . "LOOP CALCULUS AND BELIEF PROPAGATION FOR Q-ARY ALPHABET: LOOP TOWER". United States. doi:10.2172/1000500. https://www.osti.gov/servlets/purl/1000500.
@article{osti_1000500,
title = {LOOP CALCULUS AND BELIEF PROPAGATION FOR Q-ARY ALPHABET: LOOP TOWER},
author = {CHERTKOV, MICHAEL and CHERNYAK, VLADIMIR},
abstractNote = {Loop calculus introduced in [1], [2] constitutes a new theoretical tool that explicitly expresses symbol Maximum-A-Posteriori (MAP) solution of a general statistical inference problem via a solution of the Belief Propagation (BP) equations. This finding brought a new significance to the BP concept, which in the past was thought of as just a loop-free approximation. In this paper they continue a discussion of the Loop Calculus, partitioning the results into three Sections. In Section 1 they introduce a new formulation of the Loop Calculus in terms of a set of transformations (gauges) that keeping the partition function of the problem invariant. The full expression contains two terms referred to as the 'ground state' and 'excited states' contributions. The BP equations are interpreted as a special (BP) gauge fixing condition that emerges as a special orthogonality constraint between the ground state and excited states, which also selects loop contributions as the only surviving ones among the excited states. In Section 2 they demonstrate how the invariant interpretation of the Loop Calculus, introduced in Section 1, allows a natural extension to the case of a general q-ary alphabet, this is achieved via a loop tower sequential construction. The ground level in the tower is exactly equivalent to assigning one color (out of q available) to the 'ground state' and considering all 'excited' states colored in the remaining (q-1) colors, according to the loop calculus rule. Sequentially, the second level in the tower corresponds to selecting a loop from the previous step, colored in (q-1) colors, and repeating the same ground vs excited states splitting procedure into one and (q-2) colors respectively. The construction proceeds till the full (q-1)-levels deep loop tower (and the corresponding contributions to the partition function) are established. In Section 3 they discuss an ultimate relation between the loop calculus and the Bethe-Free energy variational approach of [3].},
doi = {10.2172/1000500},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed Jan 10 00:00:00 EST 2007},
month = {Wed Jan 10 00:00:00 EST 2007}
}

Technical Report:

Save / Share:
  • Markov Chain Monte Carlo (MCMC) and Belief Propagation (BP) are the most popular algorithms for computational inference in Graphical Models (GM). In principle, MCMC is an exact probabilistic method which, however, often suffers from exponentially slow mixing. In contrast, BP is a deterministic method, which is typically fast, empirically very successful, however in general lacking control of accuracy over loopy graphs. In this paper, we introduce MCMC algorithms correcting the approximation error of BP, i.e., we provide a way to compensate for BP errors via a consecutive BP-aware MCMC. Our framework is based on the Loop Calculus (LC) approach whichmore » allows to express the BP error as a sum of weighted generalized loops. Although the full series is computationally intractable, it is known that a truncated series, summing up all 2-regular loops, is computable in polynomial-time for planar pair-wise binary GMs and it also provides a highly accurate approximation empirically. Motivated by this, we first propose a polynomial-time approximation MCMC scheme for the truncated series of general (non-planar) pair-wise binary models. Our main idea here is to use the Worm algorithm, known to provide fast mixing in other (related) problems, and then design an appropriate rejection scheme to sample 2-regular loops. Furthermore, we also design an efficient rejection-free MCMC scheme for approximating the full series. The main novelty underlying our design is in utilizing the concept of cycle basis, which provides an efficient decomposition of the generalized loops. In essence, the proposed MCMC schemes run on transformed GM built upon the non-trivial BP solution, and our experiments show that this synthesis of BP and MCMC outperforms both direct MCMC and bare BP schemes.« less
  • We introduce novel results for approximate inference on planar graphical models using the loop calculus framework. The loop calculus (Chertkov and Chernyak, 2006b) allows to express the exact partition function Z of a graphical model as a finite sum of terms that can be evaluated once the belief propagation (BP) solution is known. In general, full summation over all correction terms is intractable. We develop an algorithm for the approach presented in Chertkov et al. (2008) which represents an efficient truncation scheme on planar graphs and a new representation of the series in terms of Pfaffians of matrices. We analyzemore » in detail both the loop series and the Pfaffian series for models with binary variables and pairwise interactions, and show that the first term of the Pfaffian series can provide very accurate approximations. The algorithm outperforms previous truncation schemes of the loop series and is competitive with other state-of-the-art methods for approximate inference.« less
  • An attempt is made to derive and to solve the Schrodinger equation in the low energy region (vacuum, first excitation, etc.) of the Yang-Mills theory on the lattice. The complete orthonormal basis in the physical Hilbert space is constructed by classifying independent solutions of Gauss's law. Loops of electric flux are chosen as elementary variables. The loop space Hamiltonian is derived, an ansatz is made for the low energy wave functionals and the Schrodinger equation is solved in the (truncated) loop basis. The resulting physical picture for the Yang-Mills vacuum in the cross-over region is that of, still quite dilute,more » gas of fluctuating loops. Definite candidate for the confining force emerges: the repulsive non-abelian loop-loop interaction (rather weak but persistent) generates an effective external field (''external pressure'') prohibiting unbounded loop size fluctuations. The negative sign (repulsion) is universal for all compact groups. Preliminary numerical results, so far mainly of illustrative character, are presented. 8 refs., 22 figs.« less
  • The technique known in solid state physics as the n-loop expansion is applied to calculate the critical indices of the phi/sup 3/ Gribov Reggeon calculus directly in two transverse dimensions. Infrared pathologies of the massless theory require the calculation to be done in the infinite momentum limit of the massive theory. For n = 1 the results are close to those of the epsilon-expansion in O(epsilon). For n = 2 the ..beta.. function has no zero, analogously to the case in solid state physics. Use of a Pade approximant for ..beta.. ..-->.. sigma/sub tot/ approximately equals(ln s)/sup 0/./sup 27/ atmore » infinity, close to the O(epsilon/sup 2/) result.« less
  • Fabrication and pre-irradiation data of U)/sub 2/for the x-2-q charge are reported. (auth)