Sythesis of MCMC and Belief Propagation
Abstract
Markov Chain Monte Carlo (MCMC) and Belief Propagation (BP) are the most popular algorithms for computational inference in Graphical Models (GM). In principle, MCMC is an exact probabilistic method which, however, often suffers from exponentially slow mixing. In contrast, BP is a deterministic method, which is typically fast, empirically very successful, however in general lacking control of accuracy over loopy graphs. In this paper, we introduce MCMC algorithms correcting the approximation error of BP, i.e., we provide a way to compensate for BP errors via a consecutive BPaware MCMC. Our framework is based on the Loop Calculus (LC) approach which allows to express the BP error as a sum of weighted generalized loops. Although the full series is computationally intractable, it is known that a truncated series, summing up all 2regular loops, is computable in polynomialtime for planar pairwise binary GMs and it also provides a highly accurate approximation empirically. Motivated by this, we first propose a polynomialtime approximation MCMC scheme for the truncated series of general (nonplanar) pairwise binary models. Our main idea here is to use the Worm algorithm, known to provide fast mixing in other (related) problems, and then design an appropriate rejection scheme to sample 2regularmore »
 Authors:
 Korea Advanced Institute of Science and Technology, Daejeon (South Korea)
 Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
 Korea Advanced Institute of Science and Technology, Daejeon (South Korea)
 Publication Date:
 Research Org.:
 Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
 Sponsoring Org.:
 USDOE Office of Electricity Delivery and Energy Reliability (OE)
 OSTI Identifier:
 1254988
 Report Number(s):
 LAUR1623749
 DOE Contract Number:
 AC5206NA25396
 Resource Type:
 Technical Report
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; Computer Science
Citation Formats
Ahn, Sungsoo, Chertkov, Michael, and Shin, Jinwoo. Sythesis of MCMC and Belief Propagation. United States: N. p., 2016.
Web. doi:10.2172/1254988.
Ahn, Sungsoo, Chertkov, Michael, & Shin, Jinwoo. Sythesis of MCMC and Belief Propagation. United States. doi:10.2172/1254988.
Ahn, Sungsoo, Chertkov, Michael, and Shin, Jinwoo. Fri .
"Sythesis of MCMC and Belief Propagation". United States.
doi:10.2172/1254988. https://www.osti.gov/servlets/purl/1254988.
@article{osti_1254988,
title = {Sythesis of MCMC and Belief Propagation},
author = {Ahn, Sungsoo and Chertkov, Michael and Shin, Jinwoo},
abstractNote = {Markov Chain Monte Carlo (MCMC) and Belief Propagation (BP) are the most popular algorithms for computational inference in Graphical Models (GM). In principle, MCMC is an exact probabilistic method which, however, often suffers from exponentially slow mixing. In contrast, BP is a deterministic method, which is typically fast, empirically very successful, however in general lacking control of accuracy over loopy graphs. In this paper, we introduce MCMC algorithms correcting the approximation error of BP, i.e., we provide a way to compensate for BP errors via a consecutive BPaware MCMC. Our framework is based on the Loop Calculus (LC) approach which allows to express the BP error as a sum of weighted generalized loops. Although the full series is computationally intractable, it is known that a truncated series, summing up all 2regular loops, is computable in polynomialtime for planar pairwise binary GMs and it also provides a highly accurate approximation empirically. Motivated by this, we first propose a polynomialtime approximation MCMC scheme for the truncated series of general (nonplanar) pairwise binary models. Our main idea here is to use the Worm algorithm, known to provide fast mixing in other (related) problems, and then design an appropriate rejection scheme to sample 2regular loops. Furthermore, we also design an efficient rejectionfree MCMC scheme for approximating the full series. The main novelty underlying our design is in utilizing the concept of cycle basis, which provides an efficient decomposition of the generalized loops. In essence, the proposed MCMC schemes run on transformed GM built upon the nontrivial BP solution, and our experiments show that this synthesis of BP and MCMC outperforms both direct MCMC and bare BP schemes.},
doi = {10.2172/1254988},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Fri May 27 00:00:00 EDT 2016},
month = {Fri May 27 00:00:00 EDT 2016}
}

Loop calculus introduced in [1], [2] constitutes a new theoretical tool that explicitly expresses symbol MaximumAPosteriori (MAP) solution of a general statistical inference problem via a solution of the Belief Propagation (BP) equations. This finding brought a new significance to the BP concept, which in the past was thought of as just a loopfree approximation. In this paper they continue a discussion of the Loop Calculus, partitioning the results into three Sections. In Section 1 they introduce a new formulation of the Loop Calculus in terms of a set of transformations (gauges) that keeping the partition function of the problemmore »

A novel process for methanol synthesis. [Concurrent sythesis of methly formate and methanol]
A benchscale reactor is being used to conduct studies of the conversion of synthesis gas to methanol (MeOH) by a novel process. In previous reports, we provided evidence for a twostep reaction in series, the carbonylation reaction taking place mainly in a nonequilibrium region in the vicinity of the copper chromite surface, and the hydrogenolysis reaction taking place on the surface of the copper chromite. Interaction between the two catalysts enhances the rate of methanol formation. In this quarter, we investigated the effect of pore diffusion on reaction rate and obtained an expression for the rate of reaction for themore » 
Energyrelated attitude/belief variables in conventional econometric equations: An empirical approach applied to residential energy consumption. Doctoral thesis
The study analyzes a subsample of 523 households from the 1975 Lifestyles and Household Energy Use Survey conducted for the Washington Center for Metropolitan Studies. The study explores the empirical relationship between a set of four EnergyRelated Attitude/Belief (ERAB) variables, household electricity and natural gas consumption, and three EnergyRelated Discrete Choice (ERDC) variables. Using principal components factor analysis, the ERAB variables were constructed from a portion of the survey responses dealing with what households felt should be done to handle current or future energy shortages. A key finding of the study is that in the context of a conventional econometricmore » 
The method of belief scales as a means for dealing with uncertainty in tough regulatory decisions.
Modeling and simulation is playing an increasing role in supporting tough regulatory decisions, which are typically characterized by variabilities and uncertainties in the scenarios, input conditions, failure criteria, model parameters, and even model form. Variability exists when there is a statistically significant database that is fully relevant to the application. Uncertainty, on the other hand, is characterized by some degree of ignorance. A simple algebraic problem was used to illustrate how various risk methodologies address variability and uncertainty in a regulatory context. These traditional risk methodologies include probabilistic methods (including frequensic and Bayesian perspectives) and secondorder methods where variabilities andmore » 
Validation of the thermal challenge problem using Bayesian Belief Networks.
The thermal challenge problem has been developed at Sandia National Laboratories as a testbed for demonstrating various types of validation approaches and prediction methods. This report discusses one particular methodology to assess the validity of a computational model given experimental data. This methodology is based on Bayesian Belief Networks (BBNs) and can incorporate uncertainty in experimental measurements, in physical quantities, and model uncertainties. The approach uses the prior and posterior distributions of model output to compute a validation metric based on Bayesian hypothesis testing (a Bayes' factor). This report discusses various aspects of the BBN, specifically in the context ofmore »