skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Umbrella sampling: a powerful method to sample tails of distributions

Abstract

We present the umbrella sampling (US) technique and show that it can be used to sample extremely low-probability areas of the posterior distribution that may be required in statistical analyses of data. In this approach sampling of the target likelihood is split into sampling of multiple biased likelihoods confined within individual umbrella windows. We show that the US algorithm is efficient and highly parallel and that it can be easily used with other existing Markov Chain Monte Carlo (MCMC) samplers. The method allows the user to capitalize on their intuition and define umbrella windows and increase sampling accuracy along specific directions in the parameter space. Alternatively, one can define umbrella windows using an approach similar to parallel tempering. We provide a public code that implements US as a standalone PYTHON package. We present a number of tests illustrating the power of the US method in sampling low-probability areas of the posterior and show that this ability allows a considerably more robust sampling of multimodal distributions compared to standard sampling methods. We also present an application of the method in a real world example of deriving cosmological constraints using the supernova type Ia data. We show that US can sample themore » posterior accurately down to the approximate to 15 sigma credible region in the Omega(m) - Omega(Lambda) plane, while for the same computational effort the affine-invariant MCMC sampling implemented in the emcee code samples the posterior reliably only to approximate to 3 sigma.« less

Authors:
; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
Argonne National Laboratory - Argonne Leadership Computing Facility; National Aeronautic and Space Administration (NASA); USDOE Office of Science - Office of Advanced Scientific Computing Research; University of Chicago - Kavli Institute for Cosmological Physics
OSTI Identifier:
1530578
DOE Contract Number:  
AC02-06CH11357
Resource Type:
Journal Article
Journal Name:
Monthly Notices of the Royal Astronomical Society
Additional Journal Information:
Journal Volume: 480; Journal Issue: 3
Country of Publication:
United States
Language:
English
Subject:
cosmology: cosmological parameters; methods: numerical

Citation Formats

Matthews, Charles, Weare, Jonathan, Kravtsov, Andrey, and Jennings, Elise. Umbrella sampling: a powerful method to sample tails of distributions. United States: N. p., 2018. Web. doi:10.1093/mnras/sty2140.
Matthews, Charles, Weare, Jonathan, Kravtsov, Andrey, & Jennings, Elise. Umbrella sampling: a powerful method to sample tails of distributions. United States. doi:10.1093/mnras/sty2140.
Matthews, Charles, Weare, Jonathan, Kravtsov, Andrey, and Jennings, Elise. Thu . "Umbrella sampling: a powerful method to sample tails of distributions". United States. doi:10.1093/mnras/sty2140.
@article{osti_1530578,
title = {Umbrella sampling: a powerful method to sample tails of distributions},
author = {Matthews, Charles and Weare, Jonathan and Kravtsov, Andrey and Jennings, Elise},
abstractNote = {We present the umbrella sampling (US) technique and show that it can be used to sample extremely low-probability areas of the posterior distribution that may be required in statistical analyses of data. In this approach sampling of the target likelihood is split into sampling of multiple biased likelihoods confined within individual umbrella windows. We show that the US algorithm is efficient and highly parallel and that it can be easily used with other existing Markov Chain Monte Carlo (MCMC) samplers. The method allows the user to capitalize on their intuition and define umbrella windows and increase sampling accuracy along specific directions in the parameter space. Alternatively, one can define umbrella windows using an approach similar to parallel tempering. We provide a public code that implements US as a standalone PYTHON package. We present a number of tests illustrating the power of the US method in sampling low-probability areas of the posterior and show that this ability allows a considerably more robust sampling of multimodal distributions compared to standard sampling methods. We also present an application of the method in a real world example of deriving cosmological constraints using the supernova type Ia data. We show that US can sample the posterior accurately down to the approximate to 15 sigma credible region in the Omega(m) - Omega(Lambda) plane, while for the same computational effort the affine-invariant MCMC sampling implemented in the emcee code samples the posterior reliably only to approximate to 3 sigma.},
doi = {10.1093/mnras/sty2140},
journal = {Monthly Notices of the Royal Astronomical Society},
number = 3,
volume = 480,
place = {United States},
year = {2018},
month = {11}
}