Umbrella sampling: a powerful method to sample tails of distributions
- Department of Statistics, and James Frank Institute, The University of Chicago, Chicago IL 60637
- Department of Astronomy & Astrophysics, and Kavli Institute for Cosmological Physics, University of Chicago, Chicago IL 60637, Enrico Fermi Institute, The University of Chicago, Chicago, IL 60637, USA
- Theoretical Astrophysics Division, Fermi National Laboratory, Batavia, IL, USA, Leadership Computing Facility, Argonne National Laboratory, Argonne, IL, 60439, USA
Here, we present the umbrella sampling (US) technique and show that it can be used to sample extremely low probability areas of the posterior distribution that may be required in statistical analyses of data. In this approach sampling of the target likelihood is split into sampling of multiple biased likelihoods confined within individual umbrella windows. We show that the US algorithm is efficient and highly parallel and that it can be easily used with other existing MCMC samplers. The method allows the user to capitalize on their intuition and define umbrella windows and increase sampling accuracy along specific directions in the parameter space. Alternatively, one can define umbrella windows using an approach similar to parallel tempering. We provide a public code that implements umbrella sampling as a standalone python package. We present a number of tests illustrating the power of the US method in sampling low probability areas of the posterior and show that this ability allows a considerably more robust sampling of multi-modal distributions compared to the standard sampling methods. We also present an application of the method in a real world example of deriving cosmological constraints using the supernova type Ia data. We show that umbrella sampling can sample the posterior accurately down to the ≈15σ credible region in the Ωm–ΩΛ plane, while for the same computational work the affine-invariant MCMC sampling implemented in the emcee code samples the posterior reliably only to ≈3σ.
- Research Organization:
- Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- Grant/Contract Number:
- AC02-06CH11347; SC0014205; AC02-07CH11359
- OSTI ID:
- 1465882
- Alternate ID(s):
- OSTI ID: 1497716
- Report Number(s):
- arXiv:1712.05024; FERMILAB-PUB-17-691-A
- Journal Information:
- Monthly Notices of the Royal Astronomical Society, Journal Name: Monthly Notices of the Royal Astronomical Society Vol. 480 Journal Issue: 3; ISSN 0035-8711
- Publisher:
- Oxford University PressCopyright Statement
- Country of Publication:
- United Kingdom
- Language:
- English
Web of Science
Similar Records
Implementation of a practical Markov chain Monte Carlo sampling algorithm in PyBioNetFit