Efficient LevenbergMarquardt minimization of the maximum likelihood estimator for Poisson deviates
Abstract
Histograms of counted events are Poisson distributed, but are typically fitted without justification using nonlinear least squares fitting. The more appropriate maximum likelihood estimator (MLE) for Poisson distributed data is seldom used. We extend the use of the LevenbergMarquardt algorithm commonly used for nonlinear least squares minimization for use with the MLE for Poisson distributed data. In so doing, we remove any excuse for not using this more appropriate MLE. We demonstrate the use of the algorithm and the superior performance of the MLE using simulations and experiments in the context of fluorescence lifetime imaging. Scientists commonly form histograms of counted events from their data, and extract parameters by fitting to a specified model. Assuming that the probability of occurrence for each bin is small, event counts in the histogram bins will be distributed according to the Poisson distribution. We develop here an efficient algorithm for fitting event counting histograms using the maximum likelihood estimator (MLE) for Poisson distributed data, rather than the nonlinear least squares measure. This algorithm is a simple extension of the common LevenbergMarquardt (LM) algorithm, is simple to implement, quick and robust. Fitting using a least squares measure is most common, but it is the maximummore »
 Authors:
 Publication Date:
 Research Org.:
 Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
 Sponsoring Org.:
 USDOE
 OSTI Identifier:
 991824
 Report Number(s):
 LLNLJRNL420247
TRN: US1007573
 DOE Contract Number:
 W7405ENG48
 Resource Type:
 Journal Article
 Resource Relation:
 Journal Name: Nature Methods, vol. 7, no. 5, May 1, 2010, pp. 338339; Journal Volume: 7; Journal Issue: 5
 Country of Publication:
 United States
 Language:
 English
 Subject:
 71 CLASSICAL AND QUANTUMM MECHANICS, GENERAL PHYSICS; 59 BASIC BIOLOGICAL SCIENCES; ALGORITHMS; CONVERGENCE; DISTRIBUTION; FLUORESCENCE; LIFETIME; MINIMIZATION; OPTIMIZATION; PERFORMANCE; PROBABILITY; SIMULATION; SPECTROSCOPY
Citation Formats
Laurence, T, and Chromy, B. Efficient LevenbergMarquardt minimization of the maximum likelihood estimator for Poisson deviates. United States: N. p., 2009.
Web.
Laurence, T, & Chromy, B. Efficient LevenbergMarquardt minimization of the maximum likelihood estimator for Poisson deviates. United States.
Laurence, T, and Chromy, B. 2009.
"Efficient LevenbergMarquardt minimization of the maximum likelihood estimator for Poisson deviates". United States.
doi:. https://www.osti.gov/servlets/purl/991824.
@article{osti_991824,
title = {Efficient LevenbergMarquardt minimization of the maximum likelihood estimator for Poisson deviates},
author = {Laurence, T and Chromy, B},
abstractNote = {Histograms of counted events are Poisson distributed, but are typically fitted without justification using nonlinear least squares fitting. The more appropriate maximum likelihood estimator (MLE) for Poisson distributed data is seldom used. We extend the use of the LevenbergMarquardt algorithm commonly used for nonlinear least squares minimization for use with the MLE for Poisson distributed data. In so doing, we remove any excuse for not using this more appropriate MLE. We demonstrate the use of the algorithm and the superior performance of the MLE using simulations and experiments in the context of fluorescence lifetime imaging. Scientists commonly form histograms of counted events from their data, and extract parameters by fitting to a specified model. Assuming that the probability of occurrence for each bin is small, event counts in the histogram bins will be distributed according to the Poisson distribution. We develop here an efficient algorithm for fitting event counting histograms using the maximum likelihood estimator (MLE) for Poisson distributed data, rather than the nonlinear least squares measure. This algorithm is a simple extension of the common LevenbergMarquardt (LM) algorithm, is simple to implement, quick and robust. Fitting using a least squares measure is most common, but it is the maximum likelihood estimator only for Gaussiandistributed data. Nonlinear least squares methods may be applied to event counting histograms in cases where the number of events is very large, so that the Poisson distribution is well approximated by a Gaussian. However, it is not easy to satisfy this criterion in practice  which requires a large number of events. It has been wellknown for years that least squares procedures lead to biased results when applied to Poissondistributed data; a recent paper providing extensive characterization of these biases in exponential fitting is given. The more appropriate measure based on the maximum likelihood estimator (MLE) for the Poisson distribution is also well known, but has not become generally used. This is primarily because, in contrast to nonlinear least squares fitting, there has been no quick, robust, and general fitting method. In the field of fluorescence lifetime spectroscopy and imaging, there have been some efforts to use this estimator through minimization routines such as NelderMead optimization, exhaustive line searches, and GaussNewton minimization. Minimization based on specific one or multiexponential models has been used to obtain quick results, but this procedure does not allow the incorporation of the instrument response, and is not generally applicable to models found in other fields. Methods for using the MLE for Poissondistributed data have been published by the wider spectroscopic community, including iterative minimization schemes based on GaussNewton minimization. The slow acceptance of these procedures for fitting event counting histograms may also be explained by the use of the ubiquitous, fast LevenbergMarquardt (LM) fitting procedure for fitting nonlinear models using least squares fitting (simple searches obtain {approx}10000 references  this doesn't include those who use it, but don't know they are using it). The benefits of LM include a seamless transition between GaussNewton minimization and downward gradient minimization through the use of a regularization parameter. This transition is desirable because GaussNewton methods converge quickly, but only within a limited domain of convergence; on the other hand the downward gradient methods have a much wider domain of convergence, but converge extremely slowly nearer the minimum. LM has the advantages of both procedures: relative insensitivity to initial parameters and rapid convergence. Scientists, when wanting an answer quickly, will fit data using LM, get an answer, and move on. Only those that are aware of the bias issues will bother to fit using the more appropriate MLE for Poisson deviates. However, since there is a simple, analytical formula for the appropriate MLE measure for Poisson deviates, it is inexcusable that least squares estimators are used almost exclusively when fitting event counting histograms. There have been ways found to use successive nonlinear least squares fitting to obtain similarly unbiased results, but this procedure is justified by simulation, must be retested when conditions change significantly, and requires two successive fits. There is a great need for a fitting routine for the MLE estimator for Poisson deviates that has convergence domains and rates comparable to the nonlinear least squares LM fitting. We show in this report that a simple way to achieve that goal is to use the LM fitting procedure not to minimize the least squares measure, but the MLE for Poisson deviates.},
doi = {},
journal = {Nature Methods, vol. 7, no. 5, May 1, 2010, pp. 338339},
number = 5,
volume = 7,
place = {United States},
year = 2009,
month =
}

Inverse modeling seeks model parameters given a set of observations. However, for practical problems because the number of measurements is often large and the model parameters are also numerous, conventional methods for inverse modeling can be computationally expensive. We have developed a new, computationallyefficient parallel LevenbergMarquardt method for solving inverse modeling problems with a highly parameterized model space. LevenbergMarquardt methods require the solution of a linear system of equations which can be prohibitively expensive to compute for moderate to largescale problems. Our novel method projects the original linear problem down to a Krylov subspace, such that the dimensionality of themore »


A new method for obtaining velocity and diffusivity from timedependent distributions of a tracer via the maximum likelihood estimator for the advectiondiffusion equation
An inverse problem for the advectiondiffusion equation is considered, and a method of maximum likelihood (ML) estimation is developed to derive velocity and diffusivity from timedependent distributions of a tracer. Piterbarg and Rozovskii showed theoretically that the ML estimator for diffusivity is consistent ever in an asymptotic case of infinite number of observational spatial modes. In the present work, the ML estimator is studied based on numerical experiments with a tracer in a twodimensional flow under the condition of a limited number of observations in space. The numerical experiments involve the direct and the inverse problems. For the former, themore » 
A New MaximumLikelihood Change Estimator for TwoPass SAR Coherent Change Detection.
Abstract not provided. 
A new maximumlikelihood change estimator for twopass SAR coherent change detection
In past research, twopass repeatgeometry synthetic aperture radar (SAR) coherent change detection (CCD) predominantly utilized the sample degree of coherence as a measure of the temporal change occurring between two complexvalued image collects. Previous coherencebased CCD approaches tend to show temporal change when there is none in areas of the image that have a low cluttertonoise power ratio. Instead of employing the sample coherence magnitude as a change metric, in this paper, we derive a new maximumlikelihood (ML) temporal change estimate—the complex reflectance change detection (CRCD) metric to be used for SAR coherent temporal change detection. The new CRCD estimatormore »Cited by 1