Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models

Katsevich, Anya; Bandeira, Afonso S.

doi:10.1002/cpa.22051

Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models

Journal Article · Sun Apr 17 00:00:00 EDT 2022 · Communications on Pure and Applied Mathematics

DOI:https://doi.org/10.1002/cpa.22051· OSTI ID:2424512

Katsevich, Anya ^[1]; Bandeira, Afonso S. ^[2]

Courant Institute, New York, NY (United States)
ETH Zurich (Switzerland)

We derive an asymptotic expansion for the log-likelihood of Gaussian mixture models (GMMs) with equal covariance matrices in the low signal-to-noise regime. The expansion reveals an intimate connection between two types of algorithms for parameter estimation: the method of moments and likelihood optimizing algorithms such as Expectation-Maximization (EM). We show that likelihood optimization in the low SNR regime reduces to a sequence of least squares optimization problems that match the moments of the estimate to the ground truth moments one by one. This connection is a stepping stone towards the analysis of EM and maximum likelihood estimation in a wide range of models. A motivating application for the study of low SNR mixture models is cryo-electron microscopy data, which can be modeled as a GMM with algebraic constraints imposed on the mixture centers. We discuss the application of our expansion to algebraically constrained GMMs, among other example models of interest. © 2022 The Authors. Communications on Pure and Applied Mathematics published by Wiley Periodicals LLC.

View Accepted Manuscript (DOE)

Research Organization:: Texas A & M University, College Station, TX (United States)

Sponsoring Organization:: USDOE; National Science Foundation (NSF)

OSTI ID:: 2424512

Journal Information:: Communications on Pure and Applied Mathematics, Journal Name: Communications on Pure and Applied Mathematics Journal Issue: 4 Vol. 76; ISSN 0010-3640

Publisher:: WileyCopyright Statement

Country of Publication:: United States

Language:: English

References (14)

A Maximum-Likelihood Approach to Single-Particle Image Refinement Sigworth, F. J. Journal of Structural Biology, Vol. 122, Issue 3 https://doi.org/10.1006/jsbi.1998.4014	journal	January 1998
Hyperbolic polynomials and Vandermonde mappings Arnol'd, V. I. Functional Analysis and Its Applications, Vol. 20, Issue 2 https://doi.org/10.1007/BF01077267	journal	January 1986
Approximate nonparametric maximum likelihood for mixture models: A convex optimization approach to fitting arbitrary multivariate mixing distributions Feng, Long; Dicker, Lee H. Computational Statistics & Data Analysis, Vol. 122 https://doi.org/10.1016/j.csda.2018.01.006	journal	June 2018
Nonparametric Maximum Likelihood Estimation of a Mixing Distribution Laird, Nan Journal of the American Statistical Association, Vol. 73, Issue 364 https://doi.org/10.1080/01621459.1978.10480103	journal	December 1978
Multi-target detection with application to cryo-electron microscopy Bendory, Tamir; Boumal, Nicolas; Leeb, William Inverse Problems, Vol. 35, Issue 10 https://doi.org/10.1088/1361-6420/ab2aec	journal	September 2019
Contributions to the Mathematical Theory of Evolution Pearson, K. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 185, Issue 0 https://doi.org/10.1098/rsta.1894.0003	journal	January 1894
Single-Particle Cryo-Electron Microscopy: Mathematical Theory, Computational Challenges, and Opportunities Bendory, Tamir; Bartesaghi, Alberto; Singer, Amit IEEE Signal Processing Magazine, Vol. 37, Issue 2 https://doi.org/10.1109/MSP.2019.2957822	journal	March 2020
Bispectrum Inversion With Application to Multireference Alignment Bendory, Tamir; Boumal, Nicolas; Ma, Chao IEEE Transactions on Signal Processing, Vol. 66, Issue 4 https://doi.org/10.1109/TSP.2017.2775591	journal	February 2018
Multi-Target Detection With an Arbitrary Spacing Distribution Lan, Ti-Yen; Bendory, Tamir; Boumal, Nicolas IEEE Transactions on Signal Processing, Vol. 68 https://doi.org/10.1109/TSP.2020.2975943	journal	January 2020
Heterogeneous Multireference Alignment for Images With Application to 2D Classification in Single Particle Reconstruction Ma, Chao; Bendory, Tamir; Boumal, Nicolas IEEE Transactions on Image Processing, Vol. 29 https://doi.org/10.1109/tip.2019.2945686	journal	January 2020
Mathematics for Cryo-Electron Microscopy Singer, Amit Proceedings of the International Congress of Mathematicians (ICM 2018) https://doi.org/10.1142/9789813272880_0209	conference	May 2019
Statistical guarantees for the EM algorithm: From population to sample-based analysis Balakrishnan, Sivaraman; Wainwright, Martin J.; Yu, Bin The Annals of Statistics, Vol. 45, Issue 1 https://doi.org/10.1214/16-AOS1435	journal	February 2017
On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising Saha, Sujayam; Guntuboyina, Adityanand The Annals of Statistics, Vol. 48, Issue 2 https://doi.org/10.1214/19-AOS1817	journal	April 2020
Computing the Moments of the Complex Gaussian: Full and Sparse Covariance Matrix Fassino, Claudia; Pistone, Giovanni; Rogantin, Maria Piera Mathematics, Vol. 7, Issue 3 https://doi.org/10.3390/math7030263	journal	March 2019

Similar Records

Likelihood-based docking of models into cryo-EM maps

Journal Article · Wed Mar 15 00:00:00 EDT 2023 · Acta Crystallographica. Section D. Structural Biology · OSTI ID:2470496

A row-action alternative to the EM algorithm for maximizing likelihoods in emission tomography

Journal Article · Tue Oct 01 00:00:00 EDT 1996 · IEEE Transactions on Medical Imaging · OSTI ID:418023

MALCOM X: Combining maximum likelihood continuity mapping with Gaussian mixture models

Technical Report · Sat Oct 31 23:00:00 EST 1998 · OSTI ID:677150

Related Subjects

97 MATHEMATICS AND COMPUTING
Mathematics

Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models

Citation Formats

References (14)

Similar Records

Related Subjects