Inferring the shape of data: a probabilistic framework for analysing experiments in the natural sciences

Ray, Korak Kumar; Verma, Anjali R.; Gonzalez, Ruben L.; Kinz-Thompson, Colin D.

doi:10.1098/rspa.2022.0177

Inferring the shape of data: a probabilistic framework for analysing experiments in the natural sciences

Journal Article · 01 October 2022 · Proceedings of the Royal Society. A. Mathematical, Physical and Engineering Sciences

DOI:https://doi.org/10.1098/rspa.2022.0177· OSTI ID:2424091

^[1]; ^[1]; ^[1]; ^[2]

Department of Chemistry, Columbia University, New York, NY 10027, USA
Department of Chemistry, Rutgers University-Newark, Newark, NJ 07102, USA

A critical step in data analysis for many different types of experiments is the identification of features with theoretically defined shapes in $N$ -dimensional datasets; examples of this process include finding peaks in multi-dimensional molecular spectra or emitters in fluorescence microscopy images. Identifying such features involves determining if the overall shape of the data is consistent with an expected shape; however, it is generally unclear how to quantitatively make this determination. In practice, many analysis methods employ subjective, heuristic approaches, which complicates the validation of any ensuing results—especially as the amount and dimensionality of the data increase. Here, we present a probabilistic solution to this problem by using Bayes’ rule to calculate the probability that the data have any one of several potential shapes. This probabilistic approach may be used to objectively compare how well different theories describe a dataset, identify changes between datasets and detect features within data using a corollary method called Bayesian Inference-based Template Search; several proof-of-principle examples are provided. Altogether, this mathematical framework serves as an automated ‘engine’ capable of computationally executing analysis decisions currently made by visual inspection across the sciences.

Research Organization:: Oak Ridge Associated Universities (ORAU), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

DOE Contract Number:: AC05-06OR23100

OSTI ID:: 2424091

Journal Information:: Proceedings of the Royal Society. A. Mathematical, Physical and Engineering Sciences, Journal Name: Proceedings of the Royal Society. A. Mathematical, Physical and Engineering Sciences Journal Issue: 2266 Vol. 478; ISSN 1364-5021

Publisher:: The Royal Society Publishing

Country of Publication:: United States

Language:: English

References (33)

Imaging modes of atomic force microscopy for application in molecular and cell biology Dufrêne, Yves F.; Ando, Toshio; Garcia, Ricardo Nature Nanotechnology, Vol. 12, Issue 4 https://doi.org/10.1038/nnano.2017.45	journal	April 2017
Proton Fingerprints Portray Molecular Structures: Enhanced Description of the¹H NMR Spectra of Small Molecules Napolitano, José G.; Lankin, David C.; McAlpine, James B. The Journal of Organic Chemistry, Vol. 78, Issue 19 https://doi.org/10.1021/jo4011624	journal	September 2013
Bayesian analysis of individual electron microscopy images: Towards structures of dynamic and heterogeneous biomolecular assemblies Cossio, Pilar; Hummer, Gerhard Journal of Structural Biology, Vol. 184, Issue 3 https://doi.org/10.1016/j.jsb.2013.10.006	journal	December 2013
Estimating the evidence - a review Friel, Nial; Wyse, Jason Statistica Neerlandica, Vol. 66, Issue 3 https://doi.org/10.1111/j.1467-9574.2011.00515.x	journal	January 2012
Multi-wavelength single-molecule fluorescence analysis of transcription mechanisms Friedman, Larry J.; Gelles, Jeff Methods, Vol. 86 https://doi.org/10.1016/j.ymeth.2015.05.026	journal	September 2015
Bayesian Detection of Intensity Changes in Single Molecule and Molecular Dynamics Trajectories Ensign, Daniel L.; Pande, Vijay S. The Journal of Physical Chemistry B, Vol. 114, Issue 1 https://doi.org/10.1021/jp906786b	journal	January 2010
Statistical challenges of high-dimensional data Johnstone, Iain M.; Titterington, D. Michael Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 367, Issue 1906 https://doi.org/10.1098/rsta.2009.0159	journal	November 2009
Probability Theory Jaynes, E. T. https://doi.org/10.1017/CBO9780511790423	book	January 2003
A Review of Calibration Transfer Practices and Instrument Differences in Spectroscopy Workman, Jerome J. Applied Spectroscopy, Vol. 72, Issue 3 https://doi.org/10.1177/0003702817736064	journal	October 2017
A table of integrals of the Error functions Ng, Edward W.; Geller, Murray Journal of Research of the National Bureau of Standards, Section B: Mathematical Sciences, Vol. 73B, Issue 1 https://doi.org/10.6028/jres.073B.001	journal	January 1969
Automated projection spectroscopy (APSY) Hiller, Sebastian; Fiorito, Francesco; Wüthrich, Kurt Proceedings of the National Academy of Sciences, Vol. 102, Issue 31 https://doi.org/10.1073/pnas.0504818102	journal	July 2005
Optical tweezers in single-molecule biophysics Bustamante, Carlos J.; Chemla, Yann R.; Liu, Shixin Nature Reviews Methods Primers, Vol. 1, Issue 1 https://doi.org/10.1038/s43586-021-00021-6	journal	March 2021
Bayesian inference in physics von Toussaint, Udo Reviews of Modern Physics, Vol. 83, Issue 3 https://doi.org/10.1103/RevModPhys.83.943	journal	September 2011
Single Particle Tracking: From Theory to Biophysical Applications Shen, Hao; Tauzin, Lawrence J.; Baiyasi, Rashad Chemical Reviews, Vol. 117, Issue 11 https://doi.org/10.1021/acs.chemrev.6b00815	journal	May 2017
A Review of Super-Resolution Single-Molecule Localization Microscopy Cluster Analysis and Quantification Methods Khater, Ismail M.; Nabi, Ivan Robert; Hamarneh, Ghassan Patterns, Vol. 1, Issue 3 https://doi.org/10.1016/j.patter.2020.100038	journal	June 2020
Template matching using fast normalized cross correlation Briechle, Kai; Hanebeck, Uwe D. SPIE Proceedings https://doi.org/10.1117/12.421129	conference	March 2001
Three-dimensional sub–100 nm resolution fluorescence microscopy of thick samples Juette, Manuel F.; Gould, Travis J.; Lessard, Mark D. Nature Methods, Vol. 5, Issue 6 https://doi.org/10.1038/nmeth.1211	journal	May 2008
Mass Spectrometry and Protein Analysis Domon, B. Science, Vol. 312, Issue 5771 https://doi.org/10.1126/science.1124619	journal	April 2006
Understanding and evaluating blind deconvolution algorithms Levin, Anat; Weiss, Yair; Durand, Fredo 2009 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2009.5206815	conference	June 2009
Biological mechanisms, one molecule at a time Tinoco, Ignacio; Gonzalez, Ruben L. Genes & Development, Vol. 25, Issue 12 https://doi.org/10.1101/gad.2050011	journal	June 2011
Free R value: a novel statistical quantity for assessing the accuracy of crystal structures Brünger, Axel T. Nature, Vol. 355, Issue 6359 https://doi.org/10.1038/355472a0	journal	January 1992
Statistical Shape Analysis, with Applications in R Dryden, Ian L.; Mardia, Kanti V. Wiley Series in Probability and Statistics https://doi.org/10.1002/9781119072492	book	September 2016
Bayes Offers a 'New' Way to Make Sense of Numbers Malakoff, David Science, Vol. 286, Issue 5444 https://doi.org/10.1126/science.286.5444.1460	journal	November 1999
Hierarchical Dirichlet Processes Teh, Yee Whye; Jordan, Michael I.; Beal, Matthew J. Journal of the American Statistical Association, Vol. 101, Issue 476 https://doi.org/10.1198/016214506000000302	journal	December 2006
Prevention of overfitting in cryo-EM structure determination Scheres, Sjors H. W.; Chen, Shaoxia Nature Methods, Vol. 9, Issue 9 https://doi.org/10.1038/nmeth.2115	journal	July 2012
Tackling the widespread and critical impact of batch effects in high-throughput data Leek, Jeffrey T.; Scharpf, Robert B.; Bravo, Héctor Corrada Nature Reviews Genetics, Vol. 11, Issue 10 https://doi.org/10.1038/nrg2825	journal	September 2010
Deformed alignment of super-resolution images for semi-flexible structures Shi, Xiaoyu; Garcia, Galo; Wang, Yina PLOS ONE, Vol. 14, Issue 3 https://doi.org/10.1371/journal.pone.0212735	journal	March 2019
BioAFMviewer: An interactive interface for simulated AFM scanning of biomolecular structures and dynamics Amyot, Romain; Flechsig, Holger PLOS Computational Biology, Vol. 16, Issue 11 https://doi.org/10.1371/journal.pcbi.1008444	journal	November 2020
Single-Molecule Reaction Chemistry in Patterned Nanowells Bouilly, Delphine; Hon, Jason; Daly, Nathan S. Nano Letters, Vol. 16, Issue 7 https://doi.org/10.1021/acs.nanolett.6b02149	journal	June 2016
Bayesian Inference: The Comprehensive Approach to Analyzing Single-Molecule Experiments Kinz-Thompson, Colin D.; Ray, Korak Kumar; Gonzalez, Ruben L. Annual Review of Biophysics, Vol. 50, Issue 1 https://doi.org/10.1146/annurev-biophys-082120-103921	journal	May 2021
Automated interpretation of vibrational spectra Luinge, H. J. Vibrational Spectroscopy, Vol. 1, Issue 1 https://doi.org/10.1016/0924-2031(90)80002-L	journal	December 1990
Three-Dimensional Electron Microscopy of Macromolecular Assemblies Frank, Joachim Oxford University Press https://doi.org/10.1093/acprof:oso/9780195182187.001.0001	book	January 2006
Single-molecule fluorescence to study molecular motors Park, Hyokeun; Toprak, Erdal; Selvin, Paul R. Quarterly Reviews of Biophysics, Vol. 40, Issue 1 https://doi.org/10.1017/S0033583507004611	journal	February 2007

Similar Records

Artificial intelligence inferred microstructural properties from voltage–capacity curves

Journal Article · 2022 · Scientific Reports · OSTI ID:1879796

Coincident learning for unsupervised anomaly detection of scientific instruments

Journal Article · 2024 · Machine Learning: Science and Technology · OSTI ID:2426670

Dimensionally reduced machine learning model for predicting single component octanol–water partition coefficients

Journal Article · 2023 · Journal of Cheminformatics · OSTI ID:1909929

Related Subjects

Science & Technology - Other Topics

Inferring the shape of data: a probabilistic framework for analysing experiments in the natural sciences

Citation Formats

References (33)

Similar Records

Related Subjects