skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights

Abstract

Formulas for incremental or parallel computation of second order central moments have long been known, and recent extensions of these formulas to univariate and multivariate moments of arbitrary order have been developed. Such formulas are of key importance in scenarios where incremental results are required and in parallel and distributed systems where communication costs are high. We survey these recent results, and improve them with arbitrary-order, numerically stable one-pass formulas which we further extend with weighted and compound variants. We also develop a generalized correction factor for standard two-pass algorithms that enables the maintenance of accuracy over nearly the full representable range of the input, avoiding the need for extended-precision arithmetic. We then empirically examine algorithm correctness for pairwise update formulas up to order four as well as condition number and relative error bounds for eight different central moment formulas, each up to degree six, to address the trade-offs between numerical accuracy and speed of the various algorithms. Finally, we demonstrate the use of the most elaborate among the above mentioned formulas, with the utilization of the compound moments for a practical large-scale scientific application.

Authors:
 [1];  [2];  [1];  [1]
  1. Sandia National Lab. (SNL-CA), Livermore, CA (United States)
  2. The Xiph.Org Foundation, Arlington, VA (United States)
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1427275
Report Number(s):
SAND-2015-9451J
Journal ID: ISSN 0943-4062; 607859
Grant/Contract Number:  
AC04-94AL85000
Resource Type:
Accepted Manuscript
Journal Name:
Computational Statistics
Additional Journal Information:
Journal Volume: 31; Journal Issue: 4; Journal ID: ISSN 0943-4062
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Descriptive Statistics; Statistical Moments; Parallel Computing; Large Data Analysis

Citation Formats

Pebay, Philippe, Terriberry, Timothy B., Kolla, Hemanth, and Bennett, Janine. Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights. United States: N. p., 2016. Web. doi:10.1007/s00180-015-0637-z.
Pebay, Philippe, Terriberry, Timothy B., Kolla, Hemanth, & Bennett, Janine. Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights. United States. doi:10.1007/s00180-015-0637-z.
Pebay, Philippe, Terriberry, Timothy B., Kolla, Hemanth, and Bennett, Janine. Tue . "Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights". United States. doi:10.1007/s00180-015-0637-z. https://www.osti.gov/servlets/purl/1427275.
@article{osti_1427275,
title = {Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights},
author = {Pebay, Philippe and Terriberry, Timothy B. and Kolla, Hemanth and Bennett, Janine},
abstractNote = {Formulas for incremental or parallel computation of second order central moments have long been known, and recent extensions of these formulas to univariate and multivariate moments of arbitrary order have been developed. Such formulas are of key importance in scenarios where incremental results are required and in parallel and distributed systems where communication costs are high. We survey these recent results, and improve them with arbitrary-order, numerically stable one-pass formulas which we further extend with weighted and compound variants. We also develop a generalized correction factor for standard two-pass algorithms that enables the maintenance of accuracy over nearly the full representable range of the input, avoiding the need for extended-precision arithmetic. We then empirically examine algorithm correctness for pairwise update formulas up to order four as well as condition number and relative error bounds for eight different central moment formulas, each up to degree six, to address the trade-offs between numerical accuracy and speed of the various algorithms. Finally, we demonstrate the use of the most elaborate among the above mentioned formulas, with the utilization of the compound moments for a practical large-scale scientific application.},
doi = {10.1007/s00180-015-0637-z},
journal = {Computational Statistics},
number = 4,
volume = 31,
place = {United States},
year = {2016},
month = {3}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 4 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Contributions to the Mathematical Theory of Evolution
journal, January 1894

  • Pearson, K.
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 185, Issue 0
  • DOI: 10.1098/rsta.1894.0003

Structure of a spatially developing turbulent lean methane–air Bunsen flame
journal, January 2007

  • Sankaran, Ramanan; Hawkes, Evatt R.; Chen, Jacqueline H.
  • Proceedings of the Combustion Institute, Vol. 31, Issue 1
  • DOI: 10.1016/j.proci.2006.08.025

Note on a Method for Calculating Corrected Sums of Squares and Products
journal, August 1962


CAPM, Higher Co-moment and Factor Models of UK Stock Returns
journal, January 2004


Adaptive estimation of the fourth-order cumulant of a white stochastic process
journal, February 1995


Conditional Skewness in Asset Pricing Tests
journal, June 2000


The Theory of Unbiased Estimation
journal, March 1946


Signal processing with higher-order spectra
journal, July 1993

  • Nikias, C. L.; Mendel, J. M.
  • IEEE Signal Processing Magazine, Vol. 10, Issue 3
  • DOI: 10.1109/79.221324

Design and Performance of a Scalable, Parallel Statistics Toolkit
conference, May 2011

  • Pebay, Philippe; Thompson, David; Bennett, Janine
  • Distributed Processing, Workshops and Phd Forum (IPDPSW), 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum
  • DOI: 10.1109/IPDPS.2011.293

A blind image restoration system using higher-order statistics and Radon transform
conference, January 1998

  • Ibrahim, M. A.; Hussein, A. -W. F.; Mashali, S. A.
  • 1998 IEEE International Conference on Electronics, Circuits and Systems. Surfing the Waves of Science and Technology (Cat. No.98EX196)
  • DOI: 10.1109/ICECS.1998.814068

Blind removal of image non-linearities
conference, January 2001

  • Faird, H.; Popescu, A. C.
  • Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001
  • DOI: 10.1109/ICCV.2001.937501

M-ary Shift Keying Modulation Scheme Identification Algorithm Using Wavelet Transform and Higher Order Statistical Moment
journal, January 2008


Numerically stable, single-pass, parallel statistics algorithms
conference, August 2009

  • Bennett, Janine; Grout, Ray; Pebay, Philippe
  • 2009 IEEE International Conference on Cluster Computing and Workshops
  • DOI: 10.1109/CLUSTR.2009.5289161

New criteria for blind deconvolution of nonminimum phase systems (channels)
journal, March 1990

  • Shalvi, O.; Weinstein, E.
  • IEEE Transactions on Information Theory, Vol. 36, Issue 2
  • DOI: 10.1109/18.52478

Robust voice activity detection using higher-order statistics in the LPC residual domain
journal, March 2001

  • Nemer, E.; Goubran, R.; Mahmoud, S.
  • IEEE Transactions on Speech and Audio Processing, Vol. 9, Issue 3
  • DOI: 10.1109/89.905996

Accurate Sum and Dot Product
journal, January 2005

  • Ogita, Takeshi; Rump, Siegfried M.; Oishi, Shin'ichi
  • SIAM Journal on Scientific Computing, Vol. 26, Issue 6
  • DOI: 10.1137/030601818

Updating mean and variance estimates: an improved method
journal, September 1979


An adaptive order-statistic noise filter for gamma-corrected image sequences
journal, January 1997

  • Kleihorst, R. P.; Lagendiik, R. L.; Biemond, J.
  • IEEE Transactions on Image Processing, Vol. 6, Issue 10
  • DOI: 10.1109/83.624968

How to Ensure a Faithful Polynomial Evaluation with the Compensated Horner Algorithm
conference, June 2007

  • Langlois, Philippe; Louvet, Nicolas
  • 18th IEEE Symposium on Computer Arithmetic (ARITH '07)
  • DOI: 10.1109/ARITH.2007.21

Comparison of several algorithms for computation of means, standard deviations and correlation coefficients
journal, July 1966


Recursive estimation of fourth-order cumulants with application to identification
journal, July 1998


A Blind Image Restoration Algorithm Based on Cumulants
conference, January 2006

  • Wang, Hongzhi; Huo, Jinming; Song, Chunpeng
  • 2006 8th international Conference on Signal Processing
  • DOI: 10.1109/ICOSP.2006.344508

Direction finding algorithms based on high-order statistics
journal, January 1991

  • Porat, B.; Friedlander, B.
  • IEEE Transactions on Signal Processing, Vol. 39, Issue 9
  • DOI: 10.1109/78.134434

Performance of chromatic dispersion monitoring using statistical moments of asynchronously sampled waveform histograms
journal, May 2005

  • Kikuchi, N.; Hayase, S.; Sekine, K.
  • IEEE Photonics Technology Letters, Vol. 17, Issue 5
  • DOI: 10.1109/LPT.2005.846752

The Fundamental Approximation Theorem of Portfolio Analysis in terms of Means, Variances and Higher Moments
journal, October 1970

  • Samuelson, P. A.
  • The Review of Economic Studies, Vol. 37, Issue 4
  • DOI: 10.2307/2296483

Image blur identification by using higher order statistic techniques
conference, January 1996

  • You Xu, ; Crebbin, G.
  • Proceedings of 3rd IEEE International Conference on Image Processing
  • DOI: 10.1109/ICIP.1996.560373