Numerically stable, scalable formulas for parallel and online computation of higherorder multivariate central moments with arbitrary weights
Abstract
Formulas for incremental or parallel computation of second order central moments have long been known, and recent extensions of these formulas to univariate and multivariate moments of arbitrary order have been developed. Such formulas are of key importance in scenarios where incremental results are required and in parallel and distributed systems where communication costs are high. We survey these recent results, and improve them with arbitraryorder, numerically stable onepass formulas which we further extend with weighted and compound variants. We also develop a generalized correction factor for standard twopass algorithms that enables the maintenance of accuracy over nearly the full representable range of the input, avoiding the need for extendedprecision arithmetic. We then empirically examine algorithm correctness for pairwise update formulas up to order four as well as condition number and relative error bounds for eight different central moment formulas, each up to degree six, to address the tradeoffs between numerical accuracy and speed of the various algorithms. Finally, we demonstrate the use of the most elaborate among the above mentioned formulas, with the utilization of the compound moments for a practical largescale scientific application.
 Authors:

 Sandia National Lab. (SNLCA), Livermore, CA (United States)
 The Xiph.Org Foundation, Arlington, VA (United States)
 Publication Date:
 Research Org.:
 Sandia National Lab. (SNLNM), Albuquerque, NM (United States)
 Sponsoring Org.:
 USDOE National Nuclear Security Administration (NNSA)
 OSTI Identifier:
 1427275
 Report Number(s):
 SAND20159451J
Journal ID: ISSN 09434062; 607859
 Grant/Contract Number:
 AC0494AL85000
 Resource Type:
 Accepted Manuscript
 Journal Name:
 Computational Statistics
 Additional Journal Information:
 Journal Volume: 31; Journal Issue: 4; Journal ID: ISSN 09434062
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; Descriptive Statistics; Statistical Moments; Parallel Computing; Large Data Analysis
Citation Formats
Pebay, Philippe, Terriberry, Timothy B., Kolla, Hemanth, and Bennett, Janine. Numerically stable, scalable formulas for parallel and online computation of higherorder multivariate central moments with arbitrary weights. United States: N. p., 2016.
Web. doi:10.1007/s001800150637z.
Pebay, Philippe, Terriberry, Timothy B., Kolla, Hemanth, & Bennett, Janine. Numerically stable, scalable formulas for parallel and online computation of higherorder multivariate central moments with arbitrary weights. United States. doi:10.1007/s001800150637z.
Pebay, Philippe, Terriberry, Timothy B., Kolla, Hemanth, and Bennett, Janine. Tue .
"Numerically stable, scalable formulas for parallel and online computation of higherorder multivariate central moments with arbitrary weights". United States. doi:10.1007/s001800150637z. https://www.osti.gov/servlets/purl/1427275.
@article{osti_1427275,
title = {Numerically stable, scalable formulas for parallel and online computation of higherorder multivariate central moments with arbitrary weights},
author = {Pebay, Philippe and Terriberry, Timothy B. and Kolla, Hemanth and Bennett, Janine},
abstractNote = {Formulas for incremental or parallel computation of second order central moments have long been known, and recent extensions of these formulas to univariate and multivariate moments of arbitrary order have been developed. Such formulas are of key importance in scenarios where incremental results are required and in parallel and distributed systems where communication costs are high. We survey these recent results, and improve them with arbitraryorder, numerically stable onepass formulas which we further extend with weighted and compound variants. We also develop a generalized correction factor for standard twopass algorithms that enables the maintenance of accuracy over nearly the full representable range of the input, avoiding the need for extendedprecision arithmetic. We then empirically examine algorithm correctness for pairwise update formulas up to order four as well as condition number and relative error bounds for eight different central moment formulas, each up to degree six, to address the tradeoffs between numerical accuracy and speed of the various algorithms. Finally, we demonstrate the use of the most elaborate among the above mentioned formulas, with the utilization of the compound moments for a practical largescale scientific application.},
doi = {10.1007/s001800150637z},
journal = {Computational Statistics},
number = 4,
volume = 31,
place = {United States},
year = {2016},
month = {3}
}
Web of Science
Works referenced in this record:
Contributions to the Mathematical Theory of Evolution
journal, January 1894
 Pearson, K.
 Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 185, Issue 0
Structure of a spatially developing turbulent lean methane–air Bunsen flame
journal, January 2007
 Sankaran, Ramanan; Hawkes, Evatt R.; Chen, Jacqueline H.
 Proceedings of the Combustion Institute, Vol. 31, Issue 1
Note on a Method for Calculating Corrected Sums of Squares and Products
journal, August 1962
 Welford, B. P.
 Technometrics, Vol. 4, Issue 3
CAPM, Higher Comoment and Factor Models of UK Stock Returns
journal, January 2004
 Hung, Daniel ChiHsiou; Shackleton, Mark; Xu, Xinzhong

Journal of Business Finance
Accounting, Vol. 31, Issue 12
Adaptive estimation of the fourthorder cumulant of a white stochastic process
journal, February 1995
 Amblard, PierreOlivier; Brossier, JeanMarc
 Signal Processing, Vol. 42, Issue 1
Conditional Skewness in Asset Pricing Tests
journal, June 2000
 Harvey, Campbell R.; Siddique, Akhtar
 The Journal of Finance, Vol. 55, Issue 3
The Theory of Unbiased Estimation
journal, March 1946
 Halmos, Paul R.
 The Annals of Mathematical Statistics, Vol. 17, Issue 1
Signal processing with higherorder spectra
journal, July 1993
 Nikias, C. L.; Mendel, J. M.
 IEEE Signal Processing Magazine, Vol. 10, Issue 3
Design and Performance of a Scalable, Parallel Statistics Toolkit
conference, May 2011
 Pebay, Philippe; Thompson, David; Bennett, Janine
 Distributed Processing, Workshops and Phd Forum (IPDPSW), 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum
A blind image restoration system using higherorder statistics and Radon transform
conference, January 1998
 Ibrahim, M. A.; Hussein, A. W. F.; Mashali, S. A.
 1998 IEEE International Conference on Electronics, Circuits and Systems. Surfing the Waves of Science and Technology (Cat. No.98EX196)
Tutorial on higherorder statistics (spectra) in signal processing and system theory: theoretical results and some applications
journal, March 1991
 Mendel, J. M.
 Proceedings of the IEEE, Vol. 79, Issue 3
Blind removal of image nonlinearities
conference, January 2001
 Faird, H.; Popescu, A. C.
 Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001
Mary Shift Keying Modulation Scheme Identification Algorithm Using Wavelet Transform and Higher Order Statistical Moment
journal, January 2008
 ., P. Prakasam; ., M. Madheswaran
 Journal of Applied Sciences, Vol. 8, Issue 1
Numerically stable, singlepass, parallel statistics algorithms
conference, August 2009
 Bennett, Janine; Grout, Ray; Pebay, Philippe
 2009 IEEE International Conference on Cluster Computing and Workshops
New criteria for blind deconvolution of nonminimum phase systems (channels)
journal, March 1990
 Shalvi, O.; Weinstein, E.
 IEEE Transactions on Information Theory, Vol. 36, Issue 2
Robust voice activity detection using higherorder statistics in the LPC residual domain
journal, March 2001
 Nemer, E.; Goubran, R.; Mahmoud, S.
 IEEE Transactions on Speech and Audio Processing, Vol. 9, Issue 3
Accurate Sum and Dot Product
journal, January 2005
 Ogita, Takeshi; Rump, Siegfried M.; Oishi, Shin'ichi
 SIAM Journal on Scientific Computing, Vol. 26, Issue 6
Updating mean and variance estimates: an improved method
journal, September 1979
 West, D. H. D.
 Communications of the ACM, Vol. 22, Issue 9
An adaptive orderstatistic noise filter for gammacorrected image sequences
journal, January 1997
 Kleihorst, R. P.; Lagendiik, R. L.; Biemond, J.
 IEEE Transactions on Image Processing, Vol. 6, Issue 10
How to Ensure a Faithful Polynomial Evaluation with the Compensated Horner Algorithm
conference, June 2007
 Langlois, Philippe; Louvet, Nicolas
 18th IEEE Symposium on Computer Arithmetic (ARITH '07)
Comparison of several algorithms for computation of means, standard deviations and correlation coefficients
journal, July 1966
 Neely, Peter M.
 Communications of the ACM, Vol. 9, Issue 7
Recursive estimation of fourthorder cumulants with application to identification
journal, July 1998
 Dembélé, Doulaye; Favier, Gérard
 Signal Processing, Vol. 68, Issue 2
A Blind Image Restoration Algorithm Based on Cumulants
conference, January 2006
 Wang, Hongzhi; Huo, Jinming; Song, Chunpeng
 2006 8th international Conference on Signal Processing
Identification and deconvolution of multichannel linear nonGaussian processes using higher order statistics and inverse filter criteria
journal, March 1997
 Tugnait, J. K.
 IEEE Transactions on Signal Processing, Vol. 45, Issue 3
Direction finding algorithms based on highorder statistics
journal, January 1991
 Porat, B.; Friedlander, B.
 IEEE Transactions on Signal Processing, Vol. 39, Issue 9
Performance of chromatic dispersion monitoring using statistical moments of asynchronously sampled waveform histograms
journal, May 2005
 Kikuchi, N.; Hayase, S.; Sekine, K.
 IEEE Photonics Technology Letters, Vol. 17, Issue 5
The Fundamental Approximation Theorem of Portfolio Analysis in terms of Means, Variances and Higher Moments
journal, October 1970
 Samuelson, P. A.
 The Review of Economic Studies, Vol. 37, Issue 4
Image blur identification by using higher order statistic techniques
conference, January 1996
 You Xu, ; Crebbin, G.
 Proceedings of 3rd IEEE International Conference on Image Processing