Algorithms for Efficient Reproducible Floating Point Summation
- Massachusetts Institute of Technology, Cambridge, MA, USA
- University of California Berkeley, Berkeley, CA, USA
We define “reproducibility” as getting bitwise identical results from multiple runs of the same program, perhaps with different hardware resources or other changes that should not affect the answer. Many users depend on reproducibility for debugging or correctness. However, dynamic scheduling of parallel computing resources, combined with nonassociative floating point addition, makes reproducibility challenging even for summation, or operations like the BLAS. We describe a “reproducible accumulator” data structure (the “binned number”) and associated algorithms to reproducibly sum binary floating point numbers, independent of summation order. We use a subset of the IEEE Floating Point Standard 754-2008 and bitwise operations on the standard representations in memory. Our approach requires only one read-only pass over the data, and one reduction in parallel, using a 6-word reproducible accumulator (more words can be used for higher accuracy), enabling standard tiling optimization techniques. Summingnwords with a 6-word reproducible accumulator requires approximately 9nfloating point operations (arithmetic, comparison, and absolute value) and approximately 3nbitwise operations. The final error bound with a 6-word reproducible accumulator and our default settings can be up to 229times smaller than the error bound for conventional (recursive) summation on ill-conditioned double-precision inputs.
- Research Organization:
- Krell Institute, Ames, IA (United States); Univ. of California, Oakland, CA (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC)
- DOE Contract Number:
- FG02-97ER25308; SC0008699; SC0008700; SC0010200; AC02-05CH11231
- OSTI ID:
- 1800924
- Journal Information:
- ACM Transactions on Mathematical Software, Vol. 46, Issue 3; ISSN 0098-3500
- Publisher:
- Association for Computing Machinery
- Country of Publication:
- United States
- Language:
- English
Similar Records
Using accurate arithmetics to improve numerical reproducibility and stability in parallel applications
Using accurate arithmetics to improve numerical reproducibility and stability in parallel applications