Skip to main content
U.S. Department of Energy
Office of Scientific and Technical Information

Rounding errors in weighing

Journal Article · · Nucl. Mater. Manage.; (United States)
OSTI ID:7129654
When rounding error is large relative to weighing error, it cannot be ignored when estimating scale precision and bias from calibration data. Further, if the data grouping is coarse, rounding error is correlated with weighing error and may also have a mean quite different from zero. These facts are taken into account in a moment estimation method. A copy of the program listing for the MERDA program that provides moment estimates is available from the author. Experience suggests that if the data fall into four or more cells or groups, it is not necessary to apply the moment estimation method. Rather, the estimate given by equation (3) is valid in this instance. 5 tables.
OSTI ID:
7129654
Journal Information:
Nucl. Mater. Manage.; (United States), Journal Name: Nucl. Mater. Manage.; (United States) Vol. 5:2; ISSN NUMMB
Country of Publication:
United States
Language:
English