Effects of measurement errors on linear calibration equations
This report summarizes the results of the developmental work in the area of statistical treatment of calibration data performed in the preparation of American National Standard N15.19. The effects of measurement errors in both the dependent and independent variables in linear calibrations are studied by examining (1) a simulation model in which the lack of fit was zero with assigned known values of the measurement errors, (2) data from a laboratory calibration of a process vessel where the measurement errors are small compared to the lack of fit, (3) data from a process vessel calibration where the lack of fit can be removed through the use of a cubic equation, and (4) data from a process vessel calibration where the major error appears to be in the dependent variable. The effect of measurement errors on the estimation of the standard deviation of the slope is found to be highly dependent on the experimental design used and on the subsequent treatment of the data.
- Research Organization:
- Brookhaven National Lab. (BNL), Upton, NY (United States)
- DOE Contract Number:
- EY-76-C-02-0016
- OSTI ID:
- 7309204
- Report Number(s):
- BNL-50432; TRN: 77-011749
- Country of Publication:
- United States
- Language:
- English
Similar Records
Use of a non-linear method for including the mass uncertainty of gravimetric standards and system measurement errors in the fitting of calibration curves for XRFA freeze-dried UNO/sub 3/ standards
A Single Model Procedure for Estimating Tank Calibration Equations