skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Imputation for multisource data with comparison and assessment techniques

Abstract

Missing data are prevalent issue in analyses involving data collection. The problem of missing data is exacerbated for multisource analysis, where data from multiple sensors are combined to arrive at a single conclusion. In this scenario, it is more likely to occur and can lead to discarding a large amount of data collected; however, the information from observed sensors can be leveraged to estimate those values not observed. We propose two methods for imputation of multisource data, both of which take advantage of potential correlation between data from different sensors, through ridge regression and a state-space model. These methods, as well as the common median imputation, are applied to data collected from a variety of sensors monitoring an experimental facility. Performance of imputation methods is compared with the mean absolute deviation; however, rather than using this metric to solely rank themethods,we also propose an approach to identify significant differences. Imputation techniqueswill also be assessed by their ability to produce appropriate confidence intervals, through coverage and length, around the imputed values. Finally, performance of imputed datasets is compared with a marginalized dataset through a weighted k-means clustering. In general, we found that imputation through a dynamic linearmodel tended to be themore » most accurate and to produce the most precise confidence intervals, and that imputing the missing values and down weighting them with respect to observed values in the analysis led to the most accurate performance.« less

Authors:
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]
  1. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1416279
Report Number(s):
LA-UR-17-23333
Journal ID: ISSN 1524-1904
Grant/Contract Number:  
AC52-06NA25396
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Applied Stochastic Models in Business and Industry
Additional Journal Information:
Journal Volume: 34; Journal Issue: 1; Journal ID: ISSN 1524-1904
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Imputation; assessment; multisource data; dynamic linear model; comparison; k-means clustering

Citation Formats

Casleton, Emily Michele, Osthus, David Allen, and Van Buren, Kendra Lu. Imputation for multisource data with comparison and assessment techniques. United States: N. p., 2017. Web. doi:10.1002/asmb.2299.
Casleton, Emily Michele, Osthus, David Allen, & Van Buren, Kendra Lu. Imputation for multisource data with comparison and assessment techniques. United States. doi:10.1002/asmb.2299.
Casleton, Emily Michele, Osthus, David Allen, and Van Buren, Kendra Lu. Wed . "Imputation for multisource data with comparison and assessment techniques". United States. doi:10.1002/asmb.2299.
@article{osti_1416279,
title = {Imputation for multisource data with comparison and assessment techniques},
author = {Casleton, Emily Michele and Osthus, David Allen and Van Buren, Kendra Lu},
abstractNote = {Missing data are prevalent issue in analyses involving data collection. The problem of missing data is exacerbated for multisource analysis, where data from multiple sensors are combined to arrive at a single conclusion. In this scenario, it is more likely to occur and can lead to discarding a large amount of data collected; however, the information from observed sensors can be leveraged to estimate those values not observed. We propose two methods for imputation of multisource data, both of which take advantage of potential correlation between data from different sensors, through ridge regression and a state-space model. These methods, as well as the common median imputation, are applied to data collected from a variety of sensors monitoring an experimental facility. Performance of imputation methods is compared with the mean absolute deviation; however, rather than using this metric to solely rank themethods,we also propose an approach to identify significant differences. Imputation techniqueswill also be assessed by their ability to produce appropriate confidence intervals, through coverage and length, around the imputed values. Finally, performance of imputed datasets is compared with a marginalized dataset through a weighted k-means clustering. In general, we found that imputation through a dynamic linearmodel tended to be the most accurate and to produce the most precise confidence intervals, and that imputing the missing values and down weighting them with respect to observed values in the analysis led to the most accurate performance.},
doi = {10.1002/asmb.2299},
journal = {Applied Stochastic Models in Business and Industry},
number = 1,
volume = 34,
place = {United States},
year = {Wed Dec 27 00:00:00 EST 2017},
month = {Wed Dec 27 00:00:00 EST 2017}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on December 27, 2018
Publisher's Version of Record

Save / Share: