A bootstrapping approach to social media quantification
Abstract This work considers the use of classifiers in a downstream aggregation task estimating class proportions, such as estimating the percentage of reviews for a movie with positive sentiment. We derive the bias and variance of the class proportion estimator when taking classification error into account to determine how to best trade off different error types when tuning a classifier for these tasks. Additionally, we propose a method for constructing confidence intervals that correctly adjusts for classification error when estimating these statistics. We conduct experiments on four document classification tasks comparing our methods to prior approaches across classifier thresholds, sample sizes, and label distributions. Prior approaches have focused on providing the most accurate point estimate while this work focuses on the creation of correct confidence intervals that appropriately account for classifier error. Compared to the prior approaches, our methods provide lower error and more accurate confidence intervals.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE; USDOE National Nuclear Security Administration (NNSA)
- Grant/Contract Number:
- 89233218CNA000001
- OSTI ID:
- 1812428
- Report Number(s):
- LA-UR--20-20309; 73; PII: 760
- Journal Information:
- Social Network Analysis and Mining, Journal Name: Social Network Analysis and Mining Journal Issue: 1 Vol. 11; ISSN 1869-5450
- Publisher:
- Springer Science + Business MediaCopyright Statement
- Country of Publication:
- Austria
- Language:
- English
Similar Records
Sentiment analysis of the United States public support of nuclear power on social media using large language models
Structure classification and melting temperature prediction in octet AB solids via machine learning