Title: Comparison of Social Media, Syndromic Surveillance, and Microbiologic Acute Respiratory Infection Data: Observational Study

Abstract

Internet data can be used to improve infectious disease models. However, the representativeness and individual-level validity of internet-derived measures are largely unexplored as this requires ground truth data for study. This study sought to identify relationships between Web-based behaviors and/or conversation topics and health status using a ground truth, survey-based dataset. This study leveraged a unique dataset of self-reported surveys, microbiological laboratory tests, and social media data from the same individuals toward understanding the validity of individual-level constructs pertaining to influenza-like illness in social media data. Logistic regression models were used to identify illness in Twitter posts using user posting behaviors and topic model features extracted from users’ tweets. Of 396 original study participants, only 81 met the inclusion criteria for this study. Of these participants’ tweets, we identified only two instances that were related to health and occurred within 2 weeks (before or after) of a survey indicating symptoms. It was not possible to predict when participants reported symptoms using features derived from topic models (area under the curve [AUC]=0.51; P=.38), though it was possible using behavior features, albeit with a very small effect size (AUC=0.53; P≤.001). Individual symptoms were also generally not predictable either. The study sample andmore » a random sample from Twitter are predictably different on held-out data (AUC=0.67; P≤.001), meaning that the content posted by people who participated in this study was predictably different from that posted by random Twitter users. Individuals in the random sample and the GoViral sample used Twitter with similar frequencies (similar @ mentions, number of tweets, and number of retweets; AUC=0.50; P=.19). To our knowledge, this is the first instance of an attempt to use a ground truth dataset to validate infectious disease observations in social media data. The lack of signal, the lack of predictability among behaviors or topics, and the demonstrated volunteer bias in the study population are important findings for the large and growing body of disease surveillance using internet-sourced data.« less

Authors:

Daughton, Ashlynn R.  ^[1];

• Search DOE PAGES for author "Daughton, Ashlynn R."
• Search DOE PAGES for ORCID "0000-0003-1694-3268"

Chunara, Rumi  ^[2];

• Search DOE PAGES for author "Chunara, Rumi"
• Search DOE PAGES for ORCID "0000-0002-5346-7259"

Paul, Michael J.  ^[3]

• Search DOE PAGES for author "Paul, Michael J."
• Search DOE PAGES for ORCID "0000-0002-9149-7539"

---

+ Show Author Affiliations

1. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
2. New York Univ. (NYU), NY (United States)
3. Univ. of Colorado, Boulder, CO (United States)

Publication Date:

Fri Apr 24 00:00:00 EDT 2020

Research Org.:

Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Org.:

USDOE; National Science Foundation (NSF)

OSTI Identifier:

1716788

Report Number(s):

LA-UR-19-21141
Journal ID: ISSN 2369-2960

Grant/Contract Number:  

89233218CNA000001; IIS-1643576; IIS-1551036

Resource Type:

Accepted Manuscript

Journal Name:

JMIR Public Health and Surveillance

Additional Journal Information:

Journal Volume: 6; Journal Issue: 2; Journal ID: ISSN 2369-2960

Publisher:

JMIR Publications

Country of Publication:

United States

Language:

English

Subject:

59 BASIC BIOLOGICAL SCIENCES; social media; infodemiology; influenza; selection bias; bias; logistic models