Title: EPIsembleVis: A geo-visual analysis and comparison of the prediction ensembles of multiple COVID-19 models

Abstract

In this work, we present EPIsembleVis, a web-based comparative visual analysis tool for evaluating the consistency of multiple COVID-19 prediction models. Our approach analyzes a collection of COVID-19 predictions from different epidemiological models as an ensemble and utilizes two metrics to quantify model performance. These metrics include (a) prediction uncertainty (represented as the dispersion of predictions in each ensemble) and (b) prediction error (calculated by comparing individual model predictions with the recorded data). Through an interactive visual interface, our approach provides a data-driven workflow for (a) selecting and constructing the COVID-19 model prediction ensemble based on the spatiotemporal overlap of available predictions of multiple epidemiological models, (b) quantifying the model performance using both the uncertainty of each model prediction ensemble, and the error of each ensemble member that represents individual model predictions, and (c) visualizing the spatiotemporal variability in the projection performance of individual models using a suite of novel ensemble visualization techniques, such as the data availability map, a spatiotemporal textured-tile calendar, multivariate rose chart, and time-series leaflet glyph. We demonstrate the capability of our ensemble visual interface through a case study that investigates the performance of weekly COVID-19 predictions, which are provided through the COVID-19 Forecast Hub UMass-Amherstmore » Influenza Forecasting Center of Excellence [47] for the United States and United States Territories. The EPIsembleVis tool is implemented using open-source web technologies and adaptive system design, rendering it interoperable with Elasticsearch and Kibana for automatically ingesting COVID-19 predictions from online repositories, and it is generalizable for analyzing worldwide projections from more epidemiological models.« less

Authors:

Xu, Haowen  ^[1];

• Search DOE PAGES for author "Xu, Haowen"
• Search DOE PAGES for ORCID "0000-0002-2309-9176"
• Search orcid.org for ORCID "0000-0002-2309-9176"

Berres, Andy  ^[1];

• Search DOE PAGES for author "Berres, Andy"
• Search DOE PAGES for ORCID "0000-0002-6048-4313"
• Search orcid.org for ORCID "0000-0002-6048-4313"

Thakur, Gautam  ^[1];

• Search DOE PAGES for author "Thakur, Gautam"
• Search DOE PAGES for ORCID "0000-0002-8341-4596"
• Search orcid.org for ORCID "0000-0002-8341-4596"

Sanyal, Jibo  ^[1];

• Search DOE PAGES for author "Sanyal, Jibo"
• Search DOE PAGES for ORCID "0000-0002-7789-3199"
• Search orcid.org for ORCID "0000-0002-7789-3199"

Chinthavali, Supriya  ^[1]

• Search DOE PAGES for author "Chinthavali, Supriya"
• Search DOE PAGES for ORCID "0000-0002-4611-1086"
• Search orcid.org for ORCID "0000-0002-4611-1086"

---

+ Show Author Affiliations

1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Publication Date:

2021-11-01

Research Org.:

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:

USDOE

OSTI Identifier:

1830131

Grant/Contract Number:  

AC05-00OR22725

Resource Type:

Accepted Manuscript

Journal Name:

Journal of Biomedical Informatics

Additional Journal Information:

Journal Volume: 124; Journal Issue: 12; Journal ID: ISSN 1532-0464

Publisher:

Elsevier

Country of Publication:

United States

Language:

English

Subject:

60 APPLIED LIFE SCIENCES; Geographic visualization; Spatiotemporal; Ensemble visualization; Web-based; Epidemiological models; Multivariate; Health geography; COVID-19; COVID-19 data ontology