Recommended reporting items for epidemic forecasting and prediction research: The EPIFORGE 2020 guidelines

Pollett, Simon; Johansson, Michael A.; Reich, Nicholas G.; Brett-Major, David; Del Valle, Sara Y.; Venkatramanan, Srinivasan; Lowe, Rachel; Porco, Travis; Berry, Irina Maljkovic; Deshpande, Alina; Kraemer, Moritz G.; Blazes, David L.; Pan-ngum, Wirichada; Vespigiani, Alessandro; Mate, Suzanne E.; Silal, Sheetal P.; Kandula, Sasikiran; Sippy, Rachel; Quandelacy, Talia M.; Morgan, Jeffrey J.; Ball, Jacob; Morton, Lindsay C.; Althouse, Benjamin M.; Pavlin, Julie; van Panhuis, Wilbert; Riley, Steven; Biggerstaff, Matthew; Viboud, Cecile; Brady, Oliver; Rivers, Caitlin

doi:10.1371/journal.pmed.1003793

Title: Recommended reporting items for epidemic forecasting and prediction research: The EPIFORGE 2020 guidelines

Journal Article · 18 October 2021 · PLoS Medicine

DOI: https://doi.org/10.1371/journal.pmed.1003793 · OSTI ID:1835770

Pollett, Simon ^[1];

^[2];

^[3];

^[4];

^[5];

^[6];

^[7];

^[8];

^[1]; Deshpande, Alina ^[5]; Kraemer, Moritz G. ^[9]; Blazes, David L. ^[10];

^[11]; Vespigiani, Alessandro ^[12]; Mate, Suzanne E. ^[1]; Silal, Sheetal P. ^[13];

^[14];

^[15]; Quandelacy, Talia M. ^[2];

^[16] more »

Walter Reed Army Institute of Research, Silver Springs, MD (United States)
Centers for Disease Control and Prevention (CDC), San Juan, PR (United States)
Univ. of Massachusetts, Amherst, MA (United States)
Univ. of Nebraska Medical Center, Omaha, NE (United States)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Univ. of Virginia, Charlottesville, VA (United States)
London School of Hygiene & Tropical Medicine (United Kingdom); Barcelona Institute for Global Health (Spain)
Univ. of California, San Francisco, CA (United States)
Univ. of Oxford (United Kingdom)
Bill and Melinda Gates Foundation, Seattle, WA (United States)
Mahidol University, Salaya (Thailand)
Northeastern Univ., Boston, MA (United States)
Univ. of Cape Town (South Africa); Univ. of Oxford (United Kingdom)
Columbia Univ., New York, NY (United States)
State Univ. of New York (SUNY), Syracuse, NY (United States)
The Catholic Univ. of America, Washington, DC (United States)
US Army Public Health Center, Edgewood, MD (United States)
Global Emerging Infections Surveillance, Silver Spring, MD (United States); George Washington Univ., Washington, DC (United States)
Univ. of Washington, Seattle, WA (United States); Institute for Disease Modeling, Bellevue, WA (United States); New Mexico State Univ., Las Cruces, NM (United States)
National Academies of Sciences, Engineering, and Medicine, Washington, DC (United States)
Univ. of Pittsburgh, PA (United States)
Imperial College, London (United Kingdom)
Centers for Disease Control and Prevention (CDC), Atlanta, GA (United States)
National Institutes of Health (NIH), Bethesda, MD (United States)
London School of Hygiene & Tropical Medicine (United Kingdom)
Johns Hopkins Univ., Baltimore, MD (United States)

Background: The importance of infectious disease epidemic forecasting and prediction research is underscored by decades of communicable disease outbreaks, including COVID-19. Unlike other fields of medical research, such as clinical trials and systematic reviews, no reporting guidelines exist for reporting epidemic forecasting and prediction research despite their utility. We therefore developed the EPIFORGE checklist, a guideline for standardized reporting of epidemic forecasting research. Methods and findings: We developed this checklist using a best-practice process for development of reporting guidelines, involving a Delphi process and broad consultation with an international panel of infectious disease modelers and model end users. The objectives of these guidelines are to improve the consistency, reproducibility, comparability, and quality of epidemic forecasting reporting. The guidelines are not designed to advise scientists on how to perform epidemic forecasting and prediction research, but rather to serve as a standard for reporting critical methodological details of such studies. Conclusions: These guidelines have been submitted to the EQUATOR network, in addition to hosting by other dedicated webpages to facilitate feedback and journal endorsement.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program; National Institutes of Health (NIH)

Grant/Contract Number:: 89233218CNA000001

OSTI ID:: 1835770

Report Number(s):: LA-UR--20-24531

Journal Information:: PLoS Medicine, Journal Name: PLoS Medicine Journal Issue: 10 Vol. 18; ISSN 1549-1676

Publisher:: Public Library of ScienceCopyright Statement

Country of Publication:: United States

Language:: English

References (44)

Model-based projections of Zika virus infections in childbearing women in the Americas Perkins, T. Alex; Siraj, Amir S.; Ruktanonchai, Corrine W. Nature Microbiology, Vol. 1, Issue 9 https://doi.org/10.1038/nmicrobiol.2016.126	journal	July 2016
Data-driven identification of potential Zika virus vectors Evans, Michelle V.; Dallas, Tad A.; Han, Barbara A. eLife, Vol. 6 https://doi.org/10.7554/eLife.22053	journal	February 2017
Projecting hospital utilization during the COVID-19 outbreaks in the United States Moghadas, Seyed M.; Shoukat, Affan; Fitzpatrick, Meagan C. Proceedings of the National Academy of Sciences, Vol. 117, Issue 16 https://doi.org/10.1073/pnas.2004064117	journal	April 2020
Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. Reich, Nicholas G.; McGowan, Craig J.; Yamana, Teresa K. PLOS Computational Biology, Vol. 15, Issue 11 https://doi.org/10.1371/journal.pcbi.1007486	journal	November 2019
On the origin and timing of Zika virus introduction in Brazil Massad, E.; Burattini, M. Nascimento; Khan, K. Epidemiology and Infection, Vol. 145, Issue 11 https://doi.org/10.1017/S0950268817001200	journal	June 2017
Reducing Unintended Pregnancies as a Strategy to Avert Zika-Related Microcephaly Births in the United States: A Simulation Study Ahrens, Katherine A.; Hutcheon, Jennifer A.; Gavin, Loretta Maternal and Child Health Journal, Vol. 21, Issue 5 https://doi.org/10.1007/s10995-017-2275-2	journal	January 2017
Real-time forecasting of infectious disease dynamics with a stochastic semi-mechanistic model Funk, Sebastian; Camacho, Anton; Kucharski, Adam J. Epidemics, Vol. 22 https://doi.org/10.1016/j.epidem.2016.11.003	journal	March 2018
Comparing Predictive Accuracy Diebold, Francis X.; Mariano, Roberto S. Journal of Business & Economic Statistics, Vol. 13, Issue 3 https://doi.org/10.1080/07350015.1995.10524599	journal	July 1995
A systematic review and evaluation of Zika virus forecasting and prediction research during a public health emergency of international concern Kobres, Pei-Ying; Chretien, Jean-Paul; Johansson, Michael A. PLOS Neglected Tropical Diseases, Vol. 13, Issue 10 https://doi.org/10.1371/journal.pntd.0007451	journal	October 2019
Summary results of the 2014-2015 DARPA Chikungunya challenge Del Valle, Sara Y.; McMahon, Benjamin H.; Asher, Jason BMC Infectious Diseases, Vol. 18, Issue 1 https://doi.org/10.1186/s12879-018-3124-7	journal	May 2018
Dengue outlook for the World Cup in Brazil: an early warning model framework driven by real-time seasonal climate forecasts Lowe, Rachel; Barcellos, Christovam; Coelho, Caio A. S. The Lancet Infectious Diseases, Vol. 14, Issue 7 https://doi.org/10.1016/S1473-3099(14)70781-9	journal	July 2014
The Logic of Scientific Discovery Popper, Karl R.; Weiss, George Physics Today, Vol. 12, Issue 11 https://doi.org/10.1063/1.3060577	journal	November 1959
Estimating Ebola Treatment Needs, United States Rainisch, Gabriel; Asher, Jason; George, Dylan Emerging Infectious Diseases, Vol. 21, Issue 7 https://doi.org/10.3201/eid2107.150286	journal	July 2015
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews Page, Matthew J.; McKenzie, Joanne E.; Bossuyt, Patrick M. figshare https://doi.org/10.6084/m9.figshare.c.5391881	collection	January 2021
Modelling for policy: The five principles of the Neglected Tropical Diseases Modelling Consortium Behrend, Matthew R.; Basáñez, María-Gloria; Hamley, Jonathan I. D. PLOS Neglected Tropical Diseases, Vol. 14, Issue 4 https://doi.org/10.1371/journal.pntd.0008033	journal	April 2020
Guidance for Developers of Health Research Reporting Guidelines Moher, David; Schulz, Kenneth F.; Simera, Iveta PLoS Medicine, Vol. 7, Issue 2 https://doi.org/10.1371/journal.pmed.1000217	journal	February 2010
Dynamic Transmission Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-5 Pitman, Richard; Fisman, David; Zaric, Gregory S. Value in Health, Vol. 15, Issue 6 https://doi.org/10.1016/j.jval.2012.06.011	journal	September 2012
Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period Kissler, Stephen M.; Tedijanto, Christine; Goldstein, Edward Science, Vol. 368, Issue 6493 https://doi.org/10.1126/science.abb5793	journal	April 2020
Regional Spread of Ebola Virus, West Africa, 2014 Rainisch, Gabriel; Shankar, Manjunath; Wellman, Michael Emerging Infectious Diseases, Vol. 21, Issue 3 https://doi.org/10.3201/eid2103.141845	journal	March 2015
An open challenge to advance probabilistic forecasting for dengue epidemics Johansson, Michael A.; Apfeldorf, Karyn M.; Dobson, Scott Proceedings of the National Academy of Sciences, Vol. 116, Issue 48 https://doi.org/10.1073/pnas.1909865116	journal	November 2019
Evaluating Google Flu Trends in Latin America: Important Lessons for the Next Phase of Digital Disease Detection Pollett, Simon; Boscardin, W. John; Azziz-Baumgartner, Eduardo Clinical Infectious Diseases, Vol. 64, Issue 1 https://doi.org/10.1093/cid/ciw657	journal	September 2016
Evaluating probabilistic dengue risk forecasts from a prototype early warning system for Brazil Lowe, Rachel; Coelho, Caio AS; Barcellos, Christovam eLife, Vol. 5 https://doi.org/10.7554/eLife.11285	journal	February 2016
Using “outbreak science” to strengthen the use of models during epidemics Rivers, Caitlin; Chretien, Jean-Paul; Riley, Steven Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-11067-2	journal	July 2019
Dynamic Bayesian Influenza Forecasting in the United States with Hierarchical Discrepancy (with Discussion) Osthus, Dave; Gattiker, James; Priedhorsky, Reid Bayesian Analysis, Vol. 14, Issue 1 https://doi.org/10.1214/18-BA1117	journal	March 2019
Global Disease Monitoring and Forecasting with Wikipedia Generous, Nicholas; Fairchild, Geoffrey; Deshpande, Alina PLoS Computational Biology, Vol. 10, Issue 11 https://doi.org/10.1371/journal.pcbi.1003892	journal	November 2014
Nonlinear and delayed impacts of climate on dengue risk in Barbados: A modelling study Lowe, Rachel; Gasparrini, Antonio; Van Meerbeeck, Cédric J. PLOS Medicine, Vol. 15, Issue 7 https://doi.org/10.1371/journal.pmed.1002613	journal	July 2018
Forecasting Zika Incidence in the 2016 Latin America Outbreak Combining Traditional Disease Surveillance with Search, Social Media, and News Report Data McGough, Sarah F.; Brownstein, John S.; Hawkins, Jared B. PLOS Neglected Tropical Diseases, Vol. 11, Issue 1 https://doi.org/10.1371/journal.pntd.0005295	journal	January 2017
Make Data Sharing Routine to Prepare for Public Health Emergencies Chretien, Jean-Paul; Rivers, Caitlin M.; Johansson, Michael A. PLOS Medicine, Vol. 13, Issue 8 https://doi.org/10.1371/journal.pmed.1002109	journal	August 2016
The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed Hopewell, S.; Dutton, S.; Yu, L. -M. BMJ, Vol. 340, Issue mar23 1 https://doi.org/10.1136/bmj.c723	journal	March 2010
Using data-driven agent-based models for forecasting emerging infectious diseases Venkatramanan, Srinivasan; Lewis, Bryan; Chen, Jiangzhuo Epidemics, Vol. 22 https://doi.org/10.1016/j.epidem.2017.02.010	journal	March 2018
Results from the second year of a collaborative effort to forecast influenza seasons in the United States Biggerstaff, Matthew; Johansson, Michael; Alper, David Epidemics, Vol. 24 https://doi.org/10.1016/j.epidem.2018.02.003	journal	September 2018
Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group–6 Briggs, Andrew H.; Weinstein, Milton C.; Fenwick, Elisabeth A. L. Medical Decision Making, Vol. 32, Issue 5 https://doi.org/10.1177/0272989X12458348	journal	September 2012
Identification and evaluation of epidemic prediction and forecasting reporting guidelines: A systematic review and a call for action Pollett, Simon; Johansson, Michael; Biggerstaff, Matthew Epidemics, Vol. 33 https://doi.org/10.1016/j.epidem.2020.100400	journal	December 2020
The use of mathematical modeling studies for evidence synthesis and guideline development: A glossary Porgo, Teegwendé V.; Norris, Susan L.; Salanti, Georgia Research Synthesis Methods, Vol. 10, Issue 1 https://doi.org/10.1002/jrsm.1333	journal	October 2018
Reassessing Google Flu Trends Data for Detection of Seasonal and Pandemic Influenza: A Comparative Epidemiological Study at Three Geographic Scales Olson, Donald R.; Konty, Kevin J.; Paladini, Marc PLoS Computational Biology, Vol. 9, Issue 10 https://doi.org/10.1371/journal.pcbi.1003256	journal	October 2013
Evidence informing the UK's COVID-19 public health response must be transparent Alwan, Nisreen A.; Bhopal, Raj; Burgess, Rochelle A. The Lancet, Vol. 395, Issue 10229 https://doi.org/10.1016/S0140-6736(20)30667-X	journal	March 2020
Collaborative efforts to forecast seasonal influenza in the United States, 2015–2016 McGowan, Craig J.; Biggerstaff , Matthew; Johansson, Michael Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-018-36361-9	journal	January 2019
Climate services for health: predicting the evolution of the 2016 dengue season in Machala, Ecuador Lowe, Rachel; Stewart-Ibarra, Anna M.; Petrova, Desislava The Lancet Planetary Health, Vol. 1, Issue 4 https://doi.org/10.1016/S2542-5196(17)30064-5	journal	July 2017
Prediction of infectious disease epidemics via weighted density ensembles Ray, Evan L.; Reich, Nicholas G. PLOS Computational Biology, Vol. 14, Issue 2 https://doi.org/10.1371/journal.pcbi.1005910	journal	February 2018
Probabilistic seasonal dengue forecasting in Vietnam: A modelling study using superensembles Colón-González, Felipe J.; Soares Bastos, Leonardo; Hofmann, Barbara PLOS Medicine, Vol. 18, Issue 3 https://doi.org/10.1371/journal.pmed.1003542	journal	March 2021
A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States Reich, Nicholas G.; Brooks, Logan C.; Fox, Spencer J. Proceedings of the National Academy of Sciences, Vol. 116, Issue 8 https://doi.org/10.1073/pnas.1812594116	journal	January 2019
Predicting the Epidemic Sizes of Influenza A/H1N1, A/H3N2, and B: A Statistical Method Goldstein, Edward; Cobey, Sarah; Takahashi, Saki PLoS Medicine, Vol. 8, Issue 7 https://doi.org/10.1371/journal.pmed.1001051	journal	July 2011
Something for everyone Shailes, Sarah eLife, Vol. 6 https://doi.org/10.7554/eLife.25411	journal	March 2017
Modeling Good Research Practices—Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force–1 Caro, J. Jaime; Briggs, Andrew H.; Siebert, Uwe Medical Decision Making, Vol. 32, Issue 5 https://doi.org/10.1177/0272989X12454577	journal	September 2012

Similar Records

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Journal Article · 2023 · eLife · OSTI ID:1971178

Sherratt, Katharine; Gruson, Hugo; Grah, Rok; +126 more

Forecasting Multi-Wave Epidemics Through Bayesian Inference

Journal Article · 2021 · Archives of Computational Methods in Engineering · OSTI ID:1810763

Blonigan, Patrick; Ray, Jaideep; Safta, Cosmin

Deriving a model for influenza epidemics from historical data.

Technical Report · 2011 · OSTI ID:1030332

Ray, Jaideep; Lefantzi, Sophia

Related Subjects

60 APPLIED LIFE SCIENCES
Computer Science
Mathematics

Title: Recommended reporting items for epidemic forecasting and prediction research: The EPIFORGE 2020 guidelines

Citation Formats

References (44)

Similar Records

Related Subjects