Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Abstract

Background: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance. Results: Over 52 weeks,more » we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks. Funding: AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 ( https://www.nfdi4health.de/task-force-covid-19-2 ) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).« less

Authors:
ORCiD logo; ; ; ; ; ; ; ; ORCiD logo; ; ; ; ; ; ; ORCiD logo; ; ; ORCiD logo; more »; ORCiD logo; ; ; ; ; ; ; ; ORCiD logo; ; ORCiD logo; ; ; ; ; ; ; ; ; ORCiD logo; ; ; ORCiD logo; ; ; ; ; ; ; ; ; ; ORCiD logo; ORCiD logo; ; ; ; ; ; ; ; ; ORCiD logo; ; ; ; ; ; ; ; ; ; ; ; ; ORCiD logo; ; ; ORCiD logo; ORCiD logo; ; ; ; ; ; ; ; ; ; ; ; ; ; ORCiD logo; ; ; ; ; ORCiD logo; ; ORCiD logo; ; ORCiD logo; ; ; ORCiD logo; ; ; ; ; ; ; ; ; ; ORCiD logo; ; ; ; ; ; ; ; ; ; ; ; ; ORCiD logo « less
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA); USDOE Laboratory Directed Research and Development (LDRD) Program; National Institute of General Medical Sciences (NIGMS); National Institutes of Health (NIH); Federal Ministry of Education and Research (BMBF)
OSTI Identifier:
1971178
Alternate Identifier(s):
OSTI ID: 1971179; OSTI ID: 1975030
Report Number(s):
LA-UR-22-27431
Journal ID: ISSN 2050-084X; e81916
Grant/Contract Number:  
Mathematical and Statistical modelling project (MUNI/A/1615/2020),MUNI/11/02202001/2020; 89233218CNA000001; R35GM119582; 05M18SIA; NIHR200908
Resource Type:
Published Article
Journal Name:
eLife
Additional Journal Information:
Journal Name: eLife Journal Volume: 12; Journal ID: ISSN 2050-084X
Publisher:
eLife Sciences Publications, Ltd.
Country of Publication:
United States
Language:
English
Subject:
60 APPLIED LIFE SCIENCES

Citation Formats

Sherratt, Katharine, Gruson, Hugo, Grah, Rok, Johnson, Helen, Niehus, Rene, Prasse, Bastian, Sandmann, Frank, Deuschel, Jannik, Wolffram, Daniel, Abbott, Sam, Ullrich, Alexander, Gibson, Graham, Ray, Evan L., Reich, Nicholas G., Sheldon, Daniel, Wang, Yijin, Wattanachit, Nutcha, Wang, Lijing, Trnka, Jan, Obozinski, Guillaume, Sun, Tao, Thanou, Dorina, Pottier, Loic, Krymova, Ekaterina, Meinke, Jan H., Barbarossa, Maria Vittoria, Leithauser, Neele, Mohring, Jan, Schneider, Johanna, Wlazlo, Jaroslaw, Fuhrmann, Jan, Lange, Berit, Rodiah, Isti, Baccam, Prasith, Gurung, Heidi, Stage, Steven, Suchoski, Bradley, Budzinski, Jozef, Walraven, Robert, Villanueva, Inmaculada, Tucek, Vit, Smid, Martin, Zajicek, Milan, Perez Alvarez, Cesar, Reina, Borja, Bosse, Nikos I., Meakin, Sophie R., Castro, Lauren, Fairchild, Geoffrey, Michaud, Isaac, Osthus, Dave, Alaimo Di Loro, Pierfrancesco, Maruotti, Antonello, Eclerova, Veronika, Kraus, Andrea, Kraus, David, Pribylova, Lenka, Dimitris, Bertsimas, Li, Michael Lingzhi, Saksham, Soni, Dehning, Jonas, Mohr, Sebastian, Priesemann, Viola, Redlarski, Grzegorz, Bejar, Benjamin, Ardenghi, Giovanni, Parolini, Nicola, Ziarelli, Giovanni, Bock, Wolfgang, Heyder, Stefan, Hotz, Thomas, Singh, David E., Guzman-Merino, Miguel, Aznarte, Jose L., Morina, David, Alonso, Sergio, Alvarez, Enric, Lopez, Daniel, Prats, Clara, Burgard, Jan Pablo, Rodloff, Arne, Zimmermann, Tom, Kuhlmann, Alexander, Zibert, Janez, Pennoni, Fulvia, Divino, Fabio, Catala, Marti, Lovison, Gianfranco, Giudici, Paolo, Tarantino, Barbara, Bartolucci, Francesco, Jona Lasinio, Giovanna, Mingione, Marco, Farcomeni, Alessio, Srivastava, Ajitesh, Montero-Manso, Pablo, Adiga, Aniruddha, Hurt, Benjamin, Lewis, Bryan, Marathe, Madhav, Porebski, Przemyslaw, Venkatramanan, Srinivasan, Bartczuk, Rafal P., Dreger, Filip, Gambin, Anna, Gogolewski, Krzysztof, Gruziel-Slomka, Magdalena, Krupa, Bartosz, Moszyński, Antoni, Niedzielewski, Karol, Nowosielski, Jedrzej, Radwan, Maciej, Rakowski, Franciszek, Semeniuk, Marcin, Szczurek, Ewa, Zielinski, Jakub, Kisielewski, Jan, Pabjan, Barbara, Holger, Kirsten, Kheifetz, Yuri, Scholz, Markus, Przemyslaw, Biecek, Bodych, Marcin, Filinski, Maciej, Idzikowski, Radoslaw, Krueger, Tyll, Ozanski, Tomasz, Bracher, Johannes, and Funk, Sebastian. Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations. United States: N. p., 2023. Web. doi:10.7554/eLife.81916.
Copy to clipboard
Sherratt, Katharine, Gruson, Hugo, Grah, Rok, Johnson, Helen, Niehus, Rene, Prasse, Bastian, Sandmann, Frank, Deuschel, Jannik, Wolffram, Daniel, Abbott, Sam, Ullrich, Alexander, Gibson, Graham, Ray, Evan L., Reich, Nicholas G., Sheldon, Daniel, Wang, Yijin, Wattanachit, Nutcha, Wang, Lijing, Trnka, Jan, Obozinski, Guillaume, Sun, Tao, Thanou, Dorina, Pottier, Loic, Krymova, Ekaterina, Meinke, Jan H., Barbarossa, Maria Vittoria, Leithauser, Neele, Mohring, Jan, Schneider, Johanna, Wlazlo, Jaroslaw, Fuhrmann, Jan, Lange, Berit, Rodiah, Isti, Baccam, Prasith, Gurung, Heidi, Stage, Steven, Suchoski, Bradley, Budzinski, Jozef, Walraven, Robert, Villanueva, Inmaculada, Tucek, Vit, Smid, Martin, Zajicek, Milan, Perez Alvarez, Cesar, Reina, Borja, Bosse, Nikos I., Meakin, Sophie R., Castro, Lauren, Fairchild, Geoffrey, Michaud, Isaac, Osthus, Dave, Alaimo Di Loro, Pierfrancesco, Maruotti, Antonello, Eclerova, Veronika, Kraus, Andrea, Kraus, David, Pribylova, Lenka, Dimitris, Bertsimas, Li, Michael Lingzhi, Saksham, Soni, Dehning, Jonas, Mohr, Sebastian, Priesemann, Viola, Redlarski, Grzegorz, Bejar, Benjamin, Ardenghi, Giovanni, Parolini, Nicola, Ziarelli, Giovanni, Bock, Wolfgang, Heyder, Stefan, Hotz, Thomas, Singh, David E., Guzman-Merino, Miguel, Aznarte, Jose L., Morina, David, Alonso, Sergio, Alvarez, Enric, Lopez, Daniel, Prats, Clara, Burgard, Jan Pablo, Rodloff, Arne, Zimmermann, Tom, Kuhlmann, Alexander, Zibert, Janez, Pennoni, Fulvia, Divino, Fabio, Catala, Marti, Lovison, Gianfranco, Giudici, Paolo, Tarantino, Barbara, Bartolucci, Francesco, Jona Lasinio, Giovanna, Mingione, Marco, Farcomeni, Alessio, Srivastava, Ajitesh, Montero-Manso, Pablo, Adiga, Aniruddha, Hurt, Benjamin, Lewis, Bryan, Marathe, Madhav, Porebski, Przemyslaw, Venkatramanan, Srinivasan, Bartczuk, Rafal P., Dreger, Filip, Gambin, Anna, Gogolewski, Krzysztof, Gruziel-Slomka, Magdalena, Krupa, Bartosz, Moszyński, Antoni, Niedzielewski, Karol, Nowosielski, Jedrzej, Radwan, Maciej, Rakowski, Franciszek, Semeniuk, Marcin, Szczurek, Ewa, Zielinski, Jakub, Kisielewski, Jan, Pabjan, Barbara, Holger, Kirsten, Kheifetz, Yuri, Scholz, Markus, Przemyslaw, Biecek, Bodych, Marcin, Filinski, Maciej, Idzikowski, Radoslaw, Krueger, Tyll, Ozanski, Tomasz, Bracher, Johannes, & Funk, Sebastian. Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations. United States. https://doi.org/10.7554/eLife.81916
Copy to clipboard
Sherratt, Katharine, Gruson, Hugo, Grah, Rok, Johnson, Helen, Niehus, Rene, Prasse, Bastian, Sandmann, Frank, Deuschel, Jannik, Wolffram, Daniel, Abbott, Sam, Ullrich, Alexander, Gibson, Graham, Ray, Evan L., Reich, Nicholas G., Sheldon, Daniel, Wang, Yijin, Wattanachit, Nutcha, Wang, Lijing, Trnka, Jan, Obozinski, Guillaume, Sun, Tao, Thanou, Dorina, Pottier, Loic, Krymova, Ekaterina, Meinke, Jan H., Barbarossa, Maria Vittoria, Leithauser, Neele, Mohring, Jan, Schneider, Johanna, Wlazlo, Jaroslaw, Fuhrmann, Jan, Lange, Berit, Rodiah, Isti, Baccam, Prasith, Gurung, Heidi, Stage, Steven, Suchoski, Bradley, Budzinski, Jozef, Walraven, Robert, Villanueva, Inmaculada, Tucek, Vit, Smid, Martin, Zajicek, Milan, Perez Alvarez, Cesar, Reina, Borja, Bosse, Nikos I., Meakin, Sophie R., Castro, Lauren, Fairchild, Geoffrey, Michaud, Isaac, Osthus, Dave, Alaimo Di Loro, Pierfrancesco, Maruotti, Antonello, Eclerova, Veronika, Kraus, Andrea, Kraus, David, Pribylova, Lenka, Dimitris, Bertsimas, Li, Michael Lingzhi, Saksham, Soni, Dehning, Jonas, Mohr, Sebastian, Priesemann, Viola, Redlarski, Grzegorz, Bejar, Benjamin, Ardenghi, Giovanni, Parolini, Nicola, Ziarelli, Giovanni, Bock, Wolfgang, Heyder, Stefan, Hotz, Thomas, Singh, David E., Guzman-Merino, Miguel, Aznarte, Jose L., Morina, David, Alonso, Sergio, Alvarez, Enric, Lopez, Daniel, Prats, Clara, Burgard, Jan Pablo, Rodloff, Arne, Zimmermann, Tom, Kuhlmann, Alexander, Zibert, Janez, Pennoni, Fulvia, Divino, Fabio, Catala, Marti, Lovison, Gianfranco, Giudici, Paolo, Tarantino, Barbara, Bartolucci, Francesco, Jona Lasinio, Giovanna, Mingione, Marco, Farcomeni, Alessio, Srivastava, Ajitesh, Montero-Manso, Pablo, Adiga, Aniruddha, Hurt, Benjamin, Lewis, Bryan, Marathe, Madhav, Porebski, Przemyslaw, Venkatramanan, Srinivasan, Bartczuk, Rafal P., Dreger, Filip, Gambin, Anna, Gogolewski, Krzysztof, Gruziel-Slomka, Magdalena, Krupa, Bartosz, Moszyński, Antoni, Niedzielewski, Karol, Nowosielski, Jedrzej, Radwan, Maciej, Rakowski, Franciszek, Semeniuk, Marcin, Szczurek, Ewa, Zielinski, Jakub, Kisielewski, Jan, Pabjan, Barbara, Holger, Kirsten, Kheifetz, Yuri, Scholz, Markus, Przemyslaw, Biecek, Bodych, Marcin, Filinski, Maciej, Idzikowski, Radoslaw, Krueger, Tyll, Ozanski, Tomasz, Bracher, Johannes, and Funk, Sebastian. Fri . "Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations". United States. https://doi.org/10.7554/eLife.81916.
Copy to clipboard
@article{osti_1971178,
title = {Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations},
author = {Sherratt, Katharine and Gruson, Hugo and Grah, Rok and Johnson, Helen and Niehus, Rene and Prasse, Bastian and Sandmann, Frank and Deuschel, Jannik and Wolffram, Daniel and Abbott, Sam and Ullrich, Alexander and Gibson, Graham and Ray, Evan L. and Reich, Nicholas G. and Sheldon, Daniel and Wang, Yijin and Wattanachit, Nutcha and Wang, Lijing and Trnka, Jan and Obozinski, Guillaume and Sun, Tao and Thanou, Dorina and Pottier, Loic and Krymova, Ekaterina and Meinke, Jan H. and Barbarossa, Maria Vittoria and Leithauser, Neele and Mohring, Jan and Schneider, Johanna and Wlazlo, Jaroslaw and Fuhrmann, Jan and Lange, Berit and Rodiah, Isti and Baccam, Prasith and Gurung, Heidi and Stage, Steven and Suchoski, Bradley and Budzinski, Jozef and Walraven, Robert and Villanueva, Inmaculada and Tucek, Vit and Smid, Martin and Zajicek, Milan and Perez Alvarez, Cesar and Reina, Borja and Bosse, Nikos I. and Meakin, Sophie R. and Castro, Lauren and Fairchild, Geoffrey and Michaud, Isaac and Osthus, Dave and Alaimo Di Loro, Pierfrancesco and Maruotti, Antonello and Eclerova, Veronika and Kraus, Andrea and Kraus, David and Pribylova, Lenka and Dimitris, Bertsimas and Li, Michael Lingzhi and Saksham, Soni and Dehning, Jonas and Mohr, Sebastian and Priesemann, Viola and Redlarski, Grzegorz and Bejar, Benjamin and Ardenghi, Giovanni and Parolini, Nicola and Ziarelli, Giovanni and Bock, Wolfgang and Heyder, Stefan and Hotz, Thomas and Singh, David E. and Guzman-Merino, Miguel and Aznarte, Jose L. and Morina, David and Alonso, Sergio and Alvarez, Enric and Lopez, Daniel and Prats, Clara and Burgard, Jan Pablo and Rodloff, Arne and Zimmermann, Tom and Kuhlmann, Alexander and Zibert, Janez and Pennoni, Fulvia and Divino, Fabio and Catala, Marti and Lovison, Gianfranco and Giudici, Paolo and Tarantino, Barbara and Bartolucci, Francesco and Jona Lasinio, Giovanna and Mingione, Marco and Farcomeni, Alessio and Srivastava, Ajitesh and Montero-Manso, Pablo and Adiga, Aniruddha and Hurt, Benjamin and Lewis, Bryan and Marathe, Madhav and Porebski, Przemyslaw and Venkatramanan, Srinivasan and Bartczuk, Rafal P. and Dreger, Filip and Gambin, Anna and Gogolewski, Krzysztof and Gruziel-Slomka, Magdalena and Krupa, Bartosz and Moszyński, Antoni and Niedzielewski, Karol and Nowosielski, Jedrzej and Radwan, Maciej and Rakowski, Franciszek and Semeniuk, Marcin and Szczurek, Ewa and Zielinski, Jakub and Kisielewski, Jan and Pabjan, Barbara and Holger, Kirsten and Kheifetz, Yuri and Scholz, Markus and Przemyslaw, Biecek and Bodych, Marcin and Filinski, Maciej and Idzikowski, Radoslaw and Krueger, Tyll and Ozanski, Tomasz and Bracher, Johannes and Funk, Sebastian},
abstractNote = {Background: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance. Results: Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks. Funding: AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 ( https://www.nfdi4health.de/task-force-covid-19-2 ) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).},
doi = {10.7554/eLife.81916},
journal = {eLife},
number = ,
volume = 12,
place = {United States},
year = {Fri Apr 21 00:00:00 EDT 2023},
month = {Fri Apr 21 00:00:00 EDT 2023}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.7554/eLife.81916
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Comparing Predictive Accuracy
journal, July 1995

The RAPIDD ebola forecasting challenge: Synthesis and lessons learnt
journal, March 2018

Three Ways in Which Pandemic Models May Perform a Pandemic
journal, July 2021

  • Van Basshuysen, Philippe; White, Lucie; Khosrowi, Donal
  • Erasmus Journal for Philosophy and Economics, Vol. 14, Issue 1
  • DOI: 10.23941/ejpe.v14i1.582

A Poisson Autoregressive Model to Understand COVID-19 Contagion Dynamics
journal, July 2020

Vincentization Revisited
journal, June 1992

Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S.
journal, November 2019

Robust estimation of diagnostic rate and real incidence of COVID-19 for European policymakers
journal, January 2021

An open challenge to advance probabilistic forecasting for dengue epidemics
journal, November 2019

  • Johansson, Michael A.; Apfeldorf, Karyn M.; Dobson, Scott
  • Proceedings of the National Academy of Sciences, Vol. 116, Issue 48
  • DOI: 10.1073/pnas.1909865116

A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States
journal, January 2019

  • Reich, Nicholas G.; Brooks, Logan C.; Fox, Spencer J.
  • Proceedings of the National Academy of Sciences, Vol. 116, Issue 8
  • DOI: 10.1073/pnas.1812594116

Monitoring COVID‐19 contagion growth
journal, May 2021

  • Agosto, Arianna; Campmas, Alexandra; Giudici, Paolo
  • Statistics in Medicine, Vol. 40, Issue 18
  • DOI: 10.1002/sim.9020

Serial interval and incubation period of COVID-19: a systematic review and meta-analysis
journal, March 2021

  • Alene, Muluneh; Yismaw, Leltework; Assemie, Moges Agazhe
  • BMC Infectious Diseases, Vol. 21, Issue 1
  • DOI: 10.1186/s12879-021-05950-x

Predictive performance of international COVID-19 mortality forecasting models
journal, May 2021

Modeling of Future COVID-19 Cases, Hospitalizations, and Deaths, by Vaccination Rates and Nonpharmaceutical Intervention Scenarios — United States, April–September 2021
journal, May 2021

  • Borchering, Rebecca K.; Viboud, Cécile; Howerton, Emily
  • MMWR. Morbidity and Mortality Weekly Report, Vol. 70, Issue 19
  • DOI: 10.15585/mmwr.mm7019e3

Evaluating epidemic forecasts in an interval format
journal, February 2021

A pre-registered short-term forecasting study of COVID-19 in Germany and Poland during the second wave
journal, August 2021

Accounting for uncertainty during a pandemic
journal, August 2021

An interactive web-based dashboard to track COVID-19 in real time
journal, May 2020

The Lag between Daily Reported Covid-19 Cases and Deaths and Its Relationship to Age
journal, June 2021

Strictly Proper Scoring Rules, Prediction, and Estimation
journal, March 2007

  • Gneiting, Tilmann; Raftery, Adrian E.
  • Journal of the American Statistical Association, Vol. 102, Issue 477
  • DOI: 10.1198/016214506000001437

Modelling the COVID-19 pandemic in context: an international participatory approach
journal, December 2020

A new distribution-free quantile estimator
journal, January 1982

The turning point and end of an expanding epidemic cannot be precisely forecast
journal, October 2020

  • Castro, Mario; Ares, Saúl; Cuesta, José A.
  • Proceedings of the National Academy of Sciences, Vol. 117, Issue 42
  • DOI: 10.1073/pnas.2007868117

European Covid-19 Forecast Hub
dataset, January 2022

Epidemic Forecasting is Messier Than Weather Forecasting: The Role of Human Behavior and Internet Data Streams in Epidemic Forecast
journal, November 2016

  • Moran, Kelly R.; Fairchild, Geoffrey; Generous, Nicholas
  • Journal of Infectious Diseases, Vol. 214, Issue suppl 4
  • DOI: 10.1093/infdis/jiw375

Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States
journal, July 2022

Combining probabilistic forecasts of COVID-19 mortality in the United States
journal, June 2021

The Use and Misuse of Mathematical Modeling for Infectious Disease Policymaking: Lessons for the COVID-19 Pandemic
journal, February 2021

  • James, Lyndon P.; Salomon, Joshua A.; Buckee, Caroline O.
  • Medical Decision Making, Vol. 41, Issue 4
  • DOI: 10.1177/0272989X21990391

Introduction to the special issue on “25 years of ensemble forecasting”
journal, October 2018

  • Buizza, Roberto
  • Quarterly Journal of the Royal Meteorological Society, Vol. 145, Issue S1
  • DOI: 10.1002/qj.3370

The Zoltar forecast archive, a tool to standardize and store interdisciplinary prediction research
journal, February 2021

    Sort by:
    [ × clear filter / sort ]

    Similar Records in DOE PAGES and OSTI.GOV collections: