Research - COVID 19 forecast hub

Citing the Forecast Hub

To cite the US COVID-19 Forecast Hub dataset and project as a whole, please cite the dataset descriptor preprint:
Cramer EY, Huang Y, Wang Y, et al. “The United States COVID-19 Forecast Hub dataset.” 2021. medRxiv. (preprint)

bibtex:

@article {Cramer2021-hub-dataset,
	author = {Cramer, Estee Y and Huang, Yuxin and Wang, Yijin and Ray, Evan L and Cornell, Matthew and Bracher, Johannes and Brennen, Andrea and Castro Rivadeneira, Alvaro J and Gerding, Aaron and House, Katie and Jayawardena, Dasuni and Kanji, Abdul H and Khandelwal, Ayush and Le, Khoa and Niemi, Jarad and Stark, Ariane and Shah, Apurv and Wattanachit, Nutcha and Zorn, Martha W and Reich, Nicholas G and US COVID-19 Forecast Hub Consortium},
	title = {The United States COVID-19 Forecast Hub dataset},
	year = {2021},
	doi = {10.1101/2021.11.04.21265886},
	URL = {https://www.medrxiv.org/content/10.1101/2021.11.04.21265886v1},
	journal = {medRxiv}
}

To cite the dataset and GitHub repository directly, we ask that you cite the Data Descriptor paper (see above) but you may also cite or refer to the permanent DOI for the GitHub repo (the DOI is updated by Zenodo when we create a new “release” of this GitHub repository).

To cite research results from the hub, please review the list below and choose the relevant research publication(s).

Research Background

Since 2016, our team has worked closely with the US CDC on forecasting seasonal influenza outbreaks in a project called FluSight. This collaborative research framework has served as a model for setting up the COVID-19 Forecast Hub. Below, we list publications that have emerged from the collaborative work of the COVID-19 Forecast Hub research team, as well as pre-COVID publications.

COVID-19 Forecast Hub publications

Reich NG, Wang Y, Burns M, Ergas R, Cramer EY, Ray EL. “Assessing the utility of COVID-19 case reports as a leading indicator for hospitalization forecasting in the United States” medRxiv. 2023. (preprint).
McAndrew T, Reich NG. “An expert judgment model to predict early stages of the COVID-19 pandemic in the United States.” PLOS Comp Bio. 2022. 18(9): e1010485.
Nixon K, Jindal S, Parker F, Reich NG, Ghobadi K, Lee EC, Truelove S, Gardner L. “An evaluation of prospective COVID-19 modelling studies in the USA: from data to science translation.” The Lancet Digital Health. 2022. 4(10): e738-e747.
Nixon K, Jindal S, Parker F, Marshall M, Reich NG, Ghobadi K, Lee EC, Truelove S, Gardner L. “Real-time COVID-19 forecasting: challenges and opportunities of model performance and translation.” The Lancet Digital Health. 2022. 4(10): e699–e701.
Ray EL, Brooks LC, Bien J, Biggerstaff M, Bosse NI, Bracher J, Cramer EY, Funk S, Gerding A, Johansson MA, Rumack A, Wang Y, Zorn M, Tibshirani RJ, Reich NG. “Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States.” International Journal of Forecasting. 2022.
Cramer EY, Huang Y, Wang Y, et al. “The United States COVID-19 Forecast Hub dataset.” Scientific Data. 2022. 9(1): 1-15.
Reich NG, Lessler J, Funk S, Viboud C, Vespignani A, Tibshirani RJ, Shea K, Schienle M, Runge MC, Rosenfeld R, Ray EL, Niehus R, Johnson HC, Johansson MA, Hochheiser H, Gardner L, Bracher J, Borchering RK, Biggerstaff M. “Collaborative Hubs: Making the Most of Predictive Epidemic Modeling.” AJPH. 2022. 112(6): 839-842.
Cramer EY, Ray EL, Lopez VK, et al. “Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US.” PNAS. 2022. 119(15): e2113561119.
Reich NG, Ray EL. “Collaborative modeling key to improving outbreak response.” PNAS. 2022. 119(14): e2200703119.
Cramer EY, Reich NG. “A modeler’s primer for working with SARS-CoV-2 genomic data.” Reich Lab Blog. 2022.
Reich NG, Tibshirani RJ, Ray EL, Rosenfeld R. “On the predictability of COVID-19.” International Institute of Forecasters Blog. 2021.
Ray EL, Brooks LC, Bien J, Bracher J, Gerding A, Rumack A, Biggerstaff M, Johansson MA, Tibshirani RJ, Reich NG. “Challenges in training ensembles to forecast COVID-19 cases and deaths in the United States.” International Institute of Forecasters Blog. 2021.
Bracher J, Ray EL, Gneiting T, Reich NG. “Evaluating epidemic forecasts in an interval format.” PLOS Comp Bio. 2021. 17(2): e1008618.
Brooks LC, Ray EL, Bien J, Bracher J, Rumack A, Tibshirani RJ, Reich NG. “Comparing ensemble approaches for short-term probabilistic COVID-19 forecasts in the U.S.” International Institute of Forecasters Blog. 2020.
Ray EL, Wattanachit N, Niemi J, et al. “Ensemble Forecasts of Coronavirus Disease 2019 (COVID-19) in the U.S.” medRxiv. 2020. (preprint).

Partial list of relevant publications from our team (pre-COVID):

We have published numerous peer-reviewed research articles on infectious disease forecasting that provide a solid foundation for understanding how to best forecast outbreaks. For example, our work has found that there was not a large difference in predictive performance between mechanistic and statistical models of seasonal influenza outbreaks. We also showed modest but measurable improvements in accuracy when using an ensemble forecast to forecast influenza.

Reich NG, McGowan CJ, Yamana TK, Tushar A, Ray E, Osthus D, Kandula S, Brooks LC, Crawford-Crudell W, Gibson GC, Moore E, Silva R, Biggerstaff M, Johansson MA, Rosenfeld R, Shaman J. “Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S.” PLOS Comp Bio. 2019. 15(11): e1007486.
Reich NG, Brooks L, Spencer F, Kandula S, McGowan C, Moore E, Osthus D, Ray E, Tushar A, Yamana T, Biggerstaff M, Johansson MA, Rosenfeld R, and Shaman J. “A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States.” PNAS. 2019. 116(8): 3146-3154.
McAndrew T, Reich NG. “Adaptively stacking ensembles for influenza forecasting with incomplete data.” arXiv. 2019. (preprint)
McGowan C, Biggerstaff M, Johansson M, Apfeldorf K, Ben-Nun M, Brooks L, Convertino M, Erraguntla M, Farrow D, Freeze J, Ghosh S, Hyun S, Kandula S, Lega J, Liu Y, Michaud N, Morita H, Niemi J, Ramakrishnan N, Ray EL, Reich NG, Riley P, Shaman J, Tibshirani R, Vespignani A, Zhang Q, Reed C. Collaborative efforts to forecast seasonal influenza in the United States, 2015–2016.” Sci Rep. 2019. 9(683).
Ray EL, Reich NG. Prediction of infectious disease epidemics via weighted density ensembles.”” PLOS Comp Bio. 2018. 14(2): e1005910.