Each week, we generate ensemble forecasts of cumulative deaths and incident deaths over the next four weeks that combine the forecasts from a designated model submitted by each team. This is helpful because it gives a sense of the general consensus forecast across all teams. Previous work in infectious disease forecasting and other fields has also shown that ensemble forecasts are often more accurate than any individual model that went into the ensemble. Readers who are more familiar with the forecasting methods may also find it helpful to explore forecasts from individual models to obtain a more detailed understanding of the underlying uncertainty and the range of projections generated by models built on different assumptions.
To be included in the ensemble, a team’s designated model must meet certain specified inclusion criteria. Most importantly, forecasts for 1 through 4 week ahead cumulative or incident deaths must be submitted on either Sunday or Monday of a given week to be included in that week’s ensemble. For forecasts of cumulative deaths, we also perform two additional checks for internal consistency. By definition, cumulative deaths cannot decrease over time (other than possibly because of revisions to reporting). We therefore require that (1) a team assigns at most a 10% chance that cumulative deaths would decrease in their one-week ahead forecasts, and (2) at each quantile level of the predictive distribution, that quantile is constant or increasing over time. We also perform one final plausibility check by comparing the predictive distributions to the observed data for past weeks from the JHU data repository; forecasts that are dramatically out of line with the historical data are not included in the ensemble. All models that are eligible for inclusion in the ensemble are given equal weight. Details on which models were included each week in the ensemble are available in the ensemble metadata folder.