Advertisement

Modeling the Effects of H1N1 Influenza Vaccine Distribution in the United States

  • Richard C. Larson
    Correspondence
    Address correspondence to: Richard C. Larson, Center for Engineering Systems Fundamentals (CESF), Massachusetts Institute of Technology, 77 Massachusetts Avenue, Building E40-233, Cambridge, MA 02139, USA
    Affiliations
    Engineering Systems Division (ESD), Massachusetts Institute of Technology, Cambridge, MA, USA

    Center for Engineering Systems Fundamentals (CESF), Massachusetts Institute of Technology, Cambridge, MA, USA
    Search for articles by this author
  • Anna Teytelman
    Affiliations
    Operations Research Center (ORC), Massachusetts Institute of Technology, Cambridge, MA, USA
    Search for articles by this author
Open ArchivePublished:December 05, 2011DOI:https://doi.org/10.1016/j.jval.2011.07.014

      Abstract

      Objective

      We analyzed the effects of the timing of vaccine distribution in 11 US states during the 2009 H1N1 influenza pandemic.

      Methods

      By using reported data on the fraction of patients presenting with flu-related symptoms, we developed a transformation that allowed estimation of the state-specific temporal flu wave curve, representing the number of new infections during each week. We also utilized data describing the weekly numbers of vaccine doses delivered and administered. By using a simple difference equations model of flu progression, we developed two influenza wave curves: first, an “observable” curve that included the beneficial effects of vaccinations, and second, an unobservable curve that depicted how the flu would have progressed with no vaccine administered. We fit the observable curve to match the estimated epidemic curve and early exponential growth associated with R0, the reproductive number. By comparing the number of infections in each scenario, we estimated the infections averted by the administration of vaccine.

      Results

      Southern states experienced peak infection several weeks before northern states, and most of the vaccine was delivered well after the peak of the southern flu wave. Our models suggest that the vaccine had minimal ameliorative impact in the southern states and measurable positive impact in the northern states. Vaccine delivery after peak also results in a smaller fraction of the population's seeking the vaccine.

      Conclusions

      Our analysis suggests that current Centers for Disease Control and Prevention policy of allocating flu vaccine over time in direct proportion to states' populations may not be best in terms of averting nationally the maximum possible number of infections.

      Keywords

      Introduction

      The novel H1N1 influenza virus surfaced in San Diego in early April 2009 and very soon thereafter in Mexico and was announced as a pandemic by the World Health Organization in June 2009 [
      Centers for Disease Control and Prevention
      The 2009 H1N1 pandemic: summary highlights, April 2009-April 2010.
      ,
      World Health Organization
      World now at the start of 2009 influenza pandemic.
      ]. By early fall, most of the world was experiencing the wave of the H1N1 pandemic [
      World Health Organization
      Global Alert and Response (GAR): pandemic (H1N1) 2009- update 112.
      ]. The US Centers for Disease Control and Prevention (CDC) took the threat of a pandemic seriously, and by the middle of April 2009, researchers began to develop vaccines matched to the new virus [
      Centers for Disease Control and Prevention
      The 2009 H1N1 pandemic: summary highlights, April 2009-April 2010.
      ]. The vaccine production process for a novel flu strain requires at least 6 months for virus identification, vaccine invention, and then mass production using a long-standing egg-based technology. The inherent delays resulted in the first vaccines' being shipped in early October 2009 [
      Centers for Disease Control and Prevention
      Graph and table of 2009 H1N1 influenza vaccine doses allocated, ordered, and shipped.
      ]. The vaccine had varying ameliorative effects in different regions of the United States. Partly because of early school openings in the southeast [
      Centers for Disease Control and Prevention
      Press briefing transcripts.
      ], the major wave of the outbreak hit the southeastern states earlier than the northern ones. Nonetheless, vaccines as produced were delivered to states strictly on a per capita basis [
      Centers for Disease Control and Prevention
      Graph and table of 2009 H1N1 influenza vaccine doses allocated, ordered, and shipped.
      ], with each state receiving vaccine in direct proportion to its population. As a result, some states were able to vaccinate a significant portion of their population prior to the major flu wave's hitting, while others did not receive vaccines until the major flu wave had already passed and interest in vaccinations had waned. Vaccine distribution within individual states was significantly more complicated because states had relative freedom in utilizing the vaccines shipped from the CDC. We refer the reader to Hopkins [
      • Hopkins J.
      2009 H1N1 after action reports: lessons on vaccine distribution MIT ESD Working Paper 2011.
      ] for an in-depth analysis of the different approaches taken by states and the challenges faced during the H1N1 pandemic.
      In this article, we present analysis of specific states' epidemic curves and the relative effectiveness of vaccine programs. The essential analytical issue we faced was estimating the epidemic flu curve in the absence of vaccine when the information we had was limited and was available only for the presence of vaccine. For most states, readily available information includes only the percentage of all hospitalizations and outpatient visits that were caused by influenza-like illness (ILI) [
      Centers for Disease Control and Prevention
      CDC, seasonal influenza (flu), weekly report: influenza summary update.
      ]. The results of our analysis in this article suggest the importance of administering vaccination as early as possible, even if the numbers of vaccines administered are small. Early administration not only serves to decrease the peak of the epidemic curve but also encourages higher participation rates from the population.
      The problem of vaccine allocation has been analyzed in the context of distribution of vaccine to various demographic groups in a population. The public policy consensus has been to first vaccinate high-risk groups, including pregnant women, health-care workers, and those at risk of complications from influenza [
      Centers for Disease Control and Prevention
      Questions and answers: seasonal flu shot.
      ,
      • Longini I.
      • Halloran E.
      Strategy for distribution of influenza vaccine to high-risk groups and children.
      ]. Some authors, however, claim that in the event that large stockpiles of vaccine are available, the vaccines should be distributed to the “drivers” of infection, such as schoolchildren and other high-activity individuals [
      • Patel R.
      • Longini I.M.
      • Halloran M.E.
      Finding optimal vaccination strategies for pandemic influenza using genetic algorithms.
      ]. Chowell et al. [
      • Chowell G.
      • Viboud C.
      • Wang X.
      • et al.
      Adaptive vaccination strategies to mitigate pandemic influenza: Mexico as a case study.
      ] have approached the problem of measuring vaccine effectiveness by modeling a possible outbreak similar to H1N1 based on Mexican demographics. They concluded that as much as a 37% reduction in the number of hospitalizations could have been achieved with an “adaptive” vaccination strategy based on demographics and information obtained during the outbreak. The authors suggested that as the outbreak progresses, vaccines should be allocated to different age groups in proportion to the then-reported influenza-like symptoms and hospitalizations from within those age groups.
      While these articles analyze the effectiveness of vaccination programs on such parameters as the transmissibility of the flu and the demographics of the population, we chose to consider another issue, the timing of the vaccines administered. The timing of vaccine distribution has been explored by some authors in the context of both general influenza models [
      • Nigmatulina K.
      • Larson R.C.
      Living with influenza: impacts of government imposed and voluntarily selected interventions.
      ,
      • Larson R.C.
      • Nigmatulina K.R.
      Engineering reponses to pandemics.
      ] and specific outbreaks [
      • Khazeni N.
      • Hutton D.
      • Garber A.
      • et al.
      Effectiveness and cost-effectiveness of vaccination against pandemic influenza (H1N1) 2009.
      ,
      • Sander B.
      • Bauch C.
      • Fishman D.N.
      • et al.
      Is a mass immunization program for pandemic (H1N1) 2009 good value for money? Early evidence from the Canadian experience.
      ]. We, in turn, analyzed the effect of vaccine distribution timing on the spread of the 2009–2010 H1N1 virus in different US states. The large size of the country allowed us to compare the spread of the same strain of influenza as it infected different communities at different times.
      Specifically, for each of several states, we examined two timing relationships, the time of onset and growth of the flu wave and the timing of vaccinations. For each state, we applied simple temporal models to estimate the number of flu infections averted by the administration of vaccine.
      The rapid and massive spread of H1N1 in Mexico City in the spring of 2009 motivated numerous articles estimating the parameters of the epidemic and characterizing the effects of H1N1. The most important parameter in epidemiology, the reproductive number R0, is defined as the average number of new infections caused by a “typical” infectious individual in a fully susceptible population. Early estimates in Mexico estimated R0 to range between 1.4 and 1.6 [
      • Fraser C.
      • Donnelly C.A.
      • Cauchemez S.
      • et al.
      Pandemic potential of a strain of influenza A (H1N1): early findings.
      ], implying a potentially dangerous progression for other countries. This, in turn, led to predicted estimates of 1.3 to 1.8 for R0 for the United States and an estimated 60% national infection rate [
      • White L.F.
      • Wallinga J.
      • Finelli L.
      • et al.
      Estimation of the reproductive number and the serial interval in early phase of the 2009 influenza A/H1N1 pandemic in the USA.
      ,
      • Towers S.
      • Feng Z.
      Pandemic H1N1 influenza: predicting the course of a pandemic and assessing the efficacy of the planned vaccination programme in the United States.
      ]. The CDC later reestimated the range of actual US infections to be between 43 million and 88 million [
      Centers for Disease Control and Prevention
      Updated CDC estimates of 2009 H1N1 influenza cases, hospitalizations and deaths in the United States, April 2009–March 13, 2010.
      ], representing only 14% to 29% of the population.
      A study in Ontario, Canada, estimated R0 there to be about 1.31 [
      • Tuite A.R.
      • Greer A.L.
      • Whelan M.
      • et al.
      Estimated epidemiologic parameters and morbidity associated with pandemic H1N1 influenza.
      ]. The authors attributed the lower-than-predicted estimates to heightened public awareness and better adherence to hygienic behavior that reduces virus transmission. In our modeling of US outbreak progression, we also encountered relatively low values for R0 but note that these values agree with findings in Ontario and the relatively low infection rates reported in the United States.
      In a recent article, Sander et al. [
      • Sander B.
      • Bauch C.
      • Fishman D.N.
      • et al.
      Is a mass immunization program for pandemic (H1N1) 2009 good value for money? Early evidence from the Canadian experience.
      ] used an agent-based model to depict the spread of H1N1 in Ontario. They asserted that the immunization program in the province was a “cost-effective” means to prevent H1N1 cases. Ontario's program started administering vaccines 2 weeks before the peak of infections in Ontario. Because most of the states were hit by the outbreak earlier, half of the states started administering vaccine after the peak had already passed [
      • Finkelstein S.
      • Hedberg K.J.
      • Hopkins J.A.
      • et al.
      Vaccine availability in the US following the 2009 H1N1 outbreak.
      ]. Not only were vaccines delivered late, but the demand for vaccines also declined significantly by the time vaccines became available [
      • Harris K.M.
      • Maurer J.
      • Kellermann A.
      Influenza vaccine—safe, effective, and mistrusted.
      ]. Eventually, in the United States, as many as 70 million doses were left unadministered [
      • Moyer C.S.
      H1N1 vaccine: what physicians can do with leftover doses Am Med News.
      ]. Surveys showed that low participation rates were partially a result of concerns for the safety of vaccines and the relative innocuousness of the H1N1 virus [
      • Harris K.M.
      • Maurer J.
      • Kellermann A.
      Influenza vaccine—safe, effective, and mistrusted.
      ,
      • SteelFisher G.K.
      • Blendon R.J.
      • Bekheit M.M.
      • Lubell K.
      The public's response to the 2009 H1N1 influenza pandemic.
      ]. Moreover, by December 2009, when vaccines became widely available to everyone, the proportion of people in the population who were concerned about the dangers of H1N1 had dropped significantly from between 51% and 59% to about 40% [
      • SteelFisher G.K.
      • Blendon R.J.
      • Bekheit M.M.
      • Lubell K.
      The public's response to the 2009 H1N1 influenza pandemic.
      ].
      For each state, Figure 1 displays in color codes the timing of shipment of first vaccine with respect to the peak of the infection in that state [
      Centers for Disease Control and Prevention
      Graph and table of 2009 H1N1 influenza vaccine doses allocated, ordered, and shipped.
      ]. Six southeastern states received their first vaccines more than 4 weeks after the peak of their respective H1N1 outbreaks. The infection then spread northwest, with the northeastern states having the most time to prepare and vaccinate their populations. Maine and isolated Hawaii had more than 4 weeks of vaccine dispensation prior to the peak outbreak in their states.
      Figure thumbnail gr1
      Fig. 1Timing of shipment of first vaccine with respect to the peak of the infection in selected states. N/A, not available.
      This article is a natural follow-up to the Finkelstein et al. [
      • Finkelstein S.
      • Hedberg K.J.
      • Hopkins J.A.
      • et al.
      Vaccine availability in the US following the 2009 H1N1 outbreak.
      ] article as we demonstrate the potential loss of vaccine effectiveness with late shipments. We evaluate the effect of the timing of vaccination programs in 11 states and suggest some lessons learned in the event of future pandemics and immunization efforts.

      Methods

      To estimate the number of infections averted from the various vaccine programs, we first used available data to estimate the epidemic curve during the period in the fall of 2009 when H1N1 was most prevalent. Once we had obtained an estimated epidemic curve, we used it in conjunction with reported vaccine administration data to fit the observed epidemic curve to the one generated by a mathematical model based on difference equations. We used a discrete-time version of the standard Kermack–McKendrick model to estimate infection spread within each state [
      • Kermack W.O.
      • McKendrick A.G.
      Contributions to the mathematical theory of epidemics.
      ,
      • Murray J.D.
      Mathematical Biology.
      ,
      • Keeling M.J.
      • Rohani P.
      Modeling infectious diseases in humans and animals.
      ]. In the model calibration process, we estimated the relevant parameters such as R0 within each state. We estimated the R0 for each state individually, because different states have different demographic, geographic, and cultural attributes. Moreover, states experienced the H1N1 outbreak at different times and implemented their vaccination programs in different ways [
      • Hopkins J.
      2009 H1N1 after action reports: lessons on vaccine distribution MIT ESD Working Paper 2011.
      ], so the extent of the infection varied markedly.
      We then estimated a different, nonobservable flu wave curve, one assuming no available vaccine. This multistep process provided a data-informed, model-supported basis for estimating the positive effects, if any, of the vaccine as administered in each of the states.
      Before describing the methodology in full, it is worth noting some common drawbacks to using a discrete-time model that implies homogeneous mixing in the population. Each state's population is segmented into groups of varying susceptibility, infectivity, and activity levels, each of which may strongly influence the progression of the epidemic [
      • Larson R.C.
      Simple models of influenza progression within a heterogeneous population.
      ,
      • Nigmatulina K.R.
      Modeling and responding to pandemic influenza: importance of population distributional attributes and non-pharmaceutical interventions (Doctoral dissertation 2009).
      ]. While all models are imperfect, each region we investigated was large enough that homogeneous mixing provided a reasonable estimate for modeling the epidemic curve. This methodology was used throughout the H1N1 outbreak [
      • Towers S.
      • Feng Z.
      Pandemic H1N1 influenza: predicting the course of a pandemic and assessing the efficacy of the planned vaccination programme in the United States.
      ,
      • Coburn B.J.
      • Wagner B.G.
      • Blower S.
      Modeling influenza epidemics and pandemics: insights into the future of swine flu (H1N1).
      ]. In addition, while epidemics occur in continuous time, our use of a discrete-time model both simplified calculation and allowed us to incorporate the fact that during the 2009–2010 H1N1 pandemic, vaccines were shipped on a weekly basis and vaccine stocks were increased at discrete time intervals.

      Epidemic curve estimation

      We first estimated the true epidemic curve within the different states, in which the flu curve includes the effects of vaccinations. Because this virus was so prevalent in the US population, it was impossible to record an accurate epidemic curve in each state. There was no direct way to know the total number of people infected because 1) many such individuals did not present themselves to medical authorities and 2) for those who did visit a physician, confirming tests for H1N1 were not routinely ordered. Instead, we used the data released by the states' health departments that included the weekly percentage of all hospitalizations and outpatient visits resulting from ILI (%ILI) over the 2009–2010 flu seasons [
      Centers for Disease Control and Prevention
      CDC, seasonal influenza (flu), weekly report: influenza summary update.
      ]. From that, we estimated the number of H1N1 infections for each week. We assumed that the total number of cases was directly proportional to the total number of H1N1 hospitalizations and outpatient visits. While the %ILI curve provided a good indication of the spread of infection, we did not use it as the epidemic curve because the total number of hospitalizations and outpatient visits associated with all medical conditions changes throughout the year. Each point on the %ILI temporal curve represents the percentage of the total number of hospitalizations and outpatient visits that are specific to H1N1. Because the flu wave first grows and then declines, this total number is not uniform throughout the observation period. We expect a higher number of total hospitalizations and outpatient visits during the peak of H1N1, with the number of non-H1N1 hospitalizations and outpatient visits remaining relatively stable throughout the several-month observation period. By using this assumption, we performed a simple transformation on the %ILI curve to obtain a new curve that we considered to be directly proportional to the experienced flu wave curve.
      To carry out the required transformation, we split our timeline into discrete “flu generation periods,” with generations represented by t = 1, 2, 3,..., tmax. In our calculations, we let a single generation of the flu be equal to 10/3 = 2.33 days. Assuming just one number for the duration of the flu is a simplification. While actual generation of the flu varies with different viruses as well as on an individual level, we made this simplification to keep our discrete-time epidemic curve model tractable. The value we chose is consistent with the analysis of H1N1 in Mexico done in May 2009 [
      • White L.F.
      • Wallinga J.
      • Finelli L.
      • et al.
      Estimation of the reproductive number and the serial interval in early phase of the 2009 influenza A/H1N1 pandemic in the USA.
      ,
      • Tuite A.R.
      • Greer A.L.
      • Whelan M.
      • et al.
      Estimated epidemiologic parameters and morbidity associated with pandemic H1N1 influenza.
      ].
      We let
      C(t)=the number of nonflu hospital patients in generation t.


      Invoking our assumption of a constant number of such patients over time, we expressed
      C(t)=C0t=1,2,3,. . ., tmax.


      Next we considered the unobservable number of flu patients seeking hospital medical advice: N(t) = the number of patients with H1N1 who visit a hospital in generation t, for t = 1, 2, 3,..., tmax. What we observed in reported data was F(t), the fraction of all hospital visits that were attributable to H1N1,
      F(t)=N(t)N(t)+C0.


      Solving for N(t), we calculated
      F(t)[N(t)+C0]=N(t),


      or
      F(t)C0=N(t)[1F(t)],


      or N(t)=C0F(t)[1F(t)].
      Invoking the assumption that the total number of H1N1 cases is proportional to the number of H1N1-related hospital visits, we deduced that the shape of the epidemic curve was proportional to N(t). Because we had access to F(t) only, the value of C0, the total number of non-H1N1 hospital visits per generation, was unknown. While we did not know that quantity, we could approximate the value of C0 because we knew the total number of H1N1 visits during our time period. Specifically, we let I be the total number of H1N1 visits in a US state. Then,
      t=1tmax[N(t)+C0]=t=1tmaxC0F(t)1F(t)+C0=I.


      Consequently, we calculated
      C0=Itmax+t=1tmaxF(t)1F(t).


      We approximated I by using information from the CDC regarding the total number of infections in the United States.
      This transformation gave us a closer estimate of the actual epidemic curve than did the %ILI curve by itself. We were still faced with the fact, however, that people presented to doctors and hospitals at different rates throughout the pandemic. One might conjecture that the rate of hospital visits would have been higher at the beginning of the pandemic, when media reports were implying a real danger from flulike symptoms, and that the rate would decrease toward the end of the epidemic, when reports had confirmed that the consequences of H1N1 were relatively mild; but the rate of hospital admissions would not be affected by such psychological factors. Furthermore, there are still other factors, such as parents of symptomatic children not wishing to present their children to medical facilities for fear of their contracting the illness, should they not already have it. One can keep conjecturing ad infinitum. It is difficult to quantify these myriad possible effects, and so we did not include them in the transformation.
      According to the CDC, the percentage of people infected with H1N1 in the United States ranged from 14% to 29% of the population [
      Centers for Disease Control and Prevention
      Updated CDC estimates of 2009 H1N1 influenza cases, hospitalizations and deaths in the United States, April 2009–March 13, 2010.
      ]. We tried a range of values for the total number of infections in the region and fit our model to each one by finding the parameters that minimized the mean square error between the epidemic curve and the resulting modeled curve. To find the best fit, we modified three parameters: the total number of infections, the number of infections at the start of the epidemic (“patient zeros”), and R0. For each fit, we took the resulting fitted value of R0. The values were usually close together, and we chose the value of R0 that best fit the exponential growth of cases near the onset of the epidemic.

      A set of difference equations

      We used a set of difference equations that corresponds to a discrete-time version of a classic Susceptible-Infectious-Recovered (S-I-R) model [
      • Kermack W.O.
      • McKendrick A.G.
      Contributions to the mathematical theory of epidemics.
      ,
      • Murray J.D.
      Mathematical Biology.
      ,
      • Keeling M.J.
      • Rohani P.
      Modeling infectious diseases in humans and animals.
      ,
      • Diekmann O.
      • Heesterbeek J.A.P.
      Mathematical Epidemiology of Infectious Diseases: Model Building, Analysis and Interpretation.
      ].
      For each state, we adjusted model parameters to obtain a best fit between the reported data, as transformed above, and the model-generated flu wave epidemic curve. This process led to a direct estimation of R0.
      We let R(t) = the mean number of new infections generated by a randomly selected infectious (asymptomatic) patient at generation t of the epidemic. t = 1, 2, 3,..., tmax.

      The epidemic curve

      Suppose the number of infected people at “generation zero” is 1 person. That is, there is 1 “patient zero.” Then, the mean number of infected people in generation 1 is R0 persons, each of whom subsequently infects on average R(1) persons in generation 2, and so on. So, in generation t, the mean number of people infected, MI(t), will be the product R0 × R(1) × R(2) × . . . × R(t − 1). More compactly, assuming that MI(0) = 1, we calculated
      MI(t)=R0n=1t1R(n),t=1,2,3,. . .,tmax
      (2)


      where MI(t) is the mean number of infections at time t.

      Determining R(t)

      Suppose the total population is N persons, all susceptible at the beginning of the epidemic and all mixing homogeneously—whether or not they are susceptible or have recovered and have immunity. Then suppose at generation t we have I(t) people recovered and immune, recirculating in the population. Because of homogeneous mixing, R0 is now modified by a factor of [N−I(t)−MI(t)]/N, to become
      R(t)=R0NI(t)MI(t)N,t=1,2,3,. . .,tmax.
      (3)


      Determining I(t)

      At generation t we assumed that there were a total of I(t) individuals immune to the disease. This immunity can be derived from having been vaccinated or from having recovered from the disease. For simplicity in this model, we assumed that individuals infected in generation (t − 1) recovered and were immune in generation t. We also assumed that during the time of generation (t − 1), V(t − 1) individuals received vaccine that made them immune for the first time in generation t. Thus, we wrote a difference equation for determining the value of I(t):
      I(t)=I(t1)+MI(t1)+V(t1),t=1,2,3,. . ., tmax.
      (4)


      We now had a model consisting of the following three equations where V(t) is known for all t.
      MI(t)=R0n=1t1R(n),t=1,2,3,. . .,  tmax


      R(t)=R0N=I(t)MI(t)N,t=1,2,3,. . ., tmax


      I(t)=I(t1)+MI(t1)+V(t1),t=1,2,3,. . ., tmax


      These equations can be solved iteratively, via recursion starting with the boundary conditions MI(0), I(0), and R(0) = R0.

      Estimating the effects of vaccine

      The nonobservable curve

      Suppose, in the absence of vaccines, we have a pandemic infection curve C(t). Here,
      C(t) = the number of new flu infections reported during generation t of the infection period, assuming no vaccine. t = 1, 2, 3,..., tmax.
      We called C(t) the base curve, that is, the infection wave that occurs in the absence of vaccines. Also, for the sake of simplicity we assumed that human behavior, as illustrated, for instance, by hygienic steps and other nonpharmaceutical interventions, remained unchanged during the course of the infection period. This base curve is unobservable because the state-specific data reporting numbers of people infected included the effects of administered vaccines.

      Timing of vaccinations and subsequent immunity

      To depict the effect of vaccinations on the population, we used V(t), which is the number of individuals who have been vaccinated and first acquired immunity during generation t, with t = 1, 2, 3,..., tmax.
      We assumed that only susceptible individuals received vaccinations.
      On average, vaccines take effect about 2 weeks after they have been administered. In our approximations, we compared the numbers calculated in the cases in which the vaccines took effect immediately as well as the cases in which vaccines took effect after 2 weeks. Similarly, we considered vaccine effectiveness (i.e., creating immunity to H1N1) to be between 75% and 100%. According to preliminary studies, the effectiveness of H1N1 vaccine was well over 90% in 10 days [
      • Greenberg M.E.
      • Lai M.H.
      • Hartel G.F.
      • et al.
      Response after one does of a monovalent influenza A (H1N1) 2009 vaccine—preliminary report.
      ], and so the actual effect of the vaccines should be well within the bounds of our analysis.

      The observable curve

      Another pandemic infection curve, C(t, V⃗), is the number of new flu infections reported during generation t of the flu, given that immunities due to vaccinations occur according to the known time vector V⃗, with t = 1, 2, 3,..., tmax.
      Our state-derived data depicted the vaccine-affected flu wave C(t, V⃗), and we wished to infer the (unobservable) vaccine-free flu wave C(t). To estimate the effect of the vaccinations, we needed to estimate C(t) and then compare it to C(t, V⃗). The difference in the areas under the respective curves represented our estimate of the number of infections averted because of the vaccine.
      To estimate R0, we fit the model-generated C(t, V⃗) curve to the empirically estimated epidemic curve, which includes the effects of vaccinations. The influenza cases during the H1N1 epidemic were severely underreported. The underreporting appeared to be especially significant at the beginning and end of the outbreak [
      Centers for Disease Control and Prevention
      Updated CDC estimates of 2009 H1N1 influenza cases, hospitalizations and deaths in the United States, April 2009–March 13, 2010.
      ,
      Centers for Disease Control and Prevention
      Underreporting of 2009 H1N1 influenza casesfrom.
      ]. To avoid this “statistical noise” effect, we fit the model-based epidemic curve to the data surrounding the peak of the epidemic, specifically to the part of the empirical curve that contains 75% of the cases. To find a best reasonable fit, we used the difference Equations (2), (3), (4), and manipulated the parameters R0 and MI(O). The best fit was determined to be one for which the peak of the model-determined epidemic curve coincided with the estimated real epidemic curve and for which exponential growth of the early stages of the outbreak coincided with the real epidemic curve. The R0 obtained by using this method might be a slight underestimate of the true R0 for the H1N1 pandemic because it was not fitted at the very beginning of the outbreak when data were very noisy, but only once clear exponential growth was established. This methodology, however, is consistently used in the literature and provides a reasonable estimate for the basic reproductive number [
      • Fraser C.
      • Donnelly C.A.
      • Cauchemez S.
      • et al.
      Pandemic potential of a strain of influenza A (H1N1): early findings.
      ,
      • Mills C.E.
      • Robins J.M.
      • Lipsitch M.
      Transmissibility of 1918 pandemic influenza.
      ].

      Results

      Oklahoma in detail

      Consider as an illustrative example the estimation process for Oklahoma. Figure 2 shows 1) the empirical-based estimated epidemic curve for Oklahoma created using the transformation described above and 2) the time-sequenced vaccine administration data reported by the state. In Figure 3, we again included the empirical epidemic curve and three model-generated epidemic curves:
      • 1
        The curve generated by using the Oklahoma-reported vaccine administration data, fitted to correspond best to the empirical epidemic curve;
      • 2
        The curve generated in the hypothetical case in which vaccines were not administered at all; and
      • 3
        The curve generated in the hypothetical case in which vaccines were administered 2 weeks earlier than had actually occurred.
      Figure thumbnail gr2
      Fig. 2The estimated Oklahoma epidemic curve compared with vaccines as they were administered in the state. OK, Oklahoma.
      Figure thumbnail gr3
      Fig. 3The estimated epidemic curve along with the model-generated curves with and without vaccines. The best fit for Oklahoma was found with parameters R0 = 1.14, MI(0) = 1500, and I(0) = 0. OK, Oklahoma.
      The total number of infections caused by the outbreak in Oklahoma was calculated by the area under an epidemic curve. The effect of the vaccines administered in Oklahoma was determined by calculating the area between the “actual” model-generated curve and the “no-vaccine” model-generated curve (Fig. 4) .
      Figure thumbnail gr4
      Fig. 4A closer look at . The shaded region represents the difference between the estimated number of H1N1 infections that would have occurred without the intervention of vaccines and the estimated number of infections that actually occurred with the vaccine.
      We analyzed 11 states in detail and inferred the total number of infections that were prevented as a result of their respective immunization programs. The states Illinois, Indiana, Massachusetts, Mississippi, Montana, New Jersey, New York, North Dakota, Oklahoma, South Carolina, and Virginia graciously provided us with precise data on vaccines as they were dispensed throughout the outbreak. Here we display two cases for each state:
      • 1
        The optimistic case, in which all vaccines are effective immediately and are 100% effective, and
      • 2
        The pessimistic case, in which vaccines are effective 2 weeks after administration and are effective for only 75% of the individuals receiving the vaccine.
      The actual effect of the vaccine should lie within the range specified by these two cases.
      It is possible that some ILI-related cases were reported with a delay of 1 or 2 weeks with respect to symptom onset, because infected individuals may be more likely to visit a medical professional after symptoms become severe. If that is the case, our estimated epidemic curve might be shifted to the left by a few generations. If true, the infections occurred even earlier than what we had assumed, and so vaccine arrival occurred even later with respect to the peak of infections. This would imply that our results are an optimistic estimate of the effectiveness of states' vaccine programs.

      Summary

      In Table 1 we present the results of analysis of 11 states with detailed vaccination information. While we have detailed data for only these 11 states, we note that the range of estimated values for R0 is fairly narrow, with higher values falling on more populated states such as New York and Illinois and lower values corresponding to states with lower population density such as Oklahoma and Montana. Geographic proximity also seems to result in similar R0 values.
      Table 1Summary of best-fitted values for R0 and model-determined effects of vaccines as they were distributed
      In the optimistic scenario, vaccines are 100% effective and take effect immediately. In the pessimistic scenario, vaccines are 75% effective and take effect 2 wk after being administered.
      StateEstimated R0% Population vaccinatedOptimistic scenario: infections averted (in % of total population)Pessimistic scenario: infections averted (in % of total population)
      Illinois1.2193.431.2
      Indiana1.15204.281.81
      Massachusetts1.162913.716.84
      Mississippi1.1680.130.05
      Montana1.15202.811.04
      New Jersey1.20123.361.1
      New York1.20143.231.12
      North Dakota1.16272.951.06
      Oklahoma1.14132.290.93
      South Carolina1.1680.40.12
      Virginia1.19221.770.52
      low asterisk In the optimistic scenario, vaccines are 100% effective and take effect immediately. In the pessimistic scenario, vaccines are 75% effective and take effect 2 wk after being administered.
      More detailed information for all states is included in Appendix A in Supplemental Materials at doi:10.1016/j.jval.2011.07.014.

      Discussion

      While examining the estimated numbers of infections averted, we can identify two major contributing factors. The first is the total number of vaccines administered to the general population. The second is the timing of the vaccine administration with respect to the peak of the infections. Once the H1N1 virus had been identified as a potentially devastating pandemic in the spring of 2009, the CDC worked to develop and distribute H1N1 vaccines. These vaccines were sent out to the individual states at the same time, and first doses were administered on October 5, 2009. These vaccines, however, had varying effects because the peak of the outbreak in different states occurred at markedly different times.
      The peak of infection usually occurs when herd immunity occurs, that is, the time when R(t) = 1, and every infectious person at that time infects on average just one other person [
      • Anderson R.
      The concept of herd immunity and the design of community-based immunization programmes.
      ]. So, at the time of herd immunity, the number of infections in the next generation is approximately the same as it was in the previous one. We say “approximately” because of statistical fluctuations in the actual number of susceptible people that any one newly infectious person will infect. Soon afterward, infectious people no longer replace themselves in society, and the number of infected and infectious people in each generation decreases. Early administration of vaccines decreases the number of people who still need to be infected before herd immunity is achieved, and so decreases the height of the peak. Late administration of the vaccine has almost no effect on the dynamics of the outbreak and has little benefit to the society other than immunizing the people who received the vaccine. Such late immunizations may be important if the flu were to return later in a new wave.
      Consider again the southeastern states of the United States, the first region to report infection peaks. As early victims they received vaccines after the worst of the infection had already passed. Louisiana, Indiana, and South Carolina had not started administering vaccines until after the peak of outbreak. And these states were least successful in averting infections. On the other hand, Massachusetts and Virginia started administering their vaccines 5 and 3 weeks, respectively, before their respective peaks. These two states enjoyed a particularly good impact from their vaccination programs. Massachusetts, in addition to having 5 weeks of vaccinations prior to the H1N1 peak, vaccinated 29% of its population, the most of any state in our sample. As a result, as much as 7% to 14% of the population may have been spared infection and possible complications from influenza. While Massachusetts and Virginia had effective experiences with their vaccinations, most states did not. On average for our limited sample, vaccines were delivered just before the peak of the states' outbreaks.
      To quantify the effect of time in averting infection we considered one of the states that vaccinated almost 20% of its population, Indiana. Hypothetically, if the same number of vaccines had been delivered just 2 weeks earlier, more than twice the number of infections could have been averted.
      With a more granular approach, we considered the marginal benefit of administering just one vaccination at a given time. We mapped the total projected number of infections that could be averted if just one vaccination were to be administered to a susceptible person at different times during the outbreak. That is, we calculated the total number of infections that would occur in Indiana if exactly one vaccine were administered at different points in time and compared that number to the total number of infections that would happen if no vaccines were administered at all. The differences are presented in Figure 5. As expected, administered vaccines have a monotonically decreasing benefit with respect to time. A striking feature of Figure 5 is the fact that one vaccination to a susceptible person well before the flu wave starts averts almost two infections in the population, even with a low value for R0 (1.15) and even considering the fact that the vaccinated person has a greater than even chance of never becoming infected assuming no vaccination. Clearly, vaccines administered well before the peak carry the added benefit of diluting the susceptible population with immune people and are particularly useful in mitigating the spread of infection.
      Figure thumbnail gr5
      Fig. 5The plot shows the number of infections averted by administering exactly one vaccination at different points throughout the outbreak. IN, Indiana.
      Another insightful feature of this graph is the slope of the marginal benefit curve, which represents the time dependence of effective vaccines. While starting vaccine administration in Indiana in July would be most effective for Indiana, the effect of these vaccines would not change significantly until the beginning of September. That is, if vaccines were to be available in July, Indiana could have waited to receive its share until September with minimal losses. Similarly, vaccines received after December will have the same (minimal) effect whether they are administered in December or February. The effectiveness of vaccines, however, is extremely time sensitive from the end of September to mid-November, where each week results in a significant loss of effectiveness. Vaccines that become available during this critical period need to be administered as soon as possible. These results encourage us to recommend a more detailed cost–benefit analysis of trying to get some vaccine, even if in much smaller quantities, to the states at the beginning of this “critical period,” when the population is particularly sensitive to the timing of vaccination. A small amount of vaccine delivered early should have a more significant effect on the total number of infections than a batch delivered just a few weeks later.
      Furthermore, we conjecture that the timeliness of the vaccines is also closely related to the total amount of vaccine accepted by the population. While the CDC distributed its vaccines proportionally to the population of each region, states varied in the amount of vaccine that was actually used. For instance, Mississippi used less than 40% of its allocated vaccine, most likely due to “flu fatigue.” While the media are particularly helpful at warning the public of an ongoing pandemic and encouraging the use of nonpharmaceutical interventions, they can also give the impression that the outbreak is over or has been blown out of proportion. Once the peak of the outbreak had passed and H1N1 had been determined to be less dangerous than was originally feared, the populations of the “early victim” states would be less likely to spend their time and risk perceived possible side effects of getting a flu shot.
      Looking at all 50 states and comparing the percentage of vaccinated population by the end of the outbreak to the week of vaccine delivery [
      Centers for Disease Control and Prevention
      Interim results: State-specific influenza A (H1N1) 2009 monovalent vaccination coverage—United States, October 2009–January 2010.
      ], we notice in Figure 6 a weak negative correlation between the timing of vaccine delivery with respect to the peak, and the total percentage of the population that accepted vaccinations. These results are consistent with the position of Harris et al. [
      • Harris K.M.
      • Maurer J.
      • Kellermann A.
      Influenza vaccine—safe, effective, and mistrusted.
      ] that had the vaccines been delivered earlier to the states, more people would have been encouraged to accept a vaccination.
      Figure thumbnail gr6
      Fig. 6The relationship between the timing of vaccine delivery with respect to the peak and the percentage of the state population that received the H1N1 vaccine for all 50 states. w.r.t., with respect to.
      This effect would be particularly relevant to South Carolina, which started administering its vaccines 1 week after the peak of infection and subsequently managed to vaccinate only 8% of its population. If these vaccinations were to have started earlier, before the peak, we could hypothesize not only an increase in effectiveness from timing alone but also a higher participation rate in the vaccination program. Early administration is particularly important in that it increases the efficacy of vaccine along with encouraging people to accept vaccination.

      Vaccine allocation

      As shown in Figure 7, in the first few weeks of vaccine distribution, when demand for vaccine clearly exceeded supply, the CDC allocated vaccine to states proportionally to their populations [
      Centers for Disease Control and Prevention
      Graph and table of 2009 H1N1 influenza vaccine doses allocated, ordered, and shipped.
      ]. Particularly in early October, this simple distribution scheme ensured that all states received amounts that could be used to immunize the same proportion of the population. Come November, those states that saw little demand started placing fewer orders for influenza vaccine, while those with later epidemics such as Massachusetts and Virginia were still experiencing high demand and were shipped larger quantities of vaccine, confirming the intuition from Figure 6.
      Figure thumbnail gr7
      Fig. 7During the early stages of the outbreak when the demand for vaccines outweighed the supply, the Centers for Disease Control and Prevention distributed vaccine proportionally to the population of the states.
      The previous sections imply that the same vaccines administered in states that had already experienced the peak of the infection at the time vaccines started arriving were much less effective than those administered in states that had not yet experienced the peak. Moreover, the states that were past the peak saw less demand for vaccines and thus used only a small fraction of their allocated vaccines. Consider a side-by-side analysis of Mississippi and North Dakota in Table 2.
      Table 2Comparison of vaccination programs in North Dakota and Mississippi
      North DakotaMississippi
      • Started administration 1 wk before peak• Received vaccines 6 wk after peak
      • A hypothetical single batch of 32,342 vaccines (5% of the population) administered on October 10 averts 33,745 infections (5.2% of the population)• A hypothetical single batch of 147,599 vaccines (5% of the population) administered on October 10 averts 27,173 infections (0.9% of the population)
      • In the first 4 wk administered 62% of available vaccines• In the first 4 wk administered less than 25% of available vaccines
      It is clear that vaccines administered in North Dakota were significantly more effective than those in Mississippi. In fact, the Mississippi vaccines had almost no effect because the infection was barely spreading by the time vaccines became available. Coupled with this, North Dakota was experiencing more demand for the vaccines at the beginning of its program. Based on this analysis, we believe that there is a need for more effective procedures for allocating vaccines to US states.
      Naturally, allocating all of Mississippi's vaccines to North Dakota would be not only unethical but also politically infeasible. Instead, as a thought experiment, suppose that just 20% of Mississippi's unused vaccine were to be transferred to North Dakota during the first 4 weeks of vaccine distribution. Suppose that with this addition, 60% of the new vaccines were actually administered. This additional vaccine would decrease the total number of infections in North Dakota by 5%. That is a significant improvement for a relatively small cost. An adaptive decision such as this can be made during the allocation process. We can form even approximate predictions about how much vaccine will actually be demanded by the state and how effective the extra vaccines would be in reducing infections. For example, by using data collected from our 11 states, we can weakly estimate that a state that had experienced peak infection 6 weeks earlier can be expected to vaccinate no more than 4% of its population within the first 4 weeks. In the first 4 weeks of the 2009 H1N1 outbreak, the CDC allocated to Mississippi enough vaccine to cover 7% of its population. With accurate data, some portion of that could have been redirected to states that were more likely to use and benefit from the vaccine.

      Conclusions

      Our analysis shows the importance of the timing of vaccinations for infectious respiratory diseases such as influenza. We emphasize the need to start administering vaccines well before the peak of an influenza outbreak. Moreover, when a governing body such as the CDC is faced with the allocation decision, it is important to take into consideration the stages of the outbreaks in different regions and to deploy vaccine with preference to the regions that are projected to administer more of the vaccine with greater beneficial effects.
      In our analysis of individual state immunization programs we used a relatively simple model, assuming a homogeneous population and a deterministic model structure. As a result, our model, as most models, is not an exact picture of what happened during the fall of 2009, but rather a tool to gain insight about strategies that could help the public mitigate the effect of influenza pandemics. While the exact numbers almost surely differ from the estimates, the relative results should hold under a range of assumptions about R0 vaccine efficacy and the generation period of the flu.

      Acknowledgments

      We thank Dr. Stan Finkelstein, Dr. Sahar Hashmi, Kallie Hedberg, and Julia Hopkins for helpful comments on an earlier draft.
      Source of financial support: Work on this study was supported by the Sloan Foundation of New York under a grant titled “Decision-Oriented Analysis of Pandemic Flu Preparedness & Response” and under a cooperative agreement with the US Centers for Disease Control and Prevention (CDC) , grant number 1 PO1 TP000307-01 , “LAMPS (Linking Assessment and Measurement to Performance in PHEP Systems), awarded to the Harvard School of Public Health Center for Public Health Preparedness (HSPHCPHP) and the Massachusetts Institute of Technology (MIT), Center for Engineering Systems Fundamentals (CESF). The discussion and conclusions in this article are those of the authors and do not necessarily represent the views of the Sloan Foundation, the CDC, the US Department of Health and Human Services, Harvard, or MIT.

      Supplemental Materials

      References

        • Centers for Disease Control and Prevention
        The 2009 H1N1 pandemic: summary highlights, April 2009-April 2010.
        (Accessed January 20, 2011)
        • World Health Organization
        World now at the start of 2009 influenza pandemic.
        (Accessed January 20, 2011)
        • World Health Organization
        Global Alert and Response (GAR): pandemic (H1N1) 2009- update 112.
        (Accessed January 20, 2011)
        • Centers for Disease Control and Prevention
        Graph and table of 2009 H1N1 influenza vaccine doses allocated, ordered, and shipped.
        (updated February 2, 2010) (Accessed July 29, 2010)
        • Centers for Disease Control and Prevention
        Press briefing transcripts.
        (updated September 3, 2009) (Accessed June 10, 2011)
        • Hopkins J.
        2009 H1N1 after action reports: lessons on vaccine distribution.
        (Accessed October 18, 2011)
        • Centers for Disease Control and Prevention
        CDC, seasonal influenza (flu), weekly report: influenza summary update.
        (updated May 28, 2010) (Accessed July 29, 2010)
        • Centers for Disease Control and Prevention
        Questions and answers: seasonal flu shot.
        (Accessed January 20, 2011)
        • Longini I.
        • Halloran E.
        Strategy for distribution of influenza vaccine to high-risk groups and children.
        Am J Epidemiol. 2005; 161: 303-306
        • Patel R.
        • Longini I.M.
        • Halloran M.E.
        Finding optimal vaccination strategies for pandemic influenza using genetic algorithms.
        J Theor Biol. 2005; 234: 201-212
        • Chowell G.
        • Viboud C.
        • Wang X.
        • et al.
        Adaptive vaccination strategies to mitigate pandemic influenza: Mexico as a case study.
        PLoS One. 2009; 4: e8164
        • Nigmatulina K.
        • Larson R.C.
        Living with influenza: impacts of government imposed and voluntarily selected interventions.
        EJOR. 2009; 195: 613-627
        • Larson R.C.
        • Nigmatulina K.R.
        Engineering reponses to pandemics.
        Info Knowl Sys Manag. 2009; 8: 311-339
        • Khazeni N.
        • Hutton D.
        • Garber A.
        • et al.
        Effectiveness and cost-effectiveness of vaccination against pandemic influenza (H1N1) 2009.
        Ann Intern Med. 2009; 151: 829-831
        • Sander B.
        • Bauch C.
        • Fishman D.N.
        • et al.
        Is a mass immunization program for pandemic (H1N1) 2009 good value for money?.
        Vaccine. 2010; 28: 6210-6220
        • Fraser C.
        • Donnelly C.A.
        • Cauchemez S.
        • et al.
        Pandemic potential of a strain of influenza A (H1N1): early findings.
        Science. 2009; 324: 1557-1561
        • White L.F.
        • Wallinga J.
        • Finelli L.
        • et al.
        Estimation of the reproductive number and the serial interval in early phase of the 2009 influenza A/H1N1 pandemic in the USA.
        Influenza Other Respi Viruses. 2009; 3: 267-276
        • Towers S.
        • Feng Z.
        Pandemic H1N1 influenza: predicting the course of a pandemic and assessing the efficacy of the planned vaccination programme in the United States.
        Euro Surveill. 2009; 14: 19358
        • Centers for Disease Control and Prevention
        Updated CDC estimates of 2009 H1N1 influenza cases, hospitalizations and deaths in the United States, April 2009–March 13, 2010.
        (Accessed December 15, 2010)
        • Tuite A.R.
        • Greer A.L.
        • Whelan M.
        • et al.
        Estimated epidemiologic parameters and morbidity associated with pandemic H1N1 influenza.
        CMAJ. 2010; 182: 131-136
        • Finkelstein S.
        • Hedberg K.J.
        • Hopkins J.A.
        • et al.
        Vaccine availability in the US following the 2009 H1N1 outbreak.
        Am J Disaster Med. 2011; 6: 23-30
        • Harris K.M.
        • Maurer J.
        • Kellermann A.
        Influenza vaccine—safe, effective, and mistrusted.
        N Engl J Med. 2010; 363: 2183-2185
        • Moyer C.S.
        H1N1 vaccine: what physicians can do with leftover doses.
        (Accessed July 19, 2010)
        • SteelFisher G.K.
        • Blendon R.J.
        • Bekheit M.M.
        • Lubell K.
        The public's response to the 2009 H1N1 influenza pandemic.
        N Engl J Med. 2010; 362: e65
        • Kermack W.O.
        • McKendrick A.G.
        Contributions to the mathematical theory of epidemics.
        Proc R Soc Lond. 1927; 115: 700-721
        • Murray J.D.
        Mathematical Biology.
        in: Springer, Berlin1989: 610-650
        • Keeling M.J.
        • Rohani P.
        Modeling infectious diseases in humans and animals.
        Princeton University Press, Princeton, NJ2007
        • Larson R.C.
        Simple models of influenza progression within a heterogeneous population.
        Oper Res. 2007; 55: 399-412
        • Nigmatulina K.R.
        Modeling and responding to pandemic influenza: importance of population distributional attributes and non-pharmaceutical interventions (Doctoral dissertation 2009).
        (Accessed February 25, 2011)
        • Coburn B.J.
        • Wagner B.G.
        • Blower S.
        Modeling influenza epidemics and pandemics: insights into the future of swine flu (H1N1).
        BMC Med. 2009; 7: 30
        • Diekmann O.
        • Heesterbeek J.A.P.
        Mathematical Epidemiology of Infectious Diseases: Model Building, Analysis and Interpretation.
        Wiley, New York2000
        • Greenberg M.E.
        • Lai M.H.
        • Hartel G.F.
        • et al.
        Response after one does of a monovalent influenza A (H1N1) 2009 vaccine—preliminary report.
        N Engl J Med. 2009; 361: 2405-2413
        • Centers for Disease Control and Prevention
        Underreporting of 2009 H1N1 influenza casesfrom.
        (Accessed June 10, 2011)
        • Mills C.E.
        • Robins J.M.
        • Lipsitch M.
        Transmissibility of 1918 pandemic influenza.
        Nature. 2004; 432: 904-906
        • Anderson R.
        The concept of herd immunity and the design of community-based immunization programmes.
        Vaccine. 1992; 10: 928-935
        • Centers for Disease Control and Prevention
        Interim results: State-specific influenza A (H1N1) 2009 monovalent vaccination coverage—United States, October 2009–January 2010.
        (Accessed January 20, 2011)