## Abstract

### Objective

We analyzed the effects of the timing of vaccine distribution in 11 US states during the 2009 H1N1 influenza pandemic.

### Methods

By using reported data on the fraction of patients presenting with flu-related symptoms, we developed a transformation that allowed estimation of the state-specific temporal flu wave curve, representing the number of new infections during each week. We also utilized data describing the weekly numbers of vaccine doses delivered and administered. By using a simple difference equations model of flu progression, we developed two influenza wave curves: first, an “observable” curve that included the beneficial effects of vaccinations, and second, an unobservable curve that depicted how the flu would have progressed with no vaccine administered. We fit the observable curve to match the estimated epidemic curve and early exponential growth associated with

*R*_{0}, the reproductive number. By comparing the number of infections in each scenario, we estimated the infections averted by the administration of vaccine.### Results

Southern states experienced peak infection several weeks before northern states, and most of the vaccine was delivered well after the peak of the southern flu wave. Our models suggest that the vaccine had minimal ameliorative impact in the southern states and measurable positive impact in the northern states. Vaccine delivery after peak also results in a smaller fraction of the population's seeking the vaccine.

### Conclusions

Our analysis suggests that current Centers for Disease Control and Prevention policy of allocating flu vaccine over time in direct proportion to states' populations may not be best in terms of averting nationally the maximum possible number of infections.

## Keywords

## Introduction

The novel H1N1 influenza virus surfaced in San Diego in early April 2009 and very soon thereafter in Mexico and was announced as a pandemic by the World Health Organization in June 2009 [

1

, 2

]. By early fall, most of the world was experiencing the wave of the H1N1 pandemic [[3]

]. The US Centers for Disease Control and Prevention (CDC) took the threat of a pandemic seriously, and by the middle of April 2009, researchers began to develop vaccines matched to the new virus [[1]

]. The vaccine production process for a novel flu strain requires at least 6 months for virus identification, vaccine invention, and then mass production using a long-standing egg-based technology. The inherent delays resulted in the first vaccines' being shipped in early October 2009 [[4]

]. The vaccine had varying ameliorative effects in different regions of the United States. Partly because of early school openings in the southeast [[5]

], the major wave of the outbreak hit the southeastern states earlier than the northern ones. Nonetheless, vaccines as produced were delivered to states strictly on a per capita basis [[4]

], with each state receiving vaccine in direct proportion to its population. As a result, some states were able to vaccinate a significant portion of their population prior to the major flu wave's hitting, while others did not receive vaccines until the major flu wave had already passed and interest in vaccinations had waned. Vaccine distribution within individual states was significantly more complicated because states had relative freedom in utilizing the vaccines shipped from the CDC. We refer the reader to Hopkins [[6]

] for an in-depth analysis of the different approaches taken by states and the challenges faced during the H1N1 pandemic.In this article, we present analysis of specific states' epidemic curves and the relative effectiveness of vaccine programs. The essential analytical issue we faced was estimating the epidemic flu curve

*in the absence of vaccine*when the information we had was limited and was available only for the presence of vaccine. For most states, readily available information includes only the percentage of all hospitalizations and outpatient visits that were caused by influenza-like illness (ILI) [[7]

]. The results of our analysis in this article suggest the importance of administering vaccination as early as possible, even if the numbers of vaccines administered are small. Early administration not only serves to decrease the peak of the epidemic curve but also encourages higher participation rates from the population.The problem of vaccine allocation has been analyzed in the context of distribution of vaccine to various demographic groups in a population. The public policy consensus has been to first vaccinate high-risk groups, including pregnant women, health-care workers, and those at risk of complications from influenza [

8

, 9

]. Some authors, however, claim that in the event that large stockpiles of vaccine are available, the vaccines should be distributed to the “drivers” of infection, such as schoolchildren and other high-activity individuals [[10]

]. Chowell et al. [[11]

] have approached the problem of measuring vaccine effectiveness by modeling a possible outbreak similar to H1N1 based on Mexican demographics. They concluded that as much as a 37% reduction in the number of hospitalizations could have been achieved with an “adaptive” vaccination strategy based on demographics and information obtained during the outbreak. The authors suggested that as the outbreak progresses, vaccines should be allocated to different age groups in proportion to the then-reported influenza-like symptoms and hospitalizations from within those age groups.While these articles analyze the effectiveness of vaccination programs on such parameters as the transmissibility of the flu and the demographics of the population, we chose to consider another issue,

*the timing of the vaccines administered*. The timing of vaccine distribution has been explored by some authors in the context of both general influenza models [12

, 13

] and specific outbreaks [14

, 15

]. We, in turn, analyzed the effect of vaccine distribution timing on the spread of the 2009–2010 H1N1 virus in different US states. The large size of the country allowed us to compare the spread of the same strain of influenza as it infected different communities at different times.Specifically, for each of several states, we examined two timing relationships, the time of onset and growth of the flu wave and the timing of vaccinations. For each state, we applied simple temporal models to estimate the number of flu infections averted by the administration of vaccine.

The rapid and massive spread of H1N1 in Mexico City in the spring of 2009 motivated numerous articles estimating the parameters of the epidemic and characterizing the effects of H1N1. The most important parameter in epidemiology, the reproductive number

*R0*, is defined as the average number of new infections caused by a “typical” infectious individual in a fully susceptible population. Early estimates in Mexico estimated*R0*to range between 1.4 and 1.6 [[16]

], implying a potentially dangerous progression for other countries. This, in turn, led to predicted estimates of 1.3 to 1.8 for *R0*for the United States and an estimated 60% national infection rate [17

, 18

]. The CDC later reestimated the range of actual US infections to be between 43 million and 88 million [[19]

], representing only 14% to 29% of the population.A study in Ontario, Canada, estimated

*R0*there to be about 1.31 [[20]

]. The authors attributed the lower-than-predicted estimates to heightened public awareness and better adherence to hygienic behavior that reduces virus transmission. In our modeling of US outbreak progression, we also encountered relatively low values for *R0*but note that these values agree with findings in Ontario and the relatively low infection rates reported in the United States.In a recent article, Sander et al. [

[15]

] used an agent-based model to depict the spread of H1N1 in Ontario. They asserted that the immunization program in the province was a “cost-effective” means to prevent H1N1 cases. Ontario's program started administering vaccines 2 weeks before the peak of infections in Ontario. Because most of the states were hit by the outbreak earlier, half of the states started administering vaccine after the peak had already passed [[21]

]. Not only were vaccines delivered late, but the demand for vaccines also declined significantly by the time vaccines became available [[22]

]. Eventually, in the United States, as many as 70 million doses were left unadministered [[23]

]. Surveys showed that low participation rates were partially a result of concerns for the safety of vaccines and the relative innocuousness of the H1N1 virus [22

, 24

]. Moreover, by December 2009, when vaccines became widely available to everyone, the proportion of people in the population who were concerned about the dangers of H1N1 had dropped significantly from between 51% and 59% to about 40% [[24]

].For each state, Figure 1 displays in color codes the timing of shipment of first vaccine with respect to the peak of the infection in that state [

[4]

]. Six southeastern states received their first vaccines more than 4 weeks *after*the peak of their respective H1N1 outbreaks. The infection then spread northwest, with the northeastern states having the most time to prepare and vaccinate their populations. Maine and isolated Hawaii had more than 4 weeks of vaccine dispensation prior to the peak outbreak in their states.This article is a natural follow-up to the Finkelstein et al. [

[21]

] article as we demonstrate the potential loss of vaccine effectiveness with late shipments. We evaluate the effect of the timing of vaccination programs in 11 states and suggest some lessons learned in the event of future pandemics and immunization efforts.## Methods

To estimate the number of infections averted from the various vaccine programs, we first used available data to estimate the epidemic curve during the period in the fall of 2009 when H1N1 was most prevalent. Once we had obtained an estimated epidemic curve, we used it in conjunction with reported vaccine administration data to fit the observed epidemic curve to the one generated by a mathematical model based on difference equations. We used a discrete-time version of the standard Kermack–McKendrick model to estimate infection spread within each state [

25

, , 27

]. In the model calibration process, we estimated the relevant parameters such as *R0*within each state. We estimated the*R0*for each state individually, because different states have different demographic, geographic, and cultural attributes. Moreover, states experienced the H1N1 outbreak at different times and implemented their vaccination programs in different ways [[6]

], so the extent of the infection varied markedly.We then estimated a different,

*nonobservable*flu wave curve, one assuming no available vaccine. This multistep process provided a data-informed, model-supported basis for estimating the positive effects, if any, of the vaccine as administered in each of the states.Before describing the methodology in full, it is worth noting some common drawbacks to using a discrete-time model that implies homogeneous mixing in the population. Each state's population is segmented into groups of varying susceptibility, infectivity, and activity levels, each of which may strongly influence the progression of the epidemic [

28

, 29

]. While all models are imperfect, each region we investigated was large enough that homogeneous mixing provided a reasonable estimate for modeling the epidemic curve. This methodology was used throughout the H1N1 outbreak [18

, 30

]. In addition, while epidemics occur in continuous time, our use of a discrete-time model both simplified calculation and allowed us to incorporate the fact that during the 2009–2010 H1N1 pandemic, vaccines were shipped on a weekly basis and vaccine stocks were increased at discrete time intervals.### Epidemic curve estimation

We first estimated the true epidemic curve within the different states, in which the flu curve includes the effects of vaccinations. Because this virus was so prevalent in the US population, it was impossible to record an accurate epidemic curve in each state. There was no direct way to know the total number of people infected because 1) many such individuals did not present themselves to medical authorities and 2) for those who did visit a physician, confirming tests for H1N1 were not routinely ordered. Instead, we used the data released by the states' health departments that included the weekly percentage of all hospitalizations and outpatient visits resulting from ILI (%ILI) over the 2009–2010 flu seasons [

[7]

]. From that, we estimated the number of H1N1 infections for each week. We assumed that the total number of cases was directly proportional to the total number of H1N1 hospitalizations and outpatient visits. While the %ILI curve provided a good indication of the spread of infection, we did not use it as the epidemic curve because the total number of hospitalizations and outpatient visits associated with all medical conditions changes throughout the year. Each point on the %ILI temporal curve represents the percentage of the total number of hospitalizations and outpatient visits that are specific to H1N1. Because the flu wave first grows and then declines, this total number is not uniform throughout the observation period. We expect a higher number of total hospitalizations and outpatient visits during the peak of H1N1, with the number of non-H1N1 hospitalizations and outpatient visits remaining relatively stable throughout the several-month observation period. By using this assumption, we performed a simple transformation on the %ILI curve to obtain a new curve that we considered to be directly proportional to the experienced flu wave curve.To carry out the required transformation, we split our timeline into discrete “flu generation periods,” with generations represented by

*t = 1, 2, 3,..., tmax*. In our calculations, we let a single generation of the flu be equal to 10/3 = 2.33 days. Assuming just one number for the duration of the flu is a simplification. While actual generation of the flu varies with different viruses as well as on an individual level, we made this simplification to keep our discrete-time epidemic curve model tractable. The value we chose is consistent with the analysis of H1N1 in Mexico done in May 2009 [17

, 20

].We let

$C(t)=\text{thenumberofnonfluhospitalpatientsingeneration}t.$

Invoking our assumption of a constant number of such patients over time, we expressed

$C(t)={C}_{0}t=\text{\hspace{0.17em}}1,\text{\hspace{0.17em}}2,\text{\hspace{0.17em}}3,\mathrm{.\hspace{0.17em}.\hspace{0.17em}.}\text{,\hspace{0.17em}}{t}_{\mathit{max}}\mathrm{.}\text{\hspace{0.17em}}$

Next we considered the unobservable number of flu patients seeking hospital medical advice:

Solving for

or

or $N(t)=\frac{{C}_{0}F(t)}{[1-F(t)]}\mathrm{.}$

*N(t)*= the number of patients with H1N1 who visit a hospital in generation*t*, for*t = 1, 2, 3,..., tmax*. What we observed in reported data was*F(t)*, the fraction of all hospital visits that were attributable to H1N1,$F(t)=\frac{N(t)}{N(t)+{C}_{0}}\mathrm{.}$

Solving for

*N(t)*, we calculated$F(t)[N(t)+{C}_{0}]=N(t),$

or

$F(t){C}_{0}=N(t)[1-F(t)],$

or $N(t)=\frac{{C}_{0}F(t)}{[1-F(t)]}\mathrm{.}$

Invoking the assumption that the total number of H1N1 cases is proportional to the number of H1N1-related hospital visits, we deduced that the shape of the epidemic curve was proportional to

*N(t).*Because we had access to*F(t)*only, the value of*C0*, the total number of non-H1N1 hospital visits per generation, was unknown. While we did not know that quantity, we could approximate the value of*C0*because we knew the total number of H1N1 visits during our time period. Specifically, we let*I*be the total number of H1N1 visits in a US state. Then,$\sum _{t=1}^{{t}_{\mathrm{max}}}[N(t)+{C}_{0}]=}{\displaystyle \sum _{t=1}^{{t}_{\mathrm{max}}}\frac{{C}_{0}F(t)}{1-F(t)}+{C}_{0}=I.$

Consequently, we calculated

${C}_{0}=\frac{I}{{t}_{\mathrm{max}}+{\displaystyle {\sum}_{t=1}^{{t}_{\mathrm{max}}}\frac{F(t)}{1-F(t)}}}\mathrm{.}$

We approximated

*I*by using information from the CDC regarding the total number of infections in the United States.This transformation gave us a closer estimate of the actual epidemic curve than did the %ILI curve by itself. We were still faced with the fact, however, that people presented to doctors and hospitals at different rates throughout the pandemic. One might conjecture that the rate of hospital

*visits*would have been higher at the beginning of the pandemic, when media reports were implying a real danger from flulike symptoms, and that the rate would decrease toward the end of the epidemic, when reports had confirmed that the consequences of H1N1 were relatively mild; but the rate of hospital*admissions*would not be affected by such psychological factors. Furthermore, there are still other factors, such as parents of symptomatic children not wishing to present their children to medical facilities for fear of their contracting the illness, should they not already have it. One can keep conjecturing ad infinitum. It is difficult to quantify these myriad possible effects, and so we did not include them in the transformation.According to the CDC, the percentage of people infected with H1N1 in the United States ranged from 14% to 29% of the population [

[19]

]. We tried a range of values for the total number of infections in the region and fit our model to each one by finding the parameters that minimized the mean square error between the epidemic curve and the resulting modeled curve. To find the best fit, we modified three parameters: the total number of infections, the number of infections at the start of the epidemic (“patient zeros”), and *R0*. For each fit, we took the resulting fitted value of*R0*. The values were usually close together, and we chose the value of*R0*that best fit the exponential growth of cases near the onset of the epidemic.### A set of difference equations

We used a set of difference equations that corresponds to a discrete-time version of a classic Susceptible-Infectious-Recovered (S-I-R) model [

25

, , 27

, 31

].For each state, we adjusted model parameters to obtain a best fit between the reported data, as transformed above, and the model-generated flu wave epidemic curve. This process led to a direct estimation of

*R0*.We let

*R(t)*= the mean number of new infections generated by a randomly selected infectious (asymptomatic) patient at generation*t*of the epidemic.*t = 1, 2, 3,..., tmax*.#### The epidemic curve

Suppose the number of infected people at “generation zero” is 1 person. That is, there is 1 “patient zero.” Then, the mean number of infected people in generation 1 is

*R*_{0}persons, each of whom subsequently infects on average*R*(1) persons in generation 2, and so on. So, in generation*t*, the mean number of people infected, MI(*t*), will be the product*R*_{0}×*R*(1) ×*R*(2) × . . . ×*R*(*t*− 1). More compactly, assuming that MI(0) = 1, we calculated$\text{MI}(t)={R}_{0}{\displaystyle \prod _{n=1}^{t-1}R(n),\text{\hspace{0.17em}}t=\text{\hspace{0.17em}}1,\text{\hspace{0.17em}}2,\text{\hspace{0.17em}}3,\mathrm{.\hspace{0.17em}.\hspace{0.17em}.}\text{,}\text{\hspace{0.17em}}{t}_{\mathit{max}}}$

(2)

where MI(

*t*) is the mean number of infections at time*t*.#### Determining *R(t)*

Suppose the total population is N persons, all susceptible at the beginning of the epidemic and all mixing homogeneously—whether or not they are susceptible or have recovered and have immunity. Then suppose at generation

*t*we have*I*(*t*) people recovered and immune, recirculating in the population. Because of homogeneous mixing,*R0*is now modified by a factor of [*N−I(t)−*MI*(t)]/N*, to become$R(t)={R}_{0}\frac{N-I(t)-\text{MI}(t)}{N},\text{\hspace{0.17em}}t=\text{\hspace{0.17em}}1,\text{\hspace{0.17em}}2,\text{\hspace{0.17em}}3,\mathrm{.\hspace{0.17em}.\hspace{0.17em}.}\text{,}\text{\hspace{0.17em}}{t}_{\mathit{max}}\mathrm{.}$

(3)

#### Determining I(t)

At generation

*t*we assumed that there were a total of I(*t*) individuals immune to the disease. This immunity can be derived from having been vaccinated or from having recovered from the disease. For simplicity in this model, we assumed that individuals infected in generation (*t*− 1) recovered and were immune in generation*t*. We also assumed that during the time of generation (*t*− 1),*V*(*t*− 1) individuals received vaccine that made them immune for the first time in generation*t*. Thus, we wrote a difference equation for determining the value of*I*(*t*):$I(t)=I(t-1)+\text{MI}(t-1)+V(t-1),t=\text{\hspace{0.17em}}1,\text{\hspace{0.17em}}2,\text{\hspace{0.17em}}3,\mathrm{.\hspace{0.17em}.\hspace{0.17em}.}\text{,\hspace{0.17em}}{t}_{\mathit{max}}\mathrm{.}$

(4)

We now had a model consisting of the following three equations where

*V*(*t*) is known for all*t*.$\text{MI}(t)={R}_{0}{\displaystyle \prod _{n=1}^{t-1}R(n)\text{,}\text{\hspace{0.17em}}t=\text{\hspace{0.17em}}1,\text{\hspace{0.17em}}2,\text{\hspace{0.17em}}3,\mathrm{.\hspace{0.17em}.\hspace{0.17em}.}\text{,\hspace{0.17em}\hspace{0.17em}}{t}_{\mathit{max}}}$

$R(t)={R}_{0}\frac{N=I(t)-\text{MI}(t)}{N},\text{\hspace{0.17em}}t=\text{\hspace{0.17em}}1,\text{\hspace{0.17em}}2,\text{\hspace{0.17em}}3,\mathrm{.\hspace{0.17em}.\hspace{0.17em}.}\text{,\hspace{0.17em}}{t}_{\mathit{max}}$

$I(t)=I(t-1)+\text{MI}(t-1)+V(t-1),\text{\hspace{0.17em}}t=\text{\hspace{0.17em}}1,\text{\hspace{0.17em}}2,\text{\hspace{0.17em}}3,\mathrm{.\hspace{0.17em}.\hspace{0.17em}.}\text{,\hspace{0.17em}}{t}_{\mathit{max}}$

These equations can be solved iteratively, via recursion starting with the boundary conditions MI(0),

*I*(0), and*R*(0) =*R*_{0}.### Estimating the effects of vaccine

#### The nonobservable curve

Suppose, in the absence of vaccines, we have a pandemic infection curve

*C*(*t*). Here,*C*(

*t*) = the number of new flu infections reported during generation

*t*of the infection period, assuming no vaccine.

*t = 1, 2, 3,..., tmax*.

We called

*C*(*t*) the*base curve*, that is, the infection wave that occurs in the absence of vaccines. Also, for the sake of simplicity we assumed that human behavior, as illustrated, for instance, by hygienic steps and other nonpharmaceutical interventions, remained unchanged during the course of the infection period. This base curve is unobservable because the state-specific data reporting numbers of people infected included the effects of administered vaccines.#### Timing of vaccinations and subsequent immunity

To depict the effect of vaccinations on the population, we used

*V(t*), which is the number of individuals who have been vaccinated and first acquired immunity during generation*t*, with*t = 1, 2, 3,..., tmax*.We assumed that only susceptible individuals received vaccinations.

On average, vaccines take effect about 2 weeks after they have been administered. In our approximations, we compared the numbers calculated in the cases in which the vaccines took effect immediately as well as the cases in which vaccines took effect after 2 weeks. Similarly, we considered vaccine effectiveness (i.e., creating immunity to H1N1) to be between 75% and 100%. According to preliminary studies, the effectiveness of H1N1 vaccine was well over 90% in 10 days [

[32]

], and so the actual effect of the vaccines should be well within the bounds of our analysis.#### The observable curve

Another pandemic infection curve,

*C(t, V⃗)*, is the number of new flu infections reported during generation*t*of the flu, given that immunities due to vaccinations occur according to the known time vector V⃗, with*t = 1, 2, 3,..., tmax*.Our state-derived data depicted the vaccine-affected flu wave

*C(t, V⃗)*, and we wished to infer the (unobservable) vaccine-free flu wave*C(t).*To estimate the effect of the vaccinations, we needed to estimate*C*(*t*) and then compare it to*C(t, V⃗)*. The difference in the areas under the respective curves represented our estimate of the number of infections averted because of the vaccine.To estimate

*R0*, we fit the model-generated*C(t, V⃗)*curve to the empirically estimated epidemic curve, which includes the effects of vaccinations. The influenza cases during the H1N1 epidemic were severely underreported. The underreporting appeared to be especially significant at the beginning and end of the outbreak [19

, 33

]. To avoid this “statistical noise” effect, we fit the model-based epidemic curve to the data surrounding the peak of the epidemic, specifically to the part of the empirical curve that contains 75% of the cases. To find a best reasonable fit, we used the difference Equations (2), (3), (4), and manipulated the parameters *R0*and MI(O). The best fit was determined to be one for which the peak of the model-determined epidemic curve coincided with the estimated real epidemic curve and for which exponential growth of the early stages of the outbreak coincided with the real epidemic curve. The*R0*obtained by using this method might be a slight underestimate of the true*R0*for the H1N1 pandemic because it was not fitted at the very beginning of the outbreak when data were very noisy, but only once clear exponential growth was established. This methodology, however, is consistently used in the literature and provides a reasonable estimate for the basic reproductive number [16

, 34

].## Results

### Oklahoma in detail

Consider as an illustrative example the estimation process for Oklahoma. Figure 2 shows 1) the empirical-based estimated epidemic curve for Oklahoma created using the transformation described above and 2) the time-sequenced vaccine administration data reported by the state. In Figure 3, we again included the empirical epidemic curve and three model-generated epidemic curves:

- 1The curve generated by using the Oklahoma-reported vaccine administration data, fitted to correspond best to the empirical epidemic curve;
- 2The curve generated in the hypothetical case in which vaccines were not administered at all; and
- 3The curve generated in the hypothetical case in which vaccines were administered 2 weeks earlier than had actually occurred.

The total number of infections caused by the outbreak in Oklahoma was calculated by the area under an epidemic curve. The effect of the vaccines administered in Oklahoma was determined by calculating the area between the “actual” model-generated curve and the “no-vaccine” model-generated curve (Fig. 4) .

We analyzed 11 states in detail and inferred the total number of infections that were prevented as a result of their respective immunization programs. The states Illinois, Indiana, Massachusetts, Mississippi, Montana, New Jersey, New York, North Dakota, Oklahoma, South Carolina, and Virginia graciously provided us with precise data on vaccines as they were dispensed throughout the outbreak. Here we display two cases for each state:

- 1The optimistic case, in which all vaccines are effective immediately and are 100% effective, and
- 2The pessimistic case, in which vaccines are effective 2 weeks after administration and are effective for only 75% of the individuals receiving the vaccine.

The actual effect of the vaccine should lie within the range specified by these two cases.

It is possible that some ILI-related cases were reported with a delay of 1 or 2 weeks with respect to symptom onset, because infected individuals may be more likely to visit a medical professional after symptoms become severe. If that is the case, our estimated epidemic curve might be shifted to the left by a few generations. If true, the infections occurred even earlier than what we had assumed, and so vaccine arrival occurred even later with respect to the peak of infections. This would imply that our results are an optimistic estimate of the effectiveness of states' vaccine programs.

## Summary

In Table 1 we present the results of analysis of 11 states with detailed vaccination information. While we have detailed data for only these 11 states, we note that the range of estimated values for

*R*_{0}is fairly narrow, with higher values falling on more populated states such as New York and Illinois and lower values corresponding to states with lower population density such as Oklahoma and Montana. Geographic proximity also seems to result in similar*R*_{0}values.Table 1Summary of best-fitted values for

*R*_{0}and model-determined effects of vaccines as they were distributedState | Estimated R_{0} | % Population vaccinated | Optimistic scenario: infections averted (in % of total population) | Pessimistic scenario: infections averted (in % of total population) |
---|---|---|---|---|

Illinois | 1.21 | 9 | 3.43 | 1.2 |

Indiana | 1.15 | 20 | 4.28 | 1.81 |

Massachusetts | 1.16 | 29 | 13.71 | 6.84 |

Mississippi | 1.16 | 8 | 0.13 | 0.05 |

Montana | 1.15 | 20 | 2.81 | 1.04 |

New Jersey | 1.20 | 12 | 3.36 | 1.1 |

New York | 1.20 | 14 | 3.23 | 1.12 |

North Dakota | 1.16 | 27 | 2.95 | 1.06 |

Oklahoma | 1.14 | 13 | 2.29 | 0.93 |

South Carolina | 1.16 | 8 | 0.4 | 0.12 |

Virginia | 1.19 | 22 | 1.77 | 0.52 |

In the optimistic scenario, vaccines are 100% effective and take effect immediately. In the pessimistic scenario, vaccines are 75% effective and take effect 2 wk after being administered.

More detailed information for all states is included in Appendix A in Supplemental Materials at doi:10.1016/j.jval.2011.07.014.

## Discussion

While examining the estimated numbers of infections averted, we can identify two major contributing factors. The first is the total number of vaccines administered to the general population. The second is the timing of the vaccine administration with respect to the peak of the infections. Once the H1N1 virus had been identified as a potentially devastating pandemic in the spring of 2009, the CDC worked to develop and distribute H1N1 vaccines. These vaccines were sent out to the individual states at the same time, and first doses were administered on October 5, 2009. These vaccines, however, had varying effects because the peak of the outbreak in different states occurred at markedly different times.

The peak of infection usually occurs when herd immunity occurs, that is, the time when

*R(t)*= 1, and every infectious person at that time infects on average just one other person [[35]

]. So, at the time of herd immunity, the number of infections in the next generation is approximately the same as it was in the previous one. We say “approximately” because of statistical fluctuations in the actual number of susceptible people that any one newly infectious person will infect. Soon afterward, infectious people no longer replace themselves in society, and the number of infected and infectious people in each generation decreases. Early administration of vaccines decreases the number of people who still need to be infected before herd immunity is achieved, and so decreases the height of the peak. Late administration of the vaccine has almost no effect on the dynamics of the outbreak and has little benefit to the society other than immunizing the people who received the vaccine. Such late immunizations may be important if the flu were to return later in a new wave.Consider again the southeastern states of the United States, the first region to report infection peaks. As early victims they received vaccines after the worst of the infection had already passed. Louisiana, Indiana, and South Carolina had not started administering vaccines until

*after*the peak of outbreak. And these states were least successful in averting infections. On the other hand, Massachusetts and Virginia started administering their vaccines 5 and 3 weeks, respectively, before their respective peaks. These two states enjoyed a particularly good impact from their vaccination programs. Massachusetts, in addition to having 5 weeks of vaccinations prior to the H1N1 peak, vaccinated 29% of its population, the most of any state in our sample. As a result, as much as 7% to 14% of the population may have been spared infection and possible complications from influenza. While Massachusetts and Virginia had effective experiences with their vaccinations, most states did not. On average for our limited sample, vaccines were delivered just before the peak of the states' outbreaks.To quantify the effect of time in averting infection we considered one of the states that vaccinated almost 20% of its population, Indiana. Hypothetically, if the same number of vaccines had been delivered just 2 weeks earlier, more than twice the number of infections could have been averted.

With a more granular approach, we considered the marginal benefit of administering just one vaccination at a given time. We mapped the total projected number of infections that could be averted if just one vaccination were to be administered to a susceptible person at different times during the outbreak. That is, we calculated the total number of infections that would occur in Indiana if exactly one vaccine were administered at different points in time and compared that number to the total number of infections that would happen if no vaccines were administered at all. The differences are presented in Figure 5. As expected, administered vaccines have a monotonically decreasing benefit with respect to time. A striking feature of Figure 5 is the fact that one vaccination to a susceptible person well before the flu wave starts averts almost two infections in the population, even with a low value for

*R*_{0}(1.15) and even considering the fact that the vaccinated person has a greater than even chance of never becoming infected assuming no vaccination. Clearly, vaccines administered well before the peak carry the added benefit of diluting the susceptible population with immune people and are particularly useful in mitigating the spread of infection.Another insightful feature of this graph is the slope of the marginal benefit curve, which represents the time dependence of effective vaccines. While starting vaccine administration in Indiana in July would be most effective for Indiana, the effect of these vaccines would not change significantly until the beginning of September. That is, if vaccines were to be available in July, Indiana could have waited to receive its share until September with minimal losses. Similarly, vaccines received after December will have the same (minimal) effect whether they are administered in December or February. The effectiveness of vaccines, however, is extremely time sensitive from the end of September to mid-November, where each week results in a significant loss of effectiveness. Vaccines that become available during this critical period need to be administered as soon as possible. These results encourage us to recommend a more detailed cost–benefit analysis of trying to get some vaccine, even if in much smaller quantities, to the states at the beginning of this “critical period,” when the population is particularly sensitive to the timing of vaccination. A small amount of vaccine delivered early should have a more significant effect on the total number of infections than a batch delivered just a few weeks later.

Furthermore, we conjecture that the timeliness of the vaccines is also closely related to the total amount of vaccine accepted by the population. While the CDC distributed its vaccines proportionally to the population of each region, states varied in the amount of vaccine that was actually used. For instance, Mississippi used less than 40% of its allocated vaccine, most likely due to “flu fatigue.” While the media are particularly helpful at warning the public of an ongoing pandemic and encouraging the use of nonpharmaceutical interventions, they can also give the impression that the outbreak is over or has been blown out of proportion. Once the peak of the outbreak had passed and H1N1 had been determined to be less dangerous than was originally feared, the populations of the “early victim” states would be less likely to spend their time and risk perceived possible side effects of getting a flu shot.

Looking at all 50 states and comparing the percentage of vaccinated population by the end of the outbreak to the week of vaccine delivery [

[36]

], we notice in Figure 6 a weak negative correlation between the timing of vaccine delivery with respect to the peak, and the total percentage of the population that accepted vaccinations. These results are consistent with the position of Harris et al. [[22]

] that had the vaccines been delivered earlier to the states, more people would have been encouraged to accept a vaccination.This effect would be particularly relevant to South Carolina, which started administering its vaccines 1 week after the peak of infection and subsequently managed to vaccinate only 8% of its population. If these vaccinations were to have started earlier, before the peak, we could hypothesize not only an increase in effectiveness from timing alone but also a higher participation rate in the vaccination program. Early administration is particularly important in that it increases the efficacy of vaccine along with encouraging people to accept vaccination.

### Vaccine allocation

As shown in Figure 7, in the first few weeks of vaccine distribution, when demand for vaccine clearly exceeded supply, the CDC allocated vaccine to states proportionally to their populations [

[4]

]. Particularly in early October, this simple distribution scheme ensured that all states received amounts that could be used to immunize the same proportion of the population. Come November, those states that saw little demand started placing fewer orders for influenza vaccine, while those with later epidemics such as Massachusetts and Virginia were still experiencing high demand and were shipped larger quantities of vaccine, confirming the intuition from Figure 6.The previous sections imply that the same vaccines administered in states that had already experienced the peak of the infection at the time vaccines started arriving were much less effective than those administered in states that had not yet experienced the peak. Moreover, the states that were past the peak saw less demand for vaccines and thus used only a small fraction of their allocated vaccines. Consider a side-by-side analysis of Mississippi and North Dakota in Table 2.

Table 2Comparison of vaccination programs in North Dakota and Mississippi

North Dakota | Mississippi |
---|---|

• Started administration 1 wk before peak | • Received vaccines 6 wk after peak |

• A hypothetical single batch of 32,342 vaccines (5% of the population) administered on October 10 averts 33,745 infections (5.2% of the population) | • A hypothetical single batch of 147,599 vaccines (5% of the population) administered on October 10 averts 27,173 infections (0.9% of the population) |

• In the first 4 wk administered 62% of available vaccines | • In the first 4 wk administered less than 25% of available vaccines |

It is clear that vaccines administered in North Dakota were significantly more effective than those in Mississippi. In fact, the Mississippi vaccines had almost no effect because the infection was barely spreading by the time vaccines became available. Coupled with this, North Dakota was experiencing more demand for the vaccines at the beginning of its program. Based on this analysis, we believe that there is a need for more effective procedures for allocating vaccines to US states.

Naturally, allocating all of Mississippi's vaccines to North Dakota would be not only unethical but also politically infeasible. Instead, as a thought experiment, suppose that just 20% of Mississippi's

*unused*vaccine were to be transferred to North Dakota during the first 4 weeks of vaccine distribution. Suppose that with this addition, 60% of the new vaccines were actually administered. This additional vaccine would decrease the total number of infections in North Dakota by 5%. That is a significant improvement for a relatively small cost. An adaptive decision such as this can be made during the allocation process. We can form even approximate predictions about how much vaccine will actually be demanded by the state and how effective the extra vaccines would be in reducing infections. For example, by using data collected from our 11 states, we can weakly estimate that a state that had experienced peak infection 6 weeks earlier can be expected to vaccinate no more than 4% of its population within the first 4 weeks. In the first 4 weeks of the 2009 H1N1 outbreak, the CDC allocated to Mississippi enough vaccine to cover 7% of its population. With accurate data, some portion of that could have been redirected to states that were more likely to use and benefit from the vaccine.## Conclusions

Our analysis shows the importance of the timing of vaccinations for infectious respiratory diseases such as influenza. We emphasize the need to start administering vaccines well before the peak of an influenza outbreak. Moreover, when a governing body such as the CDC is faced with the allocation decision, it is important to take into consideration the stages of the outbreaks in different regions and to deploy vaccine with preference to the regions that are projected to administer more of the vaccine with greater beneficial effects.

In our analysis of individual state immunization programs we used a relatively simple model, assuming a homogeneous population and a deterministic model structure. As a result, our model, as most models, is not an exact picture of what happened during the fall of 2009, but rather a tool to gain insight about strategies that could help the public mitigate the effect of influenza pandemics. While the exact numbers almost surely differ from the estimates, the relative results should hold under a range of assumptions about

*R0*vaccine efficacy and the generation period of the flu.## Acknowledgments

We thank Dr. Stan Finkelstein, Dr. Sahar Hashmi, Kallie Hedberg, and Julia Hopkins for helpful comments on an earlier draft.

Source of financial support: Work on this study was supported by the Sloan Foundation of New York under a grant titled “Decision-Oriented Analysis of Pandemic Flu Preparedness & Response” and under a cooperative agreement with the US Centers for Disease Control and Prevention (CDC) , grant number 1 PO1 TP000307-01 , “LAMPS (Linking Assessment and Measurement to Performance in PHEP Systems), awarded to the Harvard School of Public Health Center for Public Health Preparedness (HSPHCPHP) and the Massachusetts Institute of Technology (MIT), Center for Engineering Systems Fundamentals (CESF). The discussion and conclusions in this article are those of the authors and do not necessarily represent the views of the Sloan Foundation, the CDC, the US Department of Health and Human Services, Harvard, or MIT.

## Supplemental Materials

- Appendix A

## References

- The 2009 H1N1 pandemic: summary highlights, April 2009-April 2010.(Accessed January 20, 2011)
- World now at the start of 2009 influenza pandemic.(Accessed January 20, 2011)
- Global Alert and Response (GAR): pandemic (H1N1) 2009- update 112.(Accessed January 20, 2011)
- Graph and table of 2009 H1N1 influenza vaccine doses allocated, ordered, and shipped.(updated February 2, 2010) (Accessed July 29, 2010)
- Press briefing transcripts.(updated September 3, 2009) (Accessed June 10, 2011)
- 2009 H1N1 after action reports: lessons on vaccine distribution.(Accessed October 18, 2011)
- CDC, seasonal influenza (flu), weekly report: influenza summary update.(updated May 28, 2010) (Accessed July 29, 2010)
- Questions and answers: seasonal flu shot.(Accessed January 20, 2011)
- Strategy for distribution of influenza vaccine to high-risk groups and children.
*Am J Epidemiol.*2005; 161: 303-306 - Finding optimal vaccination strategies for pandemic influenza using genetic algorithms.
*J Theor Biol.*2005; 234: 201-212 - Adaptive vaccination strategies to mitigate pandemic influenza: Mexico as a case study.
*PLoS One.*2009; 4: e8164 - Living with influenza: impacts of government imposed and voluntarily selected interventions.
*EJOR.*2009; 195: 613-627 - Engineering reponses to pandemics.
*Info Knowl Sys Manag.*2009; 8: 311-339 - Effectiveness and cost-effectiveness of vaccination against pandemic influenza (H1N1) 2009.
*Ann Intern Med.*2009; 151: 829-831 - Is a mass immunization program for pandemic (H1N1) 2009 good value for money?.
*Vaccine.*2010; 28: 6210-6220 - Pandemic potential of a strain of influenza A (H1N1): early findings.
*Science.*2009; 324: 1557-1561 - Estimation of the reproductive number and the serial interval in early phase of the 2009 influenza A/H1N1 pandemic in the USA.
*Influenza Other Respi Viruses.*2009; 3: 267-276 - Pandemic H1N1 influenza: predicting the course of a pandemic and assessing the efficacy of the planned vaccination programme in the United States.
*Euro Surveill.*2009; 14: 19358 - Updated CDC estimates of 2009 H1N1 influenza cases, hospitalizations and deaths in the United States, April 2009–March 13, 2010.(Accessed December 15, 2010)
- Estimated epidemiologic parameters and morbidity associated with pandemic H1N1 influenza.
*CMAJ.*2010; 182: 131-136 - Vaccine availability in the US following the 2009 H1N1 outbreak.
*Am J Disaster Med.*2011; 6: 23-30 - Influenza vaccine—safe, effective, and mistrusted.
*N Engl J Med.*2010; 363: 2183-2185 - H1N1 vaccine: what physicians can do with leftover doses.(Accessed July 19, 2010)
- The public's response to the 2009 H1N1 influenza pandemic.
*N Engl J Med.*2010; 362: e65 - Contributions to the mathematical theory of epidemics.
*Proc R Soc Lond.*1927; 115: 700-721 - Mathematical Biology.in: Springer, Berlin1989: 610-650
- Modeling infectious diseases in humans and animals.Princeton University Press, Princeton, NJ2007
- Simple models of influenza progression within a heterogeneous population.
*Oper Res.*2007; 55: 399-412 - Modeling and responding to pandemic influenza: importance of population distributional attributes and non-pharmaceutical interventions (Doctoral dissertation 2009).(Accessed February 25, 2011)
- Modeling influenza epidemics and pandemics: insights into the future of swine flu (H1N1).
*BMC Med.*2009; 7: 30 - Mathematical Epidemiology of Infectious Diseases: Model Building, Analysis and Interpretation.Wiley, New York2000
- Response after one does of a monovalent influenza A (H1N1) 2009 vaccine—preliminary report.
*N Engl J Med.*2009; 361: 2405-2413 - Underreporting of 2009 H1N1 influenza casesfrom.(Accessed June 10, 2011)
- Transmissibility of 1918 pandemic influenza.
*Nature.*2004; 432: 904-906 - The concept of herd immunity and the design of community-based immunization programmes.
*Vaccine.*1992; 10: 928-935 - Interim results: State-specific influenza A (H1N1) 2009 monovalent vaccination coverage—United States, October 2009–January 2010.(Accessed January 20, 2011)

## Article Info

### Publication History

Published online: December 05, 2011

### Footnotes

Conflicts of interest: The authors have no conflicts of interest to report.

### Identification

### Copyright

© 2012 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc.

### User License

Elsevier user license | How you can reuse

Elsevier's open access license policy

Elsevier user license

## Permitted

### For non-commercial purposes:

- Read, print & download
- Text & data mine
- Translate the article

## Not Permitted

- Reuse portions or extracts from the article in other works
- Redistribute or republish the final article
- Sell or re-use for commercial purposes

Elsevier's open access license policy