Advertisement

Cost-Effectiveness Analysis for Influenza Vaccination Coverage and Timing in Tropical and Subtropical Climate Settings: A Modeling Study

  • Mu Yue
    Affiliations
    School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, China

    Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore
    Search for articles by this author
  • Borame L. Dickens
    Affiliations
    Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore
    Search for articles by this author
  • Joanne Su-yin Yoong
    Affiliations
    Center for Economic and Social Research, University of Southern California, Los Angeles, CA, USA

    Yong Loo Lin School of Medicine, National University of Singapore, Singapore
    Search for articles by this author
  • Mark I-Cheng Chen
    Affiliations
    Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore
    Search for articles by this author
  • Yot Teerawattananon
    Affiliations
    Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore

    Health Intervention and Technology Assessment Program, Ministry of Public Health, Nonthaburi, Thailand
    Search for articles by this author
  • Alex R. Cook
    Correspondence
    Address correspondence to: Alex R. Cook, PhD, 12 Science Drive 2, #10-01, Singapore 117549.
    Affiliations
    Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore

    Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore

    Department of Statistics and Applied Probability, National University of Singapore, Singapore
    Search for articles by this author
Open AccessPublished:August 19, 2019DOI:https://doi.org/10.1016/j.jval.2019.07.001

      Highlights

      • Influenza transmission in the tropics occurs year-round, and epidemics are less pronounced and less predictable than in areas where there is a peak influenza season.
      • Using equatorial Singapore as a case study, we developed a simulation model of influenza transmission that accounts for individual and population trajectories of antibodies after infection or vaccination and considered varying degrees of seasonal forcing.
      • This model was embedded within a health economic framework to assess the cost-effectiveness of once- and twice-yearly influenza vaccinations.

      Abstract

      Background

      The lack of seasonality in influenza epidemics in the tropics makes the application of well-established temperate zone national vaccination plans challenging.

      Objectives

      We developed an individual-based simulation model to study optimal vaccination scheduling and assess cost-effectiveness of these vaccination schedules in scenarios of no influenza seasonality and the seasonality regimes of Singapore, Taipei, and Tokyo.

      Methods

      The simulation models heterogeneities in human contact networks, levels of protective antibodies following infection, the effectiveness of the influenza vaccine, and seasonality. Using a no intervention baseline, we consider 3 alternative vaccination strategies: (1) annual vaccination for a percentage of the elderly, (2) biannual vaccination for a percentage of the elderly, and (3) annual vaccination for all elderly and a fraction of the remaining population. We considered 5 vaccination uptake rates for each strategy and modeled the estimated costs, quality-adjusted life years, and incremental cost-effectiveness ratios (ICERs), indicating the cost-effectiveness of each scenario.

      Results

      In Singapore, annual vaccination for a proportion of elderly is largely cost-effective. However, with fixed uptake rates, partial biannual vaccination for the elderly yields a higher ICER than partial annual vaccination for the elderly, resulting in a cost-ineffective ICER. The most optimal strategy is the total vaccination of all the elderly and a proportion of individuals from other age groups, which results in a cost-saving ICER. This finding is consistent across different seasonality regimes.

      Conclusions

      Tropical countries like Singapore can have comparably cost-effective vaccination strategies as found in countries with winter epidemics. The vaccination of all the elderly and a proportion of other age groups is the most cost-effective strategy, supporting the need for an extensive national influenza vaccination program.

      Keywords

      Introduction

      Globally, the estimated health and financial burden of annual seasonal influenza is high at 291,000-646,000 associated respiratory deaths,
      • Iuliano A.D.
      • Roguski K.M.
      • Chang H.H.
      • et al.
      Estimates of global seasonal influenza-associated respiratory mortality: a modelling study.
      and the per capita cost of a case of influenza illness ranges from $30 to $64.
      • Peasah S.K.
      • Azziz-Baumgartner E.
      • Breese J.
      • Meltzer M.I.
      • Widdowson M.-A.
      Influenza cost and cost-effectiveness studies globally—a review.
      To dampen these seasonal epidemics, vaccination policies have been implemented safely and effectively for over 70 years.
      • Hampson A.W.
      Vaccines for pandemic influenza. The history of our current vaccines, their limitations and the requirements to deal with a pandemic threat.
      Several commercial vaccines are therefore being regularly released and deployed annually to reduce the severity of infection and mortality, especially among those in high-risk groups such as the elderly.
      Cost-effectiveness analyses assist decision making on large-scale investments such as vaccination programs at national levels. Evidence of the cost-effectiveness and benefits from influenza vaccination is ample across the literature, but the focus has been on countries with strong seasonality and therefore distinct influenza seasons, which is usually countries in which winter occurs. For example, Shields et al
      • Shields G.E.
      • Elvidge J.
      • Davies L.M.
      A systematic review of economic evaluations of seasonal influenza vaccination for the elderly population in the European Union.
      performed a systematic review of economic evaluations of seasonal influenza vaccination for the elderly population in the European Union and found that most studies showed that vaccination is cost-effective. Other analyses have assessed whether quadrivalent vaccines are cost-effective. You et al
      • You J.H.S.
      • Ming W.-K.
      • Chan P.K.S.
      Cost-effectiveness of quadrivalent influenza vaccine in Hong Kong—a decision analysis.
      found the quadrivalent influenza vaccine cost-effective for age groups outside of 15 to 64 years in Hong Kong, and Thommes et al
      • Thommes E.W.
      • Kruse M.
      • Kohli M.
      • Sharma R.
      • Noorduyn S.G.
      Review of seasonal influenza in Canada: burden of disease and the cost-effectiveness of quadrivalent inactivated influenza vaccines.
      also found quadrivalent inactivated vaccines highly cost-effective for Canada. Wang et al
      • Wang C.-S.
      • Wang S.-T.
      • Chou P.
      Efficacy and cost-effectiveness of influenza vaccination of the elderly in a densely populated and unvaccinated community.
      studied vaccination campaigns in Taiwan and concluded that 3 times the cost of vaccination is generated in savings. The potential for cost savings within tropical areas in which influenza infections occur year-round, such as Singapore
      • Lambach P.
      • Alvarez A.M.R.
      • Hirve S.
      • et al.
      Considerations of strategies to provide influenza vaccine year round.
      and Indonesia,
      • Root E.D.
      • Agustian D.
      • Kartasasmita C.
      • Uyeki T.M.
      • Simões E.A.
      Demographic and ecological risk factors for human influenza A virus infections in rural Indonesia.
      needs to be studied to assess whether vaccination is cost-effective in the absence of clear influenza seasons.
      Commonly used influenza vaccines induce the production of neutralizing antibody titers, which gradually wane within a year.
      • Osterholm M.T.
      • Kelley N.S.
      • Sommer A.
      • Belongia E.A.
      Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis.
      • Young B.
      • Zhao X.
      • Cook A.R.
      • Parry C.M.
      • Wilder-Smith A.
      • I-Cheng M.C.
      Do antibody responses to the influenza vaccine persist year-round in the elderly? A systematic review and meta-analysis.
      Thus, vaccination is often scheduled before peak influenza season in temperate countries where outbreaks occur in the winter months. In the tropics, however, the weak 2-peak influenza seasons

      Lai L. Singapore has two peak flu periods a year. The Straits Times. July 26, 2017. https://www.straitstimes.com/singapore/health/singapore-has-two-peak-flu-periods-a-year. Accessed August 28, 2018.

      and the lower climatic variability across the year can lead to distinct, quasi-random influenza outbreak patterns similar to the dynamics seen for other viruses, such as dengue.
      • Chen Y.
      • Cook A.R.
      • Lim A.X.L.
      Randomness of dengue outbreaks on the equator.
      At least 1 annual influenza epidemic will usually occur in these tropical regions,
      • Viboud C.
      • Alonso W.J.
      • Simonsen L.
      Influenza in tropical regions.
      and often 2,
      • Lee V.J.
      • Yap J.
      • Ong J.B.S.
      • et al.
      Influenza excess mortality from 1950–2000 in tropical Singapore.
      • Ang L.W.
      • Lim C.
      • Lee V.J.M.
      • et al.
      Influenza-associated hospitalizations, Singapore, 2004-2008 and 2010-2012.
      with year-round transmission, but predicting the timing of these epidemics can be challenging. Vaccination scheduling including the time of year and whether at-risk groups should be vaccinated once or twice a year are key concerns to address.
      The equatorial city-state of Singapore has very weak seasonality and a suggestion of 2 periods of elevated influenza transmission, consistently high temperatures, and well-established nationwide vaccination programs,
      Influenza
      Ministry of Health Singapore.
      making it a good candidate for an influenza vaccination program. In Singapore, 9.1% of the population are elderly (above 65 years of age), although this percentage is expected to rise to 30.8% by 2050.
      • Phan T.P.
      • Alkema L.
      • Tai E.S.
      • et al.
      Forecasting the burden of type 2 diabetes in Singapore using a demographic epidemiological model of Singapore.
      Ang et al
      • Ang L.W.
      • Lim C.
      • Lee V.J.M.
      • et al.
      Influenza-associated hospitalizations, Singapore, 2004-2008 and 2010-2012.
      found that the rate of influenza-associated hospitalizations in Singapore was 29.6/100 000 person-years during 2010 to 2012.
      • Ang L.W.
      • Lim C.
      • Lee V.J.M.
      • et al.
      Influenza-associated hospitalizations, Singapore, 2004-2008 and 2010-2012.
      They further found that persons above 75 years of age had influenza-related hospitalization rates 47 times higher than persons who were 25-44 years old and that approximately 12% of hospitalizations owing to pneumonia in Singapore were attributable to influenza.
      • Ang L.W.
      • Lim C.
      • Lee V.J.M.
      • et al.
      Influenza-associated hospitalizations, Singapore, 2004-2008 and 2010-2012.
      In addition, these researchers found that only 8.7% of Singaporeans aged 50 to 69 years reported having received seasonal influenza vaccination in the 2012 Health Behavior Surveillance of Singapore survey; this vaccine uptake rate was half that for young adults aged 18 to 29 years (16.9%).
      • Ang L.W.
      • Lim C.
      • Lee V.J.M.
      • et al.
      Influenza-associated hospitalizations, Singapore, 2004-2008 and 2010-2012.
      This may be due to the upfront cost of the influenza vaccine at $20 to $30 in public clinics. To boost the vaccination uptake rate, from 2014 onward, the national medical savings scheme Medisave allowed its use for the vaccination of those aged 65 and above, those aged 5 years and younger, pregnant women, and other at risk groups, removing the upfront payment obstacle.
      The Singaporean population experiences high mixing with relatively frequent travel and large influxes of daily travelers, situations that result in the introduction of new viruses and contribute toward the spread of viruses. An estimated 2.5 million residents aged 15 years and over have traveled at least once annually.
      • Tan Y.L.
      Overseas travel by Singapore residents.
      With continual introductions of viruses and higher morbidity rates among the at-risk populations, the implementation of a national program has the potential to significantly avert medical costs and reduce the disease burden.
      The aim of the present study was to perform cost-effectiveness analyses for different influenza vaccination strategies using tropical Singapore as a case study and using simulation modeling. We would then compare the findings for Singapore to those for areas with no seasonality and for areas with subtropical or temperate seasonality patterns and identify appropriate vaccination times for different seasonality regimes.

      Methods

      We developed an individual-based simulation model using R Statistical Software
      R Core Team
      R: a language and environment for statistical computing.
      to quantify the incremental economic value of vaccination and to evaluate the optimal timing of influenza vaccination in tropical Singapore, in seasonality regimes based on the seasonality of Taipei (Taiwan, China) and Tokyo (Japan), and with a no influenza seasonality baseline by measuring changes in incremental cost-effectiveness ratio (ICER). The time frame for the simulation model was 10 years. The simulation model was based on a population size of 10 000 with 1000 independent Monte Carlo simulations to obtain reliable comparisons between scenarios.

      Sources of Data

      Input variables are shown in Table 1. The detailed approach for estimating each input variable is shown in the Supplementary Materials found at https://doi.org/10.1016/j.jval.2019.07.001).
      Table 1Input variables used in analysis.
      Input variablesAge ranges
      0-1920-64≥65
      Mortality rate (%)0.0040.0050.061
      Earnings lost per death ($)1 631 2041 236 86330 789
      Hospitalization rate (%)0.1420.2050.535
      Total hospitalization cost ($)399054966116
      Average length of stay (d)3.884.616.2
      Average additional days222
      Cost per bed per day ($)701701701
      Hospitalization value loss per day215341215
      Transportation cost ($)4.203.304.70
      Outpatient rate (%)99.85499.79099.459
      Total outpatient cost ($)580822385
      Average days lost333
      Treatment cost ($)58.2058.2058.20
      Outpatient value loss per day172.40253.40107.50
      Transportation cost ($)4.203.304.70
      % Asymptomatic infections777777
      Total vaccination cost ($)374730
      Cost of vaccine6.486.486.48
      Nurse treatment cost ($)555
      Vaccination value loss per hour21.5831.7013.40
      Transportation cost ($)4.203.304.70
      Vaccine effectiveness (%)606048
      Note. Costs are reported in 2018 USD, and a 3% discount rate is used to devalue health effects.

      Antibody Response Following Infection or Vaccination

      We built an individual-level longitudinal model incorporating baseline antibody levels, the risk of infection, time of infection, and subsequent risk and decay of antibodies postinfection. We then estimated the antibody response after infection or vaccination and the protection conferred by varying antibody levels. We also assessed the optimal vaccination schedule through the calendar year.
      We adopted a previously published model of antibody titers developed from a cohort with multiple serological samples across 3 postpandemic waves.
      • Zhao X.
      • Ning Y.
      • Chen M.I.
      • Cook A.R.
      Individual and population trajectories of influenza antibody titers over multiple seasons in a tropical country.
      Antibody titers were modeled on a logarithmic scale (1 for 1:10, 2 for 1:20, etc), with titers rising and falling postinfection. The validity of this model has been previously demonstrated.
      • Zhao X.
      • Ning Y.
      • Chen M.I.
      • Cook A.R.
      Individual and population trajectories of influenza antibody titers over multiple seasons in a tropical country.
      The titer for individual i at time t was modeled by a normally distributed variable, Zit∼N (μit, σ2) The mean titer μit can be modeled as μit = Bi if individual i is neither infected nor vaccinated and Bi+Rif (tTi,ki,θi) otherwise, where Bi is the baseline titer level, Ri is the additional titer due to infection or influenza vaccination at the time of peak rise after infection, and f(x,kiθi)=1/(Γ(ki)θiki)xki1exp(x/θi) is a gamma density parameterized with shape ki and scale θi, both parameters that have been estimated in a previous study. Thus, the mean titer is constant if the individual has never been infected or has not been vaccinated, and a boost term is added to the baseline level if the individual becomes infected or is vaccinated.
      We assumed the parameters (BI, kI, θI, Ri) follow independent log-normal distributions, where the means and standard deviations are estimated from Zhao et al
      • Zhao X.
      • Ning Y.
      • Chen M.I.
      • Cook A.R.
      Individual and population trajectories of influenza antibody titers over multiple seasons in a tropical country.
      and are, respectively, μB = 0.58, sB = 0.024, μk = 1.21, sk = 0.024, μθ = 206.57, sθ = 35.10, μR = 1023.44, and sR = 72.59.
      Contact patterns by age groups estimated by Prem et al
      • Prem K.
      • Cook A.R.
      • Jit M.
      Projecting social contact matrices in 152 countries using contact surveys and demographic data.
      are used to adjust the risk of transmission between individuals. This was based on data from population-based contact diaries in 8 European countries and projected to 144 other countries using a Bayesian hierarchical model with Markov chain Monte Carlo simulation. The risk of infection for a susceptible individual i at day t was given by pit. If the individual was infected within the past 30 days, pit. is 0. Otherwise, it takes the form
      pit=[1e(α+β(ωtk,1NIt1+ωtk,2NIt2+ωtk,3NIt3))×St]×eθ×Zit,


      for age group k (=1 if individual i is aged 0-19 years, 2 if individual i is aged 20-64 years, and 3 if individual i is aged 65+ years), where ωtk,j is the number of contacts in age group j for age group k at day t, NItjis the number of infected people for age group j over the last 7 days before day t, θ indicates the protective effect conferred by antibody titers, which was −0.57 estimated from Zhao et al.
      • Zhao X.
      • Fang V.J.
      • Ohmit S.E.
      • Monto A.S.
      • Cook A.R.
      • Cowling B.J.
      Quantifying protection against influenza virus infection measured by hemagglutination-inhibition assays in vaccine trials.
      The parameters α and β allow for importation and autochthonous infection, respectively. Seasonal forcing, St, was taken to be
      st={exp{γsin(τ(t+t0)/365)},if1influenzaseasonexp{γ1sin(τ(t+t0,1)/d1)}+exp{γ2sin(τ(t+t0,2)/d2)},if2influenzaseason


      where γ, γ1, and γ2 determine the amplitude of the oscillation, τ is the transcendental circular constant ≈6.28, and d1 and d2 are the length in days for 2 influenza seasons. The method for estimating α, β, γ, γ1, γ2, d1, d2, t0, t0,1 and t0,2 under a no seasonality baseline and for Singapore-like, Taipei-like, and Tokyo-like scenarios is shown in Section 1 of the Supplementary Materials (found at https://doi.org/10.1016/j.jval.2019.07.001).
      These statistical models of antibody response and of the differential risk of infection for different antibody levels are combined with a simulation model that tracks individuals’ antibody levels and infection status with a composite strain modeled on influenza A (H1N1) with transmission between individuals. To prevent stochastic extinction, a small importation term (α) is included. Individuals in the simulation are initialized with a starting antibody level, and an arbitrary person is seeded to be infected at the beginning of each run. After infection, individuals remain infectious for 7 days and enter a 30-day refractory period in which immediate reinfection cannot occur.
      • Zhao X.
      • Ning Y.
      • Chen M.I.
      • Cook A.R.
      Individual and population trajectories of influenza antibody titers over multiple seasons in a tropical country.
      They are also assigned a time-varying boost term to their antibody levels with decreasing risk of reinfection over the first few months and increasing risk of reinfection after that. This infection pattern is modeled as a daily clock in a discrete time model to dictate risk of infection.
      We assumed the vaccine has the properties of the trivalent vaccine and treated the antibody response of a vaccinated individual as mimicking that following a natural infection.

      Optimal Time for Vaccination

      To determine the optimal period for vaccination in each seasonality scenario, we used a scenario in which there is a 40% vaccination coverage of the elderly on the first day of each year for 10 years. Other scenarios gave similar results (not shown). We defined I (t) to be the total number of infections on day t, and Aj (t) to be the antibody titer at day t for a person who is vaccinated annually on the jth day of each year. To ensure maximum effectiveness of influenza vaccination, we aimed to have a high titer value when incidence of influenza during that period was the highest, which corresponds to the time point when the 2 vectors, I (t) and Aj (t), had the most similar shapes. To do so, we defined the inner product of these 2 vectors as
      hj(t)=I(t)×Aj(t).


      The optimal day for influenza vaccination was the day j within a year such that the integral of hj(t) over all 10 years was the smallest, or
      j=argmin1j365t=13650hj(t).


      Vaccination Strategies

      Three vaccination strategies were assessed, which have been utilized by other national vaccine programs.
      • World Health Organization
      Vaccines against influenza WHO position paper—November 2012.
      • Tam Y.H.
      • Valkenburg S.A.
      • Perera R.A.P.M.
      • et al.
      Immune responses to twice-annual influenza vaccination in older adults in Hong Kong.
      • Mann A.G.
      • Mangtani P.
      • Russell C.A.
      • Whittaker J.C.
      The impact of targeting all elderly persons in England and Wales for yearly influenza vaccination: excess mortality due to pneumonia or influenza and time trend study.
      Using a no vaccination strategy as baseline, strategy 1 assumes p% of the elderly are vaccinated on the optimal day of each year, strategy 2 assumes p% of the elderly are vaccinated on both the optimal day and half a year after the optimal day of each year, and strategy 3 assumes all elderly and p% of the remaining population are vaccinated on the optimal day of each year. The values of p are taken from vectors 20, 40, 60, 80 and 100, creating a total of 15 scenarios.
      We calculated the ICER of vaccinating with each strategy versus no vaccination at a timeline of 10 years for 10 000 people. The ICER for strategy j takes the form,
      ICERj=CoststrategyjCostnovaccinationQALYstrategyjQALYnovaccination


      Assuming that incremental cost and changes of quality-adjusted life years (QALY) follow a Gaussian distribution, we calculated the confidence interval for the quotient of 2 means by Fieller’s method.
      • Zerbe G.O.
      On Fieller’s theorem and the general linear model.

      Sensitivity Analysis

      One-way sensitivity analysis was conducted to account for the uncertainty in the data owing to a lack of unambiguous reference values. By increasing (decreasing) mortality rate, mortality cost, hospital rate, hospital cost, outpatient rate, outpatient cost, and vaccination cost by 25% at each time, we plotted the percentage changes of ICER to obtain the tornado diagram.

      Results

      The simulated number of infected cases and the geometric mean titer for the baseline scenario with no vaccination under the 4 seasonality regimes are shown in Figures 1 and 2. The top 2 curves in Figures 1 and 2 provide simulated time series that are qualitatively similar to the ostensibly random timing of influenza epidemics in Singapore, without the need for other factors such as climate-based stochasticity. When there is an increasing amount of seasonal forcing, the influenza epidemic pattern becomes periodic and is consistent with patterns in temperate regions, where the number of cases peaks during, and the geometric mean titer drops before, the winter season. Figure 1 also shows the optimal time for vaccination in each year (see Figure S3 in the Supplementary Materials found at https://doi.org/10.1016/j.jval.2019.07.001). The simulation generates a sawtooth pattern in antibody levels, which determines how long after a first outbreak a second outbreak may occur. The time interval between 2 outbreaks is approximately 1 year under the Tokyo-like seasonality, with epidemics occurring in the model winters.
      Figure thumbnail gr1
      Figure 1Simulated number of cases per 10 000 people for Singapore and for scenarios derived from no influenza seasonality and from the seasonality of Taipei and Tokyo for a 10-year time horizon. Each simulation is varied from the age profile and contact patterns of Singapore for compatibility. One simulation plot is presented for each scenario.
      Figure thumbnail gr2
      Figure 2Simulated population level geometric mean titer (GMTs) for Singapore and for scenarios derived from no influenza seasonality and from the seasonality of Taipei and Tokyo for a 10-year time horizon. The simulations match those illustrated in .
      Figure 3 plots the incremental cost (in million USD) against the change of QALY for the 3 influenza vaccination strategies under different climate settings. The modeling includes both parametric uncertainty and stochasticity. The plots show that annual and biannual vaccination strategies for a percentage of the elderly are not always cost-effective because they cross the boundary of the cost-effective threshold, whereas annual vaccination for all elderly and a proportion of others is always cost-effective. This finding is consistent across different seasonality regimes.
      Figure thumbnail gr3
      Figure 3Plot of incremental costs (in million USD) against change in quality-adjusted life years (QALYs) for 1000 simulations under the 3 influenza vaccination strategies for Singapore and for scenarios matching no seasonality and the seasonality of Taipei and Tokyo. The modeling includes both parametric uncertainty and stochasticity. The red dots (0,0) represent the baseline scenario that none are vaccinated. Dots for AE represent annual vaccination, and dots for BE represent biannual vaccination of a proportion of the elderly. Dots for AEE show the outcome of annual vaccination of the elderly and a proportion of other age groups. The gray area shows the cost-effective zone using a willingness-to-pay of $52 961/QALY based on Singapore’s gross domestic product per capita.
      The average incremental cost, change in QALY, ICER, and their 95% confidence intervals among 1000 simulation runs under different strategies for the 4 seasonality-regimes are summarized in Table 2. Results show that with the fixed uptake rate, biannual vaccination for a percentage of the elderly yields higher ICERs than annual vaccination, making biannual vaccination for the elderly less cost-effective than annual vaccination for the elderly. For strategies vaccinating a proportion of the elderly annually or biannually, increasing the coverage of vaccination would increase the positive incremental cost and QALY gains together, making ICERs positive, but no clear trend is presented. However, for the strategy of vaccinating all elderly and a proportion of other age groups, increasing the uptake rate makes the incremental cost more negative, which suggests that from a societal perspective there can be greater savings by vaccinating more nonelderly people. Together with the positive QALY gains, the ICERs become negative. For example, vaccinating all elderly and 20% of the rest of the population is modeled to lead to a cost savings of $49 000 per QALY gained for Singapore. Thus, vaccinating all elderly and other age groups is consistently cost saving, making this the most cost-effective strategy of the 3 immunization strategies. With Singapore willingness-to-pay of $52 961/QALY, the annual vaccination strategies for a proportion of the elderly and for all the elderly plus some other age groups are both cost-effective, and the biannual vaccination strategy for the elderly is not cost-effective.
      Table 2Summary of mean incremental cost (in millions of USD), QALY gains, and ICER (in thousands of USD per QALY) per 100 000 persons per year together with the respective 95% confidence intervals (in parentheses) for the 1000 simulation runs under different vaccination strategies for city without seasonality, for Singapore, and for Taipei and Tokyo’s climate.
      City without seasonality
      StrategyCoverage (%)Incremental cost (million USD)QALY gainsICER ($000/QALY)
      Annual, elderly only200.03 (0.02-0.05)0.66 (0.40-0.91)51.90 (29.52-93.61)
      400.07 (0.05-0.08)1.37 (1.11-1.62)48.11 (36.63-63.08)
      600.10 (0.09-0.11)1.99 (1.73-2.24)51.14 (42.68-61.34)
      800.14 (0.13-0.15)2.51 (2.25-2.78)56.28 (48.92-64.93)
      1000.18 (0.17-0.19)3.06 (2.80-3.32)59.24 (53.0766.27)
      Biannual, elderly only200.07 (0.05-0.08)0.98 (0.72-1.24)68.96 (50.29-98.11)
      400.16 (0.15-0.17)1.44 (1.18-1.70)111.02 (92.18-137.32)
      600.24 (0.23-0.26)2.09 (1.82-2.36)116.35 (102.24-134.13)
      800.32 (0.30-0.33)2.96 (2.70-3.22)106.56 (97.01-117.75)
      1000.39 (0.38-0.41)3.70 (3.44-3.97)106.17 (98.44-114.95)
      Annual, all elderly and a proportion of other age groups20-1.80 (-1.81 to -1.79)38.28 (38.05-38.51)Cost saving
      40-2.81 (-2.82 to -2.80)59.83 (59.63-60.03)Cost saving
      60-3.25 (-3.26 to -3.24)69.39 (69.20-69.57)Cost saving
      80-3.47 (-3.48 to -3.46)74.89 (74.70-75.07)Cost saving
      100-3.58 (-3.59 to -3.57)78.28 (78.11-78.46)Cost saving
      Singapore
      StrategyCoverage (%)Incremental cost (million US dollars)QALY gainsICER ($000/QALY)
      Annual, elderly only200.04 (0.02-0.05)0.66 (0.39-0.92)53.90 (30.81-97.06)
      400.06 (0.05-0.07)1.55 (1.28-1.82)38.20 (28.61-50.00)
      600.10 (0.08-0.11)2.17 (1.91-2.43)44.18 (36.78-52.86)
      800.15 (0.13-0.16)2.52 (2.25-2.79)57.97 (50.65-66.55)
      1000.18 (0.16-0.19)3.28 (3.02-3.54)53.67 (48.19-59.83)
      Biannual, elderly only200.08 (0.06-0.09)0.82 (0.56-1.08)92.48 (66.40-139.09)
      400.17 (0.15-0.18)1.38 (1.13-1.64)119.18 (98.49-148.59)
      600.25 (0.24-0.27)1.94 (1.68-2.20)130.72 (114.09-152.12)
      800.33 (0.31-0.34)2.85 (2.59-3.11)114.28 (103.80-126.69)
      1000.41 (0.40-0.42)3.48 (3.23-3.74)117.78 (108.86-128.03)
      Annual, all elderly and a proportion of other age groups20-1.89 (-1.90 to -1.88)40.14 (39.90-40.38)Cost saving
      40-3.03 (-3.04 to -3.02)64.35 (64.14-64.56)Cost saving
      60-3.53 (-3.54 to -3.52)75.05 (74.85-75.25)Cost saving
      80-3.79 (-3.79 to -3.78)81.18 (80.99-81.36)Cost saving
      100-3.91 (-3.92 to -3.90)84.90 (84.71-85.08)Cost saving
      Taipei seasonality
      StrategyCoverage (%)Incremental cost (million USD)QALY gainsICER ($000/QALY)
      Annual, elderly only200.03 (0.01-0.05)0.73 (0.40-1.05)40.40 (17.14-83.92)
      400.07 (0.06-0.09)1.16 (0.84-1.48)63.26 (44.75-92.48)
      600.11 (0.09-0.12)1.80 (1.47-2.13)59.88 (47.34-76.61)
      800.15 (0.13-0.16)2.30 (1.98-2.62)64.15 (53.82-77.09)
      1000.18 (0.17-0.20)2.91 (2.58-3.24)62.86 (54.63-72.68)
      Biannual, elderly only200.08 (0.06-0.09)0.80 (0.47-1.13)95.52 (63.04-168.12)
      400.17 (0.15-0.18)1.29 (0.97-1.62)130.16 (101.61-176.51)
      600.25 (0.23-0.26)2.06 (1.73-2.39)119.04 (100.98-143.42)
      800.34 (0.32-0.35)2.52 (2.20-2.84)133.70 (117.55-154.21)
      1000.42 (0.41-0.44)3.12 (2.80-3.44)135.77 (122.34-152.51)
      Annual, all elderly and a proportion of other age groups20-1.98 (-1.99 to -1.96)41.88 (41.56-42.20)Cost saving
      40-3.12 (-3.13 to -3.10)65.90 (65.6-66.19)Cost saving
      60-3.62 (-3.64 to -3.61)76.69 (76.41-76.97)Cost saving
      80-3.81 (-3.83 to -3.8)81.64 (81.36-81.92)Cost saving
      100-3.89 (-3.9 to -3.87)84.29 (84.02-84.57)Cost saving
      Tokyo seasonality
      StrategyCoverage (%)Incremental cost (million US dollars)QALY gainsICER ($000/QALY)
      Annual, elderly only200.02 (0.00-0.04)0.94 (0.60-1.28)21.67 (3.95-45.88)
      400.07 (0.05-0.09)1.29 (0.95-1.62)53.75 (37.60-77.82
      600.10 (0.08-0.11)2.05 (1.70-2.39)47.42 (37.07-60.55)
      800.13 (0.12-0.15)2.63 (2.30-2.97)20.98 (42.73-60.90)
      1000.17 (0.15-0.19)3.25 (2.91-3.58)52.38 (45.47-60.41)
      Biannual, elderly only200.09 (0.07-0.10)0.65 (0.31-1.00)130.98 (81.51-227.84)
      400.16 (0.15-0.18)1.47 (1.12-1.81)112.29 (88.45-149.47)
      600.25 (0.24-0.27)2.04 (1.69-2.39)124.18 (104.47-151.36)
      800.34 (0.32-0.35)2.76 (2.42-3.09)121.56 (107.17-139.62)
      1000.40 (0.39-0.42)3.73 (3.39-4.08)108.29 (98.19-120.25)
      Annual, all elderly and a proportion of other age groups20-2.12 (-2.14 to -2.10)45.04 (44.72-45.35)Cost saving
      40-3.55 (-3.57 to -3.54)74.92 (74.63-75.21)Cost saving
      60-4.28 (-4.30 to -4.27)89.95 (89.69-90.21)Cost saving
      80-4.57 (-4.58 to -4.55)96.57 (96.30-96.85)Cost saving
      100-4.66 (-4.67 to -4.64)99.61 (99.35-99.86)Cost saving
      ICER indicates incremental cost-effectiveness ratio; QALY, quality-adjusted life year.
      The tornado diagram (Figure 4) shows that vaccination cost and vaccine efficacy have an important effect on cost-effectiveness, whereas mortality costs, hospitalization rate, and hospitalization cost have the least effect on ICERs.
      Figure thumbnail gr4
      Figure 4One-way sensitivity analyses. The tornado diagram shows results of the 1-way sensitivity analysis for the percentage change in incremental cost-effectiveness ratio (ICER) when input variables vary for Singapore. The red bar shows the percentage change of ICER when there is a 25% decrease from the original value of each parameter. The blue bar shows the percentage change in ICER when there is a 25% increase from the original value of each parameter. The orange bar shows the percentage change in ICER when there is a 10% increase from the original value of each parameter.

      Discussion

      This study assessed the cost-effectiveness of 3 broad influenza vaccination strategies for Singapore using a validated model of influenza antibody trajectories to simulate the timing of infections. Overall, vaccinating all elderly individuals and a proportion of other age groups results in cost-saving outcomes. High-risk groups such as the elderly, who have higher death and hospitalization rates, can have their quality of life substantially improved by receiving the influenza vaccine, resulting in overall benefits compared with no vaccination. However, relatively low-risk groups such as young adults also receive the benefits of lower influenza incidence because they have higher contact pattern rates.
      • Prem K.
      • Cook A.R.
      • Jit M.
      Projecting social contact matrices in 152 countries using contact surveys and demographic data.
      Policy recommendations that focus on increasing influenza vaccine uptake across the entire population are therefore beneficial. With 14.8 per 100 000 deaths being attributed to influenza a year, the potential QALY gains with widespread vaccination of 780 per 100 000 person years are sizable. The projections in this study show that even with a limited duration and partial degree of protection, influenza vaccination strategies targeting the elderly and other lower-risk groups could lead to cost savings of $36 million per 100 000 population over a 10-year time horizon.
      Despite the lack of strong seasonality—which in temperate countries has the dual effects of concentrating risk in a short period of time and acting as a cue to get or recommend vaccination—equatorial Singapore may still benefit from once-yearly vaccination programs. The amount of benefit would be similar to simulations in which the age and social structures were retained but more clearly defined seasons were present. However, because we focused on Singapore, the cost-effectiveness in lower-income equatorial settings should be assessed.
      Antibody levels peak 1 to 2 months after vaccination,
      • Young B.
      • Zhao X.
      • Cook A.R.
      • Parry C.M.
      • Wilder-Smith A.
      • I-Cheng M.C.
      Do antibody responses to the influenza vaccine persist year-round in the elderly? A systematic review and meta-analysis.
      • Zhao X.
      • Ning Y.
      • Chen M.I.
      • Cook A.R.
      Individual and population trajectories of influenza antibody titers over multiple seasons in a tropical country.
      with a half-life of around 6 months, which is enough to bracket a winter epidemic in a temperate setting but not to provide yearlong protection in settings like Singapore or Indonesia where year-round transmission occurs. We therefore anticipated that twice-yearly vaccination might be useful in the latter scenario. This modeling study suggests, however, that the benefits of twice-yearly vaccination are not sufficiently greater than annual vaccination to recommend this policy. Empirical studies to validate this finding would be ideal.
      Although this study supports the vaccination of the elderly in Singapore, the relative lack of seasonality in Singapore may lead to challenges in reaching the uptake rates considered in our modeling scenarios. Improving uptake requires a people-centric approach
      • Rashid H.
      • Yin J.K.
      • Ward K.
      • King C.
      • Seale H.
      • Booy R.
      Assessing interventions to improve influenza vaccine uptake among health care workers.
      and may include education and training sessions,
      National Vaccine Advisory Committee
      Strategies to achieve the Healthy People 2020 annual influenza vaccine coverage goal for health-care personnel: recommendations from the National Vaccine Advisory Committee.
      incentivization through rewards after vaccination,
      • Dale L.P.
      • White L.
      • Mitchell M.
      • Faulkner G.
      Smartphone app uses loyalty point incentives and push notifications to encourage influenza vaccine uptake.
      increased advertising and vaccine awareness,
      • Sheedy K.
      Communication strategies for increasing national seasonal influenza vaccine usage.
      reminders for patients
      • Herrett E.
      • Williamson E.
      • van Staa T.
      • et al.
      Text messaging reminders for influenza vaccine in primary care: a cluster randomised controlled trial (TXT4FLUJAB).
      and improved access to vaccination by running mobile clinics.
      • Nutman A.
      • Yoeli N.
      Influenza vaccination motivators among healthcare personnel in a large acute care hospital in Israel.
      Although mass vaccination annually was modeled to create the greatest cost savings, there remain questions of who should pay for vaccination. Singapore’s healthcare system is founded on a principle of individual responsibility supported by the state. At present, high-risk groups such as the elderly can pay for recommended vaccination through their or their families’ personal medical savings accounts at public clinics; others pay out of pocket. This financial framework may act as a disincentive to be vaccinated because of discounting of the future,
      • Jit M.
      • Mibei W.
      Discounting in the evaluation of the cost-effectiveness of a vaccination programme: a critical review.
      compounded by misperceptions of the risks of influenza,
      • Yuen C.Y.S.
      • Dodgson J.E.
      • Tarrant M.
      Perceptions of Hong Kong Chinese women toward influenza vaccination during pregnancy.
      and a lack of understanding about influenza and other recommended vaccinations.
      • Erbelding E.J.
      • Post D.J.
      • Stemmy E.J.
      • et al.
      A universal influenza vaccine: the strategic plan for the National Institute of Allergy and Infectious Diseases.
      In some other settings, influenza vaccination is provided free,
      • Bambery B.
      • Douglas T.
      • Selgelid M.J.
      • et al.
      Influenza vaccination strategies should target children.
      and the positive externalities that result from vaccination—namely, the reduced risk of transmitting infection to others—may justify a reconsideration of Singapore’s current payment system for influenza vaccination, either by providing vaccination free to high-risk groups or incentivizing it through subsidies.
      Limitations of our study include assumptions made for the vaccine, population, and costs for the program. Vaccine efficacy was fixed within 3 broad age groups, but immunosenescence and comorbidities may have an impact on vaccine success among the elderly.
      • Ting E.E.K.
      • Sander B.
      • Ungar W.J.
      Systematic review of the cost-effectiveness of influenza immunization programs.
      • Lang P.-O.
      • Mendes A.
      • Socquet J.
      • Assir N.
      • Govind S.
      • Aspinall R.
      Effectiveness of influenza vaccine in aging and older adults: comprehensive analysis of the evidence.
      Differences in vaccine efficacy according to strain and issues of vaccine mismatch may also affect the impact of a vaccination policy.
      • Tricco A.C.
      • Chit A.
      • Soobiah C.
      • et al.
      Comparing influenza vaccine efficacy against mismatched and matched strains: a systematic review and meta-analysis.
      Potential side effects from vaccination, which incur additional costs, were also not included within the analysis. The study also assumed that all costs relating to hospitalization and illness remained constant, but that may change as new treatments become available.
      The population size and structure were kept constant in the model, but immigration and an aging population could change the contact and consequent infection rates. Owing to computational restrictions, a population sample of 10 000 was used to represent the Singaporean population of approximately 6 million people to allow a greater number of Monte Carlo samples. Although this captures the macro trends, representing the entire population would be more ideal.
      Furthermore, factors among the elderly affecting their intention to vaccinate or ability to be vaccinated were not accounted for. These can include physical determinants that may exclude individuals from vaccination or contextual determinants such as vaccine access or sociodemographic factors. Perceptions of influenza risk and the perceived benefit of vaccination, which differ based on knowledge, attitude, and subjective norms, will also affect willingness to pay among the elderly and are largely unknown. Vaccine confidence and its impact on public uptake are ongoing issues
      • Low M.
      • Tan H.Y.
      • Hartman M.
      • Tam C.
      Knowledge, attitudes and practices of parents towards childhood influenza vaccination in Singapore.
      and could be major impediments to maintaining willingness to pay at a sustainable level where national policies and governmental subsidization are feasible.
      Vaccination of other high-risk groups should also be assessed because this has been shown to be cost-effective for those with specific chronic conditions increasing their risk of complications,
      • Schanzer D.L.
      • Langley J.M.
      • Tam T.W.S.
      Co-morbidities associated with influenza-attributed mortality, 1994-2000, Canada.
      pregnant women,
      • Mertz D.
      • Geraci J.
      • Winkup J.
      • Gessner B.D.
      • Ortiz J.R.
      • Loeb M.
      Pregnancy as a risk factor for severe outcomes from influenza virus infection: a systematic review and meta-analysis of observational studies.
      and the obese.
      • Jain S.
      • Chaves S.S.
      Obesity and influenza.
      The study considers a composite influenza strain and does not take competing virus types into account. The vaccine efficacy we adopted is approximately that of a trivalent influenza vaccine. When vaccine efficacy is increased by 10%, as might be attainable with a quadrivalent influenza vaccine, the ICER would decrease by about 40%. Thus, if a quadrivalent vaccine were available at the same price as the trivalent vaccine considered in this study, vaccination would be even more strongly supported. Future work might investigate the importance of incorporating multiple influenza strains.
      The results of this study highlight the importance of herd immunity with increasing vaccination rates,
      • Cockman P.
      • Dawson L.
      • Mathur R.
      • Hull S.
      Improving MMR vaccination rates: herd immunity is a realistic goal.
      affording the Singaporean government credence to give some priority to tackling low vaccine uptake rates.

      Acknowledgments

      This study was supported by the Health Services Research Grant NMRC/HSRG/0078/2017. A.R.C. and B.L.D. were also supported by funding from the Singapore Ministry of Health’s National Medical Research Council under the Centre Grant Programme: Singapore Population Health Improvement Centre (NMRC/CG/C026/2017_NUHS).

      Supplementary Material

      References

        • Iuliano A.D.
        • Roguski K.M.
        • Chang H.H.
        • et al.
        Estimates of global seasonal influenza-associated respiratory mortality: a modelling study.
        Lancet. 2018; 391: 1285-1300
        • Peasah S.K.
        • Azziz-Baumgartner E.
        • Breese J.
        • Meltzer M.I.
        • Widdowson M.-A.
        Influenza cost and cost-effectiveness studies globally—a review.
        Vaccine. 2013; 31: 5339-5348
        • Hampson A.W.
        Vaccines for pandemic influenza. The history of our current vaccines, their limitations and the requirements to deal with a pandemic threat.
        Ann Acad Med Singapore. 2008; 37: 510-517
        • Shields G.E.
        • Elvidge J.
        • Davies L.M.
        A systematic review of economic evaluations of seasonal influenza vaccination for the elderly population in the European Union.
        BMJ Open. 2017; 7: e014847
        • You J.H.S.
        • Ming W.-K.
        • Chan P.K.S.
        Cost-effectiveness of quadrivalent influenza vaccine in Hong Kong—a decision analysis.
        Hum Vaccin Immunother. 2015; 11: 564-571
        • Thommes E.W.
        • Kruse M.
        • Kohli M.
        • Sharma R.
        • Noorduyn S.G.
        Review of seasonal influenza in Canada: burden of disease and the cost-effectiveness of quadrivalent inactivated influenza vaccines.
        Hum Vaccin Immunother. 2017; 13: 867-876
        • Wang C.-S.
        • Wang S.-T.
        • Chou P.
        Efficacy and cost-effectiveness of influenza vaccination of the elderly in a densely populated and unvaccinated community.
        Vaccine. 2002; 20: 2494-2499
        • Lambach P.
        • Alvarez A.M.R.
        • Hirve S.
        • et al.
        Considerations of strategies to provide influenza vaccine year round.
        Vaccine. 2015; 33: 6493-6498
        • Root E.D.
        • Agustian D.
        • Kartasasmita C.
        • Uyeki T.M.
        • Simões E.A.
        Demographic and ecological risk factors for human influenza A virus infections in rural Indonesia.
        Influenza Other Respir Viruses. 2017; 11: 425-433
        • Osterholm M.T.
        • Kelley N.S.
        • Sommer A.
        • Belongia E.A.
        Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis.
        Lancet Infect Dis. 2012; 12: 36-44
        • Young B.
        • Zhao X.
        • Cook A.R.
        • Parry C.M.
        • Wilder-Smith A.
        • I-Cheng M.C.
        Do antibody responses to the influenza vaccine persist year-round in the elderly? A systematic review and meta-analysis.
        Vaccine. 2017; 35: 212-221
      1. Lai L. Singapore has two peak flu periods a year. The Straits Times. July 26, 2017. https://www.straitstimes.com/singapore/health/singapore-has-two-peak-flu-periods-a-year. Accessed August 28, 2018.

        • Chen Y.
        • Cook A.R.
        • Lim A.X.L.
        Randomness of dengue outbreaks on the equator.
        Emerg Infect Dis. 2015; 21: 1651-1653
        • Viboud C.
        • Alonso W.J.
        • Simonsen L.
        Influenza in tropical regions.
        PLoS Med. 2006; 3: e89
        • Lee V.J.
        • Yap J.
        • Ong J.B.S.
        • et al.
        Influenza excess mortality from 1950–2000 in tropical Singapore.
        PLoS One. 2009; 4: e8096
        • Ang L.W.
        • Lim C.
        • Lee V.J.M.
        • et al.
        Influenza-associated hospitalizations, Singapore, 2004-2008 and 2010-2012.
        Emerg Infect Dis. 2014; 20: 1652-1660
        • Influenza
        Ministry of Health Singapore.
        • Phan T.P.
        • Alkema L.
        • Tai E.S.
        • et al.
        Forecasting the burden of type 2 diabetes in Singapore using a demographic epidemiological model of Singapore.
        BMJ Open Diabetes Res Care. 2014; 2: e000012
        • Press release on post-NDR announcements. Ministry of Health Singapore
        • Tan Y.L.
        Overseas travel by Singapore residents.
        Department of Statistics Singapore, 2002
        • R Core Team
        R: a language and environment for statistical computing.
        R Foundation for Statistical Computing, Vienna, Austria2014
        • Zhao X.
        • Ning Y.
        • Chen M.I.
        • Cook A.R.
        Individual and population trajectories of influenza antibody titers over multiple seasons in a tropical country.
        Am J Epidemiol. 2018; 187: 135-143
        • Prem K.
        • Cook A.R.
        • Jit M.
        Projecting social contact matrices in 152 countries using contact surveys and demographic data.
        PLoS Comput Biol. 2017; 13: e1005697
        • Zhao X.
        • Fang V.J.
        • Ohmit S.E.
        • Monto A.S.
        • Cook A.R.
        • Cowling B.J.
        Quantifying protection against influenza virus infection measured by hemagglutination-inhibition assays in vaccine trials.
        Epidemiology. 2016; 27: 143-151
        • World Health Organization
        Vaccines against influenza WHO position paper—November 2012.
        Wkly Epidemiol Rec. 2012; 87: 461-476
        • Tam Y.H.
        • Valkenburg S.A.
        • Perera R.A.P.M.
        • et al.
        Immune responses to twice-annual influenza vaccination in older adults in Hong Kong.
        Clin Infect Dis. 2018; 66: 904-912
        • Mann A.G.
        • Mangtani P.
        • Russell C.A.
        • Whittaker J.C.
        The impact of targeting all elderly persons in England and Wales for yearly influenza vaccination: excess mortality due to pneumonia or influenza and time trend study.
        BMJ Open. 2013; 3: e002743
        • Zerbe G.O.
        On Fieller’s theorem and the general linear model.
        Am Stat. 1978; 32: 103-105
      2. Influenza pandemic readiness and response plan. Ministry of Health Singapore. https://www.moh.gov.sg/content/dam/moh_web/PressRoom/Current_Issues/2009/Main%20Document%20Public%20_Jan09.pdf.

        • Rashid H.
        • Yin J.K.
        • Ward K.
        • King C.
        • Seale H.
        • Booy R.
        Assessing interventions to improve influenza vaccine uptake among health care workers.
        Health Aff (Millwood). 2016; 35: 284-292
        • National Vaccine Advisory Committee
        Strategies to achieve the Healthy People 2020 annual influenza vaccine coverage goal for health-care personnel: recommendations from the National Vaccine Advisory Committee.
        Public Health Rep. 2013; 128: 7-25
        • Dale L.P.
        • White L.
        • Mitchell M.
        • Faulkner G.
        Smartphone app uses loyalty point incentives and push notifications to encourage influenza vaccine uptake.
        Vaccine. 2019; 37: 4594-4600
        • Sheedy K.
        Communication strategies for increasing national seasonal influenza vaccine usage.
        WHO GAP-II Meeting, 2011 (Accessed August 28, 2018)
        • Herrett E.
        • Williamson E.
        • van Staa T.
        • et al.
        Text messaging reminders for influenza vaccine in primary care: a cluster randomised controlled trial (TXT4FLUJAB).
        BMJ Open. 2016; 6: e010069
        • Nutman A.
        • Yoeli N.
        Influenza vaccination motivators among healthcare personnel in a large acute care hospital in Israel.
        Isr J Health Policy Res. 2016; 5: 52
        • Jit M.
        • Mibei W.
        Discounting in the evaluation of the cost-effectiveness of a vaccination programme: a critical review.
        Vaccine. 2015; 33: 3788-3794
        • Yuen C.Y.S.
        • Dodgson J.E.
        • Tarrant M.
        Perceptions of Hong Kong Chinese women toward influenza vaccination during pregnancy.
        Vaccine. 2016; 34: 33-40
        • Erbelding E.J.
        • Post D.J.
        • Stemmy E.J.
        • et al.
        A universal influenza vaccine: the strategic plan for the National Institute of Allergy and Infectious Diseases.
        J Infect Dis. 2018; 218: 347-354
        • Bambery B.
        • Douglas T.
        • Selgelid M.J.
        • et al.
        Influenza vaccination strategies should target children.
        Public Health Ethics. 2018; 11: 221-234
        • Ting E.E.K.
        • Sander B.
        • Ungar W.J.
        Systematic review of the cost-effectiveness of influenza immunization programs.
        Vaccine. 2017; 35: 1828-1843
        • Lang P.-O.
        • Mendes A.
        • Socquet J.
        • Assir N.
        • Govind S.
        • Aspinall R.
        Effectiveness of influenza vaccine in aging and older adults: comprehensive analysis of the evidence.
        Clin Interv Aging. 2012; 7: 55-64
        • Tricco A.C.
        • Chit A.
        • Soobiah C.
        • et al.
        Comparing influenza vaccine efficacy against mismatched and matched strains: a systematic review and meta-analysis.
        BMC Med. 2013; 11: 153
        • Low M.
        • Tan H.Y.
        • Hartman M.
        • Tam C.
        Knowledge, attitudes and practices of parents towards childhood influenza vaccination in Singapore.
        Int J Infect Dis. 2016; 53: 105-106
        • Schanzer D.L.
        • Langley J.M.
        • Tam T.W.S.
        Co-morbidities associated with influenza-attributed mortality, 1994-2000, Canada.
        Vaccine. 2008; 26: 4697-4703
        • Mertz D.
        • Geraci J.
        • Winkup J.
        • Gessner B.D.
        • Ortiz J.R.
        • Loeb M.
        Pregnancy as a risk factor for severe outcomes from influenza virus infection: a systematic review and meta-analysis of observational studies.
        Vaccine. 2017; 35: 521-528
        • Jain S.
        • Chaves S.S.
        Obesity and influenza.
        Clin Infect Dis. 2011; 53: 422-424
        • Cockman P.
        • Dawson L.
        • Mathur R.
        • Hull S.
        Improving MMR vaccination rates: herd immunity is a realistic goal.
        BMJ. 2011; 343: d5703