Advertisement

What We Do Not Know About the Costs of Immunization Programs in Low- and Middle-Income Countries

Open AccessPublished:November 02, 2020DOI:https://doi.org/10.1016/j.jval.2020.08.2097

      Highlights

      • For many countries, there are limited data on the costs of running immunization services, and even less on the costs of increasing immunization coverage.
      • When considering different approaches for scaling up coverage, countries and funders need to understand the marginal change in coverage produced, costs of introduction, and how cost and coverage effects change depending on programmatic context.
      • Costing studies would benefit from improved, systematic reporting and leveraging ongoing program evaluation efforts to collect costing data. Long-term investments in the health system may allow for routine data collection and improved efficiency for budgeting and planning.

      Introduction

      Evidence on immunization economics is a critical input for country immunization programs, particularly in low- and middle-income countries (LMICs) with government- and donor-funded vaccination programs. This evidence allows programs to budget for current services, plan for new vaccine introduction, and evaluate the efficiency of service delivery strategies. Evidence on immunization costs is also useful for international funders when making resource allocation decisions.
      • Brenzel L.
      What have we learned on costs and financing of routine immunization from the comprehensive multi-year plans in GAVI eligible countries?.
      • Hargreaves J.R.
      • Greenwood B.
      • Clift C.
      • et al.
      Making new vaccines affordable: a comparison of financing processes used to develop and deploy new meningococcal and pneumococcal conjugate vaccines.
      • Lydon P.
      • Levine R.
      • Makinen M.
      • et al.
      Introducing new vaccines in the poorest countries: what did we learn from the GAVI experience with financial sustainability?.
      • Vassall A.
      • Sweeney S.
      • Kahn J.
      • et al.
      Reference case for estimating the costs of global health services and interventions. Global Health Cost Consortium.
      A large number of studies have reported on the effectiveness and cost-effectiveness of childhood immunization for reducing the impact of vaccine-preventable diseases in LMICs.
      • Brenzel L.
      • Halsey N.
      • Miller M.
      • Wolfson L.
      • Fox-Rushby J.
      Vaccine preventable diseases.
      • Ozawa S.
      • Clark S.
      • Portnoy A.
      • Grewal S.
      • Brenzel L.
      • Walker D.G.
      Return on investment from childhood immunization in low- and middle-income countries, 2011-20.
      Johns Hopkins University–International Vaccine Access Center (IVAC)
      Methodology Report: Decade of Vaccines Economics (DOVE) Return on Investment Analysis.
      Nevertheless, these analyses often frame vaccine implementation strategies in general terms and make assumptions regarding costs and operational success. Bias or uncertainty in cost estimates can significantly influence the results of cost-effectiveness and budget impact analyses,
      • Frappier J.
      • Tremblay G.
      • Charny M.
      • Cloutier L.M.
      Costing bias in economic evaluations.
      and there is a substantial gap in the empirical evidence available on immunization economics. We discuss the nature of this evidence gap, and propose solutions for closing it.

      Closing the Gap: The Costs of Immunization Services

      For most countries, available estimates of the costs of providing immunization services are uncertain, unreliable, old, or missing altogether. For example, the Immunization Costing Action Network’s recently developed Immunization Delivery Cost Catalogue identifies 61 unique publications reporting immunization unit costs for both routine services and supplementary immunization activities in the peer-reviewed and gray literature, representing only 33 different countries, and with only a small fraction of these data collected in the last 5 years (Figure 1).
      Immunization Costing Action Network (ICAN)
      2019. Immunization Delivery Cost Catalogue (IDCC). Washington: ThinkWell.
      ,
      • Vaughan K.
      • Ozaltin A.
      • Mallow M.
      • et al.
      The costs of delivering vaccines in low- and middle-income countries: findings from a systematic review.
      Although recent research investments have improved the availability of immunization costing data,
      Immunization Costing Action Network (ICAN)
      2019. Immunization Delivery Cost Catalogue (IDCC). Washington: ThinkWell.
      • Vaughan K.
      • Ozaltin A.
      • Mallow M.
      • et al.
      The costs of delivering vaccines in low- and middle-income countries: findings from a systematic review.
      • Brenzel L.
      • Young D.
      • Walker D.G.
      Costs and financing of routine immunization: approach and selected findings of a multi-country study (EPIC).
      • Menzies N.A.
      • Suharlim C.
      • Geng F.
      • Ward Z.J.
      • Brenzel L.
      • Resch S.C.
      The cost determinants of routine infant immunization services: a meta-regression analysis of six country studies.
      these efforts would still need to be greatly expanded to supply all countries with up-to-date and high-quality cost estimates.
      • Portnoy A.
      • Vaughan K.
      • Clarke-Deelder E.
      • et al.
      Producing standardized country-level immunization delivery unit cost estimates.
      Because there is unlikely to be sufficient resources to conduct studies for all questions and settings of interest, researchers and decision makers need to weigh the resources required against both the magnitude of the evidence gap and the magnitude of the decisions being made. In practice, when immunization unit costs are needed for a new analysis, the necessary values are typically borrowed from neighboring countries, regional estimates, or global estimations to complete the analysis.
      Immunization Costing Action Network (ICAN)
      2019. Immunization Delivery Cost Catalogue (IDCC). Washington: ThinkWell.
      ,
      • Vaughan K.
      • Ozaltin A.
      • Mallow M.
      • et al.
      The costs of delivering vaccines in low- and middle-income countries: findings from a systematic review.
      Although this approach can be workable when there is a good understanding of the mechanisms that drive variation in cost estimates across settings,
      • Vassall A.
      • Sweeney S.
      • Kahn J.
      • et al.
      Reference case for estimating the costs of global health services and interventions. Global Health Cost Consortium.
      it can involve difficult subjective judgments when important contextual information is not reported, and strong assumptions about whether the approach used to impute the missing values is valid, often without accompanying sensitivity analysis to reflect the uncertainty in those assumptions. Although the low volume of costing research is a primary challenge, this gap has begun to close. Nevertheless, additional hurdles are introduced by the way studies are reported. Reporting of studies typically focuses on the specific research question, with less focus on the detailed reporting that would allow broader use of results. This issue of incomplete reporting in costing studies is commonly cited in the public health literature, and long recognized as affecting comparability of results across studies.
      • Vassall A.
      • Sweeney S.
      • Kahn J.
      • et al.
      Reference case for estimating the costs of global health services and interventions. Global Health Cost Consortium.
      ,
      • Frappier J.
      • Tremblay G.
      • Charny M.
      • Cloutier L.M.
      Costing bias in economic evaluations.
      ,
      • Graves N.
      • Walker D.
      • Raine R.
      • Hutchings A.
      • Roberts J.A.
      Cost data for individual patients included in clinical studies: no amount of statistical analysis can compensate for inadequate costing methods.
      • Clement Nee Shrive F.M.
      • Ghali W.A.
      • Donaldson C.
      • Manns B.J.
      The impact of using different costing methods on the results of an economic evaluation of cardiac care: microcosting vs gross-costing approaches.
      • Adam T.
      • Koopmanschap M.A.
      • Evans D.B.
      Cost-effectiveness analysis: can we reduce variability in costing methods?.
      Although some studies have developed and reported their results using systematic costing guidance,
      • Brenzel L.
      • Young D.
      • Walker D.G.
      Costs and financing of routine immunization: approach and selected findings of a multi-country study (EPIC).
      future studies across the public health sector would benefit from standardized reporting to get the most out of data collection. Although recent reference cases provide concrete guidance for implementing and reporting costing studies,
      • Vassall A.
      • Sweeney S.
      • Kahn J.
      • et al.
      Reference case for estimating the costs of global health services and interventions. Global Health Cost Consortium.
      ,
      EPIC Project (Harvard)
      Working Paper: how to cost immunization programs: a practical guide on primary data collection and analysis.
      it is unclear whether reporting practices have widely changed.

      The Remaining Gap: The Costs of Increasing Immunization Coverage

      Although there are limited data on the costs of running immunization services,
      Immunization Costing Action Network (ICAN)
      2019. Immunization Delivery Cost Catalogue (IDCC). Washington: ThinkWell.
      • Vaughan K.
      • Ozaltin A.
      • Mallow M.
      • et al.
      The costs of delivering vaccines in low- and middle-income countries: findings from a systematic review.
      • Brenzel L.
      • Young D.
      • Walker D.G.
      Costs and financing of routine immunization: approach and selected findings of a multi-country study (EPIC).
      even less are available on the economics of increasing immunization coverage. As needed efforts to scale up immunization coverage for global disease eradication, elimination, and control remain ongoing,
      • Brenzel L.
      • Halsey N.
      • Miller M.
      • Wolfson L.
      • Fox-Rushby J.
      Vaccine preventable diseases.
      ,
      World Health Organization
      Global vaccine action plan 2011-2020.
      there is an accompanying need to collect and analyze the costs of scaling up to ensure efficient resource allocation and program management to improve coverage.
      • Bishai D.
      • McQuestion M.
      • Chaudhry R.
      • Wigton A.
      The costs of scaling up vaccination in the world’s poorest countries.
      Two systematic reviews of the incremental cost of scaling up immunization coverage in LMICs published in 2004 found that there was very limited evidence on the cost-effectiveness of approaches for improving immunization coverage.
      • Batt K.
      • Fox-Rushby J.A.
      • Castillo-Riquelme M.
      The costs, effects and cost-effectiveness of strategies to increase coverage of routine immunizations in low- and middle-income countries: systematic review of the grey literature.
      ,
      • Pegurri E.
      • Fox-Rushby J.A.
      • Walker D.G.
      The effects and costs of expanding the coverage of immunisation services in developing countries: a systematic literature review.
      Little has changed in the 16 years since these reviews, with a recent systematic review finding 13 publications on the incremental costs of scaling up immunization coverage over 2003 to 2018 and coming to the same conclusion: the evidence was too scarce to aggregate in a meaningful way.
      • Munk C.
      • Portnoy A.
      • Suharlim C.
      • et al.
      Systematic review of the costs and effectiveness of interventions to increase infant vaccination coverage in low- and middle-income countries.
      A separate meta-analysis of these studies was able to identify a statistical relationship between current coverage level and the marginal costs of further coverage improvements, but only when data from high-income settings was included, and without being able to distinguish the different approaches used to achieve coverage improvements.
      • Ozawa S.
      • Yemeke T.T.
      • Thompson K.M.
      Systematic review of the incremental costs of interventions that increase immunization coverage.
      For the continued support of immunization programs in LMICs, it is important to identify approaches that can be taken to improve immunization coverage. In addition to an understanding of what these approaches are, LMICs and funders need to understand the marginal change in coverage they produce, the costs of introducing them, and how these cost and coverage effects change depending on the programmatic context. There are already many ongoing efforts to improve immunization coverage that are being evaluated for programmatic effectiveness,
      • Johri M.
      • Perez M.C.
      • Arsenault C.
      • et al.
      Strategies to increase the demand for childhood vaccination in low- and middle-income countries: a systematic review and meta-analysis.
      • LaFond A.
      • Kanagat N.
      • Steinglass R.
      • Fields R.
      • Sequeira J.
      • Mookherji S.
      Drivers of routine immunization coverage improvement in Africa: findings from district-level case studies.
      • Nelson K.N.
      • Wallace A.S.
      • Sodha S.V.
      • Daniels D.
      • Dietz V.
      Assessing strategies for increasing urban routine immunization coverage of childhood vaccines in low and middle-income countries: a systematic review of peer-reviewed literature.
      • Oyo-Ita A.
      • Wiysonge C.S.
      • Oringanje C.
      • Nwachukwu C.E.
      • Oduwole O.
      • Meremikwu M.M.
      Interventions for improving coverage of childhood immunisation in low- and middle-income countries.
      • Ryman T.K.
      • Dietz V.
      • Cairns K.L.
      Too little but not too late: results of a literature review to improve routine immunization programs in developing countries.
      • Sridhar S.
      • Maleq N.
      • Guillermet E.
      • Colombini A.
      • Gessner B.D.
      A systematic literature review of missed opportunities for immunization in low- and middle-income countries.
      but there has been a missed opportunity to also measure the costs of these efforts. In a recent systematic review, 34 of the 41 full texts reviewed identified efforts to improve routine childhood immunization coverage, whereas 27 collected and reported effectiveness data on those efforts, 14 collected and reported costs, and 11 studies reported costs that permitted the estimation of an incremental cost-effectiveness ratio, albeit with caveats.
      • Munk C.
      • Portnoy A.
      • Suharlim C.
      • et al.
      Systematic review of the costs and effectiveness of interventions to increase infant vaccination coverage in low- and middle-income countries.
      Although collecting cost data alongside interventions of uncertain effectiveness and scalability may not demonstrate good value for money, an early understanding of the costs of these efforts is also important to judge future promise of scaling up coverage.
      Figure thumbnail gr1
      Figure 1Annual number of unique publications reporting immunization costs, 2005-2018. Note: In total, the Immunization Delivery Cost Catalogue identified 61 unique publications reporting immunization unit costs for both routine services and supplementary immunization activities in the peer-reviewed and gray literature between 2005 and 2018, representing 33 different countries.
      Immunization Costing Action Network (ICAN)
      2019. Immunization Delivery Cost Catalogue (IDCC). Washington: ThinkWell.
      • Vaughan K.
      • Ozaltin A.
      • Mallow M.
      • et al.
      The costs of delivering vaccines in low- and middle-income countries: findings from a systematic review.

      A Way Forward

      One potential solution to close the evidence gap for immunization economics would be to measure both the costs and effects of interventions, that is, conduct costing of discrete interventions designed to have incremental improvement in coverage that might support a generalizable cost function. Although routine data collection would also increase the reported cost data available, this approach would likely require health systems infrastructure improvements to support the routine collection of program expenditures and outputs. Additionally, such changes may lead to increased costs of introducing vaccines in LMICs, and would require consideration of the return on investment, compared to investing in the immunization program itself. In contrast, leveraging existing opportunities to collect costs alongside ongoing efforts to improve immunization coverage does not require health systems infrastructure improvements. Despite the need for useful cost data for immunization and other public health interventions, this solution has not been widely implemented. Although the addition of a cost component to ongoing immunization scale-up efforts is not trivial, it could also be quite valuable, particularly in addressing the costs of more intensive efforts to find unvaccinated children.
      • Brenzel L.
      • Halsey N.
      • Miller M.
      • Wolfson L.
      • Fox-Rushby J.
      Vaccine preventable diseases.
      ,
      • Ozawa S.
      • Yemeke T.T.
      • Evans D.R.
      • Pallas S.E.
      • Wallace A.S.
      • Lee B.Y.
      Defining hard-to-reach populations for vaccination.
      Future research investigating the effects of efforts to improve coverage should include a detailed description of the intervention, as well as the costs of those efforts and any contextual factors that might affect impact and costs. For example, describing whether the intervention focuses on demand generation, vaccine delivery, novel technologies, or health systems strengthening, as well as contextual factors such as rurality, health care access and infrastructure, and health-seeking behaviors/vaccine hesitancy.
      • Munk C.
      • Portnoy A.
      • Suharlim C.
      • et al.
      Systematic review of the costs and effectiveness of interventions to increase infant vaccination coverage in low- and middle-income countries.
      Ideally, these evaluations would describe a causal model relating the various mechanisms thought to limit current coverage and how these are affected by the intervention.
      • Ludwig J.
      • Kling J.R.
      • Mullainathan S.
      Mechanism experiments and policy evaluations.
      This model could then be used to investigate how intervention effects might generalize across settings. Such detailed reporting is required for extrapolating study evidence to other settings so that subsequent users of research findings can evaluate the relevant mechanisms of scale-up efforts rather than aspects local to a particular setting. Country-level decision makers considering activities to scale up immunization coverage using demand generation, for example, would then be able to refer to the cost-effectiveness of specific informational campaigns, such as home-based promotional education,
      • Owais A.
      • Hanif B.
      • Siddiqui A.R.
      • Agha A.
      • Zaidi A.K.
      Does improving maternal knowledge of vaccines impact infant immunization rates? A community-based randomized-controlled trial in Karachi, Pakistan.
      community meetings and discussion groups,
      • Andersson N.
      • Cockcroft A.
      • Ansari N.M.
      • et al.
      Evidence-based discussion increases childhood vaccination uptake: a randomised cluster controlled trial of knowledge translation in Pakistan.
      ,
      • Pandey P.
      • Sehgal A.R.
      • Riboud M.
      • Levine D.
      • Goyal M.
      Informing resource-poor populations and the delivery of entitled health and social services in rural India: a cluster randomized controlled trial.
      or targeted messaging of mothers with unvaccinated children.
      • Powell-Jackson T.
      • Fabbri C.
      • Dutt V.
      • Tougher S.
      • Singh K.
      Effect and cost-effectiveness of educating mothers about childhood DPT vaccination on immunisation uptake, knowledge, and perceptions in Uttar Pradesh, India: a randomised controlled trial.
      In the short term, costing studies will benefit from improved, systematic reporting and leveraging ongoing program evaluation efforts to collect costing data, while long-term investments in the health system and infrastructure may allow for routine data collection and improved efficiency for budgeting and planning.
      Author Contributions: Concept and design: Portnoy, Resch, Suharlim, Brenzel, Menzies
      Acquisition of data: Portnoy
      Analysis and interpretation of data: Portnoy, Suharlim
      Drafting of the manuscript: Portnoy, Brenzel
      Critical revision of the paper for important intellectual content: Resch, Suharlim, Brenzel, Menzies
      Provision of study materials or patients: Brenzel
      Obtaining funding: Resch, Brenzel
      Administrative, technical, or logistic support: Suharlim
      Supervision: Resch, Brenzel, Menzies
      Conflict of Interest Disclosures: Drs Resch and Menzies reported receiving grants from the Bill & Melinda Gates Foundation during the conduct of this study. No other disclosures were reported.
      Funding/Support: This work was supported by grant OPP1158709 from the Bill & Melinda Gates Foundation
      Role of the Funder/Sponsor: An employee of the funder (Brenzel) participated as a scientific collaborator.

      References

        • Brenzel L.
        What have we learned on costs and financing of routine immunization from the comprehensive multi-year plans in GAVI eligible countries?.
        Vaccine. 2015; 33: A93-A98
        • Hargreaves J.R.
        • Greenwood B.
        • Clift C.
        • et al.
        Making new vaccines affordable: a comparison of financing processes used to develop and deploy new meningococcal and pneumococcal conjugate vaccines.
        Lancet. 2011; 378: 1885-1893
        • Lydon P.
        • Levine R.
        • Makinen M.
        • et al.
        Introducing new vaccines in the poorest countries: what did we learn from the GAVI experience with financial sustainability?.
        Vaccine. 2008; 26: 6706-6716
        • Vassall A.
        • Sweeney S.
        • Kahn J.
        • et al.
        Reference case for estimating the costs of global health services and interventions. Global Health Cost Consortium.
        • Brenzel L.
        • Halsey N.
        • Miller M.
        • Wolfson L.
        • Fox-Rushby J.
        Vaccine preventable diseases.
        in: Jamison D. Bremen J. Measham A. Disease Control Priorities in Developing Countries. Oxford University Press, Oxford, England2006
        • Ozawa S.
        • Clark S.
        • Portnoy A.
        • Grewal S.
        • Brenzel L.
        • Walker D.G.
        Return on investment from childhood immunization in low- and middle-income countries, 2011-20.
        Health Aff (Millwood). 2016; 35: 199-207
        • Johns Hopkins University–International Vaccine Access Center (IVAC)
        Methodology Report: Decade of Vaccines Economics (DOVE) Return on Investment Analysis.
        • Frappier J.
        • Tremblay G.
        • Charny M.
        • Cloutier L.M.
        Costing bias in economic evaluations.
        J Med Econ. 2015; 18: 596-599
        • Immunization Costing Action Network (ICAN)
        2019. Immunization Delivery Cost Catalogue (IDCC). Washington: ThinkWell.
        http://immunizationeconomics.org/ican-idcc/
        Date accessed: April 11, 2019
        • Vaughan K.
        • Ozaltin A.
        • Mallow M.
        • et al.
        The costs of delivering vaccines in low- and middle-income countries: findings from a systematic review.
        Vaccine X. 2019; 2100034
        • Brenzel L.
        • Young D.
        • Walker D.G.
        Costs and financing of routine immunization: approach and selected findings of a multi-country study (EPIC).
        Vaccine. 2015; 33: A13-A20
        • Menzies N.A.
        • Suharlim C.
        • Geng F.
        • Ward Z.J.
        • Brenzel L.
        • Resch S.C.
        The cost determinants of routine infant immunization services: a meta-regression analysis of six country studies.
        BMC Med. 2017; 15: 178
        • Portnoy A.
        • Vaughan K.
        • Clarke-Deelder E.
        • et al.
        Producing standardized country-level immunization delivery unit cost estimates.
        PharmacoEconomics. 2020; 38: 995-1005
        • Graves N.
        • Walker D.
        • Raine R.
        • Hutchings A.
        • Roberts J.A.
        Cost data for individual patients included in clinical studies: no amount of statistical analysis can compensate for inadequate costing methods.
        Health Econ. 2002; 11: 735-739
        • Clement Nee Shrive F.M.
        • Ghali W.A.
        • Donaldson C.
        • Manns B.J.
        The impact of using different costing methods on the results of an economic evaluation of cardiac care: microcosting vs gross-costing approaches.
        Health Econ. 2009; 18: 377-388
        • Adam T.
        • Koopmanschap M.A.
        • Evans D.B.
        Cost-effectiveness analysis: can we reduce variability in costing methods?.
        Int J Technol Assess Health Care. 2003; 19: 407-420
        • EPIC Project (Harvard)
        Working Paper: how to cost immunization programs: a practical guide on primary data collection and analysis.
        • World Health Organization
        Global vaccine action plan 2011-2020.
        • Bishai D.
        • McQuestion M.
        • Chaudhry R.
        • Wigton A.
        The costs of scaling up vaccination in the world’s poorest countries.
        Health Aff (Millwood). 2006; 25: 348-356
        • Batt K.
        • Fox-Rushby J.A.
        • Castillo-Riquelme M.
        The costs, effects and cost-effectiveness of strategies to increase coverage of routine immunizations in low- and middle-income countries: systematic review of the grey literature.
        Bull World Health Organ. 2004; 82: 689-696
        • Pegurri E.
        • Fox-Rushby J.A.
        • Walker D.G.
        The effects and costs of expanding the coverage of immunisation services in developing countries: a systematic literature review.
        Vaccine. 2004; 23: 1624-1635
        • Munk C.
        • Portnoy A.
        • Suharlim C.
        • et al.
        Systematic review of the costs and effectiveness of interventions to increase infant vaccination coverage in low- and middle-income countries.
        BMC Health Serv Res. 2019; 19: 741
        • Ozawa S.
        • Yemeke T.T.
        • Thompson K.M.
        Systematic review of the incremental costs of interventions that increase immunization coverage.
        Vaccine. 2018; 36: 3641-3649
        • Johri M.
        • Perez M.C.
        • Arsenault C.
        • et al.
        Strategies to increase the demand for childhood vaccination in low- and middle-income countries: a systematic review and meta-analysis.
        Bull World Health Organ. 2015; 93: 339-346C
        • LaFond A.
        • Kanagat N.
        • Steinglass R.
        • Fields R.
        • Sequeira J.
        • Mookherji S.
        Drivers of routine immunization coverage improvement in Africa: findings from district-level case studies.
        Health Policy Plan. 2015; 30: 298-308
        • Nelson K.N.
        • Wallace A.S.
        • Sodha S.V.
        • Daniels D.
        • Dietz V.
        Assessing strategies for increasing urban routine immunization coverage of childhood vaccines in low and middle-income countries: a systematic review of peer-reviewed literature.
        Vaccine. 2016; 34: 5495-5503
        • Oyo-Ita A.
        • Wiysonge C.S.
        • Oringanje C.
        • Nwachukwu C.E.
        • Oduwole O.
        • Meremikwu M.M.
        Interventions for improving coverage of childhood immunisation in low- and middle-income countries.
        Cochrane Database Syst Rev. 2016; 7CD008145
        • Ryman T.K.
        • Dietz V.
        • Cairns K.L.
        Too little but not too late: results of a literature review to improve routine immunization programs in developing countries.
        BMC Health Serv Res. 2008; 8: 134
        • Sridhar S.
        • Maleq N.
        • Guillermet E.
        • Colombini A.
        • Gessner B.D.
        A systematic literature review of missed opportunities for immunization in low- and middle-income countries.
        Vaccine. 2014; 32: 6870-6879
        • Ozawa S.
        • Yemeke T.T.
        • Evans D.R.
        • Pallas S.E.
        • Wallace A.S.
        • Lee B.Y.
        Defining hard-to-reach populations for vaccination.
        Vaccine. 2019; 37: 5525-5534
        • Ludwig J.
        • Kling J.R.
        • Mullainathan S.
        Mechanism experiments and policy evaluations.
        J Econ Perspect. 2011; 25: 17-38
        • Owais A.
        • Hanif B.
        • Siddiqui A.R.
        • Agha A.
        • Zaidi A.K.
        Does improving maternal knowledge of vaccines impact infant immunization rates? A community-based randomized-controlled trial in Karachi, Pakistan.
        BMC Public Health. 2011; 11: 239
        • Andersson N.
        • Cockcroft A.
        • Ansari N.M.
        • et al.
        Evidence-based discussion increases childhood vaccination uptake: a randomised cluster controlled trial of knowledge translation in Pakistan.
        BMC Int Health Hum Rights. 2009; 9: S8
        • Pandey P.
        • Sehgal A.R.
        • Riboud M.
        • Levine D.
        • Goyal M.
        Informing resource-poor populations and the delivery of entitled health and social services in rural India: a cluster randomized controlled trial.
        JAMA. 2007; 298: 1867-1875
        • Powell-Jackson T.
        • Fabbri C.
        • Dutt V.
        • Tougher S.
        • Singh K.
        Effect and cost-effectiveness of educating mothers about childhood DPT vaccination on immunisation uptake, knowledge, and perceptions in Uttar Pradesh, India: a randomised controlled trial.
        PLoS Med. 2018; 15e1002519