Advertisement

On the Estimation of the Cost-Effectiveness Threshold: Why, What, How?

Open ArchivePublished:April 22, 2016DOI:https://doi.org/10.1016/j.jval.2016.02.020

      Abstract

      Background

      Many health care systems claim to incorporate the cost-effectiveness criterion in their investment decisions. Information on the system’s willingness to pay per effectiveness unit, normally measured as quality-adjusted life-years (QALYs), however, is not available in most countries. This is partly because of the controversy that remains around the use of a cost-effectiveness threshold, about what the threshold ought to represent, and about the appropriate methodology to arrive at a threshold value.

      Objectives

      The aim of this article was to identify and critically appraise the conceptual perspectives and methodologies used to date to estimate the cost-effectiveness threshold.

      Methods

      We provided an in-depth discussion of different conceptual views and undertook a systematic review of empirical analyses. Identified studies were categorized into the two main conceptual perspectives that argue that the threshold should reflect 1) the value that society places on a QALY and 2) the opportunity cost of investment to the system given budget constraints.

      Results

      These studies showed different underpinning assumptions, strengths, and limitations, which are highlighted and discussed. Furthermore, this review allowed us to compare the cost-effectiveness threshold estimates derived from different types of studies. We found that thresholds based on society’s valuation of a QALY are generally larger than thresholds resulting from estimating the opportunity cost to the health care system.

      Conclusions

      This implies that some interventions with positive social net benefits, as informed by individuals’ preferences, might not be an appropriate use of resources under fixed budget constraints.

      Keywords

      Introduction

      Among the various countries that claim to incorporate the cost-effectiveness criterion in their investment decisions, only the National Institute for Health and Care Excellence in England and Wales explicitly reports the threshold value used as the system’s maximum willingness to pay (WTP) for a quality-adjusted life-year (QALY). The National Institute for Health and Care Excellence uses a range of £20,000 to £30,000 per QALY [

      National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013. Available from: https://www.nice.org.uk/article/pmg9/resources/non-guidance-guide-to-the-methods-of-technology-appraisal-2013-pdf. [Accessed January 11, 2015].

      ] (~€22,000–€33,000 in 2014 euros). In other countries, specific figures or ranges have been recommended but these have not been formally adopted, such as the range of $20,000 to $100,000 (~€15,000–€75,000) in the United States [
      • Kaplan R.M.
      • Bush J.W.
      Health-related quality of life measurement for evaluation research and policy analysis.
      ] and Canada [
      • Laupacis A.
      • Feeny D.
      • Detsky A.S.
      • Tugwell P.X.
      How attractive does a new technology have to be to warrant adoption and utilization? Tentative guidelines for using clinical and economic evaluations.
      ], or the widely and controversially cited threshold of $50,000 (~€36,000) also in the United States [
      • Bridges J.F.P.
      • Onukwugha E.
      • Mullins C.D.
      Healthcare rationing by proxy: cost-effectiveness analysis and the misuse of the $50,000 threshold in the US.
      ,
      • Neumann P.J.
      • Cohen J.T.
      • Weinstein M.C.
      Updating cost-effectiveness--the curious resilience of the $50,000-per-QALY threshold.
      ]. In Sweden and the Netherlands, relevant government authorities and important advisory bodies have recommended thresholds of 500,000 SEK (~€57,000) [

      Socialstyrelsen. Nationella riktlinjer för prostatacancersjukvård—Medicinskt och hälsoekonomiskt faktadokument. Available from: Disponible en: http://www.wolterskluwer.se/produkt/9789185483075. [Accessed January 15, 2015].

      ,
      • Ryen L.
      • Svensson M.
      The willingness to pay for a quality adjusted life year: a review of the empirical literature.
      ] and €80,000 [
      • Bobinac A.
      • Van Exel N.J.A.
      • Rutten F.F.H.
      • Brouwer W.B.F.
      Willingness to pay for a quality-adjusted life-year: the individual perspective.
      ], respectively. Furthermore, the World Health Organization recommends that a disability-adjusted life-year be valued at a maximum of 3 times the country’s gross domestic product per capita [

      World Health Organization. Cost effectiveness and strategic planning (WHO-CHOICE). 2005. Available from: http://www.who.int/choice/costs/CER_levels/en/. [Accessed December 4, 2014].

      ].
      The lack of theoretical and empirical basis regarding the above figures has contributed to an ongoing debate about the appropriate way of estimating the cost-effectiveness threshold and to an increasing body of empirical research in this area in many countries. However, this growing literature has not necessarily facilitated the adoption of specific thresholds values, partly because of the lack of consensus about relevant issues. There remains much controversy around the use of a threshold, about what the threshold ought to represent, and about the appropriate methodology to arrive at a threshold value.
      This study aimed to review the different approaches used to date to estimate the cost-effectiveness threshold. We provide an in-depth discussion of the proposed conceptual views, and we systematically review and critically appraise the empirical literature. Furthermore, this comprehensive review allow us to summarize and compare the estimates of the cost-effectiveness threshold available in the literature. The goal of this article was thus to facilitate researchers and decision makers with a comprehensive understanding of the existing evidence and the challenges of estimating a threshold value, and to offer some recommendations about the appropriateness of different alternatives in different contexts.

      The Cost-Effectiveness Threshold—Why?

      Some authors have argued against the use of a threshold, and more broadly against the use of the cost per QALY approach. These authors consider that this approach is consistent only with the maximization of health gains from available resources under some assumptions, including perfect divisibility of health care programs, and that these conditions do not hold in the settings faced by health care decision makers [
      • Birch S.
      • Gafni A.
      Changing the problem to fit the solution: Johannesson and Weinstein’s (mis) application of economics to real world problems.
      ,
      • Birch S.
      • Gafni A.
      The biggest bang for the buck or bigger bucks for the bang: the fallacy of the cost-effectiveness threshold.
      ]. The use of a threshold has also been related to uncontrolled increases in health care spending [
      • Gafni A.
      • Birch S.
      Incremental cost-effectiveness ratios (ICERs): the silence of the lambda.
      ], and to transferring the full value of new technologies to manufacturers, by encouraging companies to set the price where the cost per QALY equals the cost-effectiveness threshold [
      • Mccabe C.
      • Claxton K.
      • Culyer A.J.
      Cost-effectiveness threshold: what it is and what that means.
      ].
      However, a number of authors have considered the use of the threshold approach a useful approximation to improve efficiency, and have suggested means to address the above arguments, such as complementing cost-effectiveness analysis with budget impact analyses [
      • Eichler H.-G.
      • Kong S.X.
      • Gerth W.C.
      • et al.
      Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge?.
      ] and regularly adjusting the threshold to account for changes in efficiency and the budget over time [
      • Mccabe C.
      • Claxton K.
      • Culyer A.J.
      Cost-effectiveness threshold: what it is and what that means.
      ]. Furthermore, the advantages of setting a threshold have been considered beyond improvements in efficiency, such as allowing for better consistency and transparency of the decision-making process, and enhancing equity and public trust by reducing the room for decision makers’ arbitrariness [
      • Eichler H.-G.
      • Kong S.X.
      • Gerth W.C.
      • et al.
      Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge?.
      ].

      The Cost-Effectiveness Threshold—What?

      Even among those who welcome the use of a threshold there are different views as to what the threshold ought to represent. The two main conceptual perspectives include the view that the threshold should reflect 1) society’s monetary valuation of health gains, or 2) the opportunity cost resulting from the disinvestment required to adopt a new technology.
      The former perspective is traced to attempts to link cost-effectiveness analysis with cost-benefit analysis and welfare economics [
      • Phelps C.E.
      • Mushlin A.I.
      On the (near) equivalence of cost-effectiveness and cost-benefit analyses.
      ]. The authors who advocate for this perspective argue this to be in line with the general approach taken in other public sectors in many countries where cost-benefit analyses are used to make investment decisions [
      • Baker R.
      • Chilton S.
      • Donaldson C.
      • et al.
      Searchers vs surveyors in estimating the monetary value of a QALY: resolving a nasty dilemma for NICE.
      ]. The opportunity cost perspective, however, is based on the idea that to adopt a new technology that imposes additional costs on the health care system, displacement of existing services might be required. Disinvesting on existing interventions will most likely result in health losses for individuals elsewhere. The threshold should thus represent the cost per QALY of displaced services, which would allow the assessment of whether the health expected to be gained from the use of a new technology exceeds the health expected to be forgone elsewhere as other services are displaced [
      • Claxton K.
      • Martin S.
      • Soares M.
      • et al.
      Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold.
      ].
      The main argument of advocates of the opportunity cost approach is that information on society’s valuation of improvements in health is irrelevant for threshold-setting purposes, because it cannot inform on how to allocate a fixed budget within a health care system. The reason is that individuals’ monetary valuations for health gains are detached from the budget-setting process [
      • Culyer A.
      • McCabe C.
      • Briggs A.
      • et al.
      Searching for a threshold, not setting one: the role of the National Institute for Health and Clinical Excellence.
      ]. However, those who defend the society-value approach argue that the budget-setting process should, in fact, be informed about preferences of members of the public [
      • Baker R.
      • Chilton S.
      • Donaldson C.
      • et al.
      Searchers vs surveyors in estimating the monetary value of a QALY: resolving a nasty dilemma for NICE.
      ]. In a way of attempting to resolve this “dilemma,” Baker et al. [
      • Baker R.
      • Chilton S.
      • Donaldson C.
      • et al.
      Searchers vs surveyors in estimating the monetary value of a QALY: resolving a nasty dilemma for NICE.
      ] suggested a framework that considers both approaches as complementary; in particular, as the demand and supply sides of the “market” for QALYs. Society’s valuation provides an estimate of the marginal benefits to consumers of health care services (demand side), whereas the opportunity costs approach relates to the marginal costs of health care spending (supply side). The authors of this article conclude that a framework incorporating these two approaches is to be preferred.

      The Cost-Effectiveness Threshold—How?

      A systematic review of the literature was undertaken with the aim of identifying empirical research that has focused on estimating a cost-effectiveness threshold. We identified 38 studies that were categorized into demand-side (29 articles) and supply-side (9 articles) studies. A description of search methods and a complete list of references are provided in Supplemental Materials found at 10.1016/j.jval.2016.02.020.

       Demand-Side Empirical Research

      The underpinning idea of studies focusing on society’s value of health gains is that sector-allocative decisions should reflect the strengths of preferences of those members of society affected by their decisions. The study of the maximum WTP has become the norm for the monetary valuation of publicly funded goods in several countries, and especially so in the United Kingdom [
      • Mason H.
      • Jones-Lee M.
      • Donaldson C.
      Modelling the monetary value of a QALY: a new approach based on UK data.
      ]. Based on our review, there are two ways in which the WTP value for a QALY has been estimated: 1) directly eliciting individuals’ WTP using surveys (26 studies) and 2) inferring from information about the value of a statistical life (VSL) the corresponding value of a QALY (4 studies). One of the identified studies used both approaches [

      Donaldson C, Baker R, Mason H, et al. European value of a quality adjusted life year. 2010. Available from: http://research.ncl.ac.uk/eurovaq/EuroVaQ_Final_Publishable_Report_and_Appendices.pdf. [Accessed November 15, 2014].

      ].

       WTP surveys

      WTP for a QALY survey focuses on eliciting the maximum amount individuals are willing to pay for a, normally, small health gain and then aggregate the WTP needed to gain a QALY. The process normally required three steps: 1) to estimate in terms of utility values a health gain using methods such as time trade-off or standard gamble, 2) to elicit the WTP for that health gain, and 3) to combine the answers of these two estimates to arrive at a WTP for a QALY. The latter step can be applied using an aggregated approach, in which the mean WTP and the mean utility value across the sample are computed separately and combined into a ratio (ratio of means), or a disaggregated approach, which implies calculating this ratio for each individual and computing the mean across the sample (mean of ratios).
      Table 1 summarizes the characteristics and results of identified WTP articles, showing separately the values when those were reported using different techniques or when the study provided estimates for different countries. There is a wide variation in results, ranging from just over €1000 to more than 5 million per life-year (LY)/QALY (values reported in €2014). The largest and most recent international study, the EuroVaQ [

      Donaldson C, Baker R, Mason H, et al. European value of a quality adjusted life year. 2010. Available from: http://research.ncl.ac.uk/eurovaq/EuroVaQ_Final_Publishable_Report_and_Appendices.pdf. [Accessed November 15, 2014].

      ,
      • Robinson A
      • Gyrd-Hansen D
      • Bacon P
      • et al.
      Estimatinga WTP-based value of a QALY: the “chained” approach.
      ] conducted in nine European Union countries, estimated an “all countries” value of €14,430 to €26,965.
      Table 1Characteristics and findings of studies using WTP surveys
      StudySubjectQALY methodWTP formatCombinationPerspectiveCountryReported thresholdThreshold €2014
      The values were updated and converted to euros 2014 (€) using implied conversion factors based on the Purchasing Power Parity ratio in 2014.
      Blumenschein,
      • Blumenschein K.
      • Johannesson M.
      Relationship between quality of life instruments, health state utilities, and willingness to pay in patients with asthma.
      , 1998
      AsthmaVAS/SG/TTODichotomous/bidding gameAggregatedIndividualUnited States (Kentucky)$7,000–$46,0007,210–47,379
      Zethraeus,
      • Zethraeus N.
      Willingness to pay for hormone replacement therapy.
      1998
      Patients with hormone replacementVASDichotomousAggregatedIndividualSweden118,400 SEK12,297
      TTO156,100 SEK16,365
      Cunningham,
      • Cunningham S.J.
      • Hunt N.
      Relationship between utility values and willingness to pay in patients undergoing orthognathic treatment.
      , 2000
      Orthognathic patientsSGCard sortingDisaggregatedIndividualUnited Kingdom (London)£5061055
      Gyrd-Hansen,
      • Gyrd-Hansen D.
      Willingness to pay for a QALY.
      , 2003
      General healthEQ-5D tariffsDichotomous (discrete choice)AggregatedIndividualDenmark88,000 DKK9,724
      Byrne,
      • Byrne M.M.
      • O’malley K.
      • Suarez-Almazor M.E.
      Willingness to pay per quality-adjusted life year in a study of knee osteoarthritis.
      , 2005
      Osteoarthritis (general population)VASOpen-endedDisaggregatedIndividualUnited States (Texas)$1,221–$3,5271,152–3,330
      SG$2,844–$4,4912,685–4,240
      TTO$3,802–$5,6903,589–5,372
      King,
      • King J.T.
      • Tsevat J.
      • Lave J.R.
      • Roberts M.S.
      Willingness to pay for a quality-adjusted life year: implications for societal health care resource allocation.
      , 2005
      General health (patients)VASBidding gameDisaggregatedIndividualUnited States$16,60014,728
      SG$25,40022,536
      TTO$27,10024,045
      Pinto-Prades,
      • Pinto Prades J.L.
      • Martinez Perez J.
      Estimacion del valor monetario de los anos de vida ajustados por calidad: Estimaciones preliminares.
      , 2005
      General healthSGOpen-endedDisaggregatedIndividualSpain€9,000–€38,00010,000–42,500
      Lieu,
      • Lieu Ta
      • Ray G.T.
      • Ortega-Sanchez I.R.
      • Kleinman K.
      • Rusinak D.
      • Prosser La
      Willingness to pay for a QALY based on community member and patient preferences for temporary health states associated with herpes zoster.
      , 2009
      CommunityTTOBidding game followed by open-endedDisaggregatedIndividualUnited States$33,00028,500
      Shingles$37,00031,800
      Postherpetic neuralgia$70,00060,200
      Pinto-Prades et al.
      • Pinto-Prades J.L.
      • Loomes G.
      • Brey R.
      Trying to estimate a monetary value for the QALY.
      , 2009
      General healthSGCard sortingDisaggregatedIndividualSpain€4,585–€123,7244,660–125,744
      Shiroiwa,
      • Shiroiwa T
      • Sung Y
      • Fukuda T
      • Lang H
      • Bae S-C
      • Tsutani K.
      International survey on willingness-to-pay (WTP) for one additional QALY gained: what is the threshold of cost effectiveness?.
      , 2010
      General healthDescription of a QALY gainedDichotomous followed by bidding gameDisaggregatedIndividualJapan5.0 million JPY34,000
      Family member6.4 million JPY44,000
      Society5.4 million JPY37,000
      IndividualKorea68 million KWN71,000
      Family member79 million KWN82,000
      Society69 million KWN72,000
      IndividualTaiwan2.1 million NT$98,000
      Family member1.9 million NT$88,000
      Society1.8 million NT$84,000
      IndividualUnited Kingdom£23,00028,000
      Family member£26,00032,000
      Society£38,00046,000
      IndividualAustralia$64,00034,000
      Family member$78,00041,000
      Society$89,00047,000
      IndividualUnited States$62,00049,000
      Family member$69,00055,000
      Society$96,00076,000
      Baker et al.
      • Baker R
      • Bateman I
      • Donaldson C
      • et al.
      Weighting and valuing quality-adjusted life-years using stated preference methods: preliminary results from the Social Value of a QALY Project.
      , 2010, and Donaldson,
      • Donaldson C
      • Baker R
      • Mason H
      • et al.
      The social value of a QALY: raising the bar or barring the raise?.
      , 2011
      HeadacheSGCard sorting followed by open-endedAggregatedIndividualEngland£22,570–£41,53028,340–52,257
      Stomach illness£17,980–£31,15022,624–39,196
      Bobinac et al.
      • Bobinac A.
      • Van Exel N.J.A.
      • Rutten F.F.H.
      • Brouwer W.B.F.
      Willingness to pay for a quality-adjusted life-year: the individual perspective.
      ,
      • Cunningham S.J.
      • Hunt N.
      Relationship between utility values and willingness to pay in patients undergoing orthognathic treatment.
      , 2010 and 2012
      General healthVASCard sorting followed by open-endedAggregatedIndividualthe Netherlands€12,90011,657
      EQ-5D tariffs€24,50022,139
      Zhao,
      • Zhao F
      • Yue M
      • Yang H
      • Wang T
      • Wu J-H
      • Li S.
      Willingness to pay per quality-adjusted life year: is one threshold enough for decision-making?: results from a study in patients with chronic prostatitis.
      , 2011
      General healthEQ-5D TariffsDichotomousDisaggregatedIndividualChina$7,30610,352
      Patients with prostatitis$4,7116,675
      Haninger and Hammitt
      • Byrne M.M.
      • O’malley K.
      • Suarez-Almazor M.E.
      Willingness to pay per quality-adjusted life year in a study of knee osteoarthritis.
      , 2011
      Foodborne riskVAS/HUI tariffsDichotomousAggregatedIndividualUnited States$150,000–$5.6 million133,000–5.0 million
      Soini,
      • Soini E.
      Contingent Valuation of Eight New Treatments: What is the Clinician’s and Politician’s Willingness to Pay?.
      , 2012
      8 diseases (clinicians and politicians)15D tariffsClose-endedDisaggregatedSocietyFinland€94,770102,356
      EQ-5D tariffs€102,616110,830
      Gyrd-Hansen and Kjaer
      • Gyrd-Hansen D.
      Willingness to pay for a QALY.
      , 2012
      General healthSGClose-endedAggregatedIndividualDenmark€2,740–€8,3002,082–6,305
      Open-ended€20,00015,000
      Close-endedDisaggregated€12,961–€96,3669,846–73,208
      Open-ended€27,32520,758
      Shiroiwa,
      • Shiroiwa T
      • Igarashi A
      • Fukuda T
      • Ikeda S.
      WTP for a QALY and health states: More money for severer health states?.
      , 2012
      General healthEQ-5D tariffsDouble-bounded dichotomousAggregatedIndividualJapan2–8 million JPY14,000–57,000
      Bobinac,
      • Bobinac A
      • van Exel NJA
      • Rutten FFH
      • Brouwer WBF
      Valuing qaly gains by applying a societal perspective.
      , 2013
      General healthEQ-5D tariffsCard sorting followed by open-endedAggregatedSocietythe Netherlands€52,200–€65,10047,100–58,800
      Individual/society€59,200–€83,20053,400–75,100
      Thavorncharoensap,
      • Thavorncharoensap M
      • Teerawattananon Y
      • Natanant S
      • Kulpeng W
      • Yothasamut J
      • Werayingyong P.
      Estimating the willingness to pay for a quality-adjusted life year in Thailand: does the context of health gain matter?.
      , 2013
      BlindnessTTOBidding game followed by open-endedDisaggregatedIndividualThailand113,000–285,000 baht5,260–13,264
      VAS31,000–108,000 baht1,442–5,026
      ParaplegiaTTO26,000–101,000 baht1,210–4,816
      VAS26,000–79,000 baht1,210–4,700
      AllergiesTTO88,000–92,000 baht4,095–4,282
      VAS27,000–41,000 baht1,257–1,908
      Donaldson et al.

      Donaldson C, Baker R, Mason H, et al. European value of a quality adjusted life year. 2010. Available from: http://research.ncl.ac.uk/eurovaq/EuroVaQ_Final_Publishable_Report_and_Appendices.pdf. [Accessed November 15, 2014].

      , 2010, and Robinson et al.
      • Robinson A
      • Gyrd-Hansen D
      • Bacon P
      • et al.
      Estimatinga WTP-based value of a QALY: the “chained” approach.
      , 2013
      General healthSG/TTOCard sorting followed by open-endedAggregatedIndividualEngland$13,228–$29,30810,330–22,888
      the Netherlands$15,738–$27,41812,288–21,407
      France$11,317–$26,8908,846–21,019
      Spain$26,299–$52,87620,538–41,292
      Norway$24,757–$41,29819,344–32,269
      Sweden$16,908–$34,82413,211–27,210
      Denmark$24,796–$57,38919,375–44,842
      Hungary$10,938–$26,1328,547–20,419
      Poland$18,601–$40,02314,539–31,283
      All countries$18,247–$34,09714,430–26,965
      Pennington,
      • Pennington M
      • Baker R
      • Brouwer W
      • et al.
      Comparing WTP values of different types of QALY gain elicited from the general public.
      , 2013
      General healthDescriptionCard sorting followed by open-endedAggregatedIndividualAs previous$10,744–$29,0628,395–22,708
      Bobinac et al.
      • Bobinac A
      • van Exel J
      • Rutten FFH
      • Brouwer WBF.
      The value of a QALY: individual willingness to pay for health gains under risk.
      , 2014
      General healthEQ-5D tariffsCard sorting followed by open-endedAggregatedIndividualthe Netherlands€80,800–€113,00072,900–102,000
      Martín-Fernández,
      • Martín-Fernández J
      • Polentinos-Castro E
      • del Cura-González MI
      • et al.
      Willingness to pay for a quality-adjusted life year: an evaluation of attitudes towards risk and preferences.
      , 2014
      General health (patients)EQ-5D tariffsBidding game (direct payment)DisaggregatedIndividualSpain (Madrid)€10,11911,823
      Bidding game (tax payment)€28,18732,933
      VASBidding game (direct payment)€10,30512,040
      Bidding game (tax payment)€28,09332,824
      EQ-5D, EuroQol five-dimensional questionnaire; 15D, 15-dimensional generic instrument; HUI, health utilities index; JPY, Japanese yen; KWN, Korean won; NT, New Taiwanese; SEK, Swedish kroner; SG, standard gamble; TTO, time trade-off; VAS, visual analogue scale; WTP, willingness to pay.
      low asterisk The values were updated and converted to euros 2014 (€) using implied conversion factors based on the Purchasing Power Parity ratio in 2014.
      The main source of variation across values is arguably due to the violation of the implied assumption in this approach, that is, that valuation of health gains is linear with respect to changes in quality of life (QOL) and duration. This assumption has been shown to not hold; a number of studies have found evidence of nonlinearities with respect to severity, duration, and mortality risk size [
      • Pinto-Prades J.L.
      • Loomes G.
      • Brey R.
      Trying to estimate a monetary value for the QALY.
      ,
      • Robinson A
      • Gyrd-Hansen D
      • Bacon P
      • et al.
      Estimatinga WTP-based value of a QALY: the “chained” approach.
      ,
      • Bobinac A
      • van Exel J
      • Rutten FFH
      • Brouwer WBF.
      The value of a QALY: individual willingness to pay for health gains under risk.
      ,
      • Bobinac A.
      • van Exel N.J.A.
      • Rutten F.F.H.
      • Brouwer W.B.F.
      Get more, pay more? An elaborate test of construct validity of willingness to pay per QALY estimates obtained through contingent valuation.
      ,
      • Gyrd-Hansen D.
      • Kjaer T.
      Disentangling WTP per QALY data: different analytical approaches, different answers.
      ,
      • Haninger K.
      • Hammitt J.K.
      Diminishing willingness to pay per quality-adjusted life year: valuing acute foodborne illness.
      ]. A second issue is related to the aggregation of answers provided from two different processes, that is, the substitution between wealth and health implied in the WTP questionnaires and the substitution between QOL and time to death/risk of death implied in the utility questionnaires [
      • Gyrd-Hansen D.
      • Kjaer T.
      Disentangling WTP per QALY data: different analytical approaches, different answers.
      ]. Individuals are sometimes willing to make a trade off in one dimension but not in the other. This can yield to estimates of apparent infinite value of a QALY if individuals report to be willing to pay something for an improvement in QOL for which they were not prepared to trade time or a risk of death. The aggregation method (aggregate vs. disaggregate) of WTP and utility estimates has also been shown to strongly affect the results [
      • Gyrd-Hansen D.
      • Kjaer T.
      Disentangling WTP per QALY data: different analytical approaches, different answers.
      ]. Not less important is the impact that alternative procedural techniques such as standard gamble versus time trade-off or close-ended versus open-ended questions in the WTP surveys have on the values obtained, which leads to different answers depending on the choice of methods.

       VSL analysis

      The rest of the identified studies on the demand side have used information readily available on the VSL, commonly used in the context of transport and environmental policies [
      • Mason H.
      • Jones-Lee M.
      • Donaldson C.
      Modelling the monetary value of a QALY: a new approach based on UK data.
      ]. This approach consists of computing the quality-adjusted life expectancy related to the population for which the VSL was based, and then combining these two pieces of information to infer the value of a QALY. Methodologically, the studies mainly vary by the methods used in estimating the VSL and with respect to the assumptions on the rate at which the VSL changes with respect to life expectancy. The VSL can be estimated using different approaches: 1) the human capital approach based on expected earnings; 2) WTP surveys about reductions in mortality risk; and 3) the revealed preference approach, based on observation of individual market behavior usually divided between occupational (job risk) and nonoccupational safety.
      Table 2 summarizes the characteristics and results of the studies using this approach. The estimates found in these articles are generally higher than those based on WTP surveys, with the lowest estimate being more than €30,000 and with nearly half the figures being higher than €100,000 per QALY.
      Table 2Characteristics and findings of studies applying the VSL approach
      StudyVSL methodRate between VSL/life expectancyCountryReported thresholdThreshold €2014
      The values were updated and converted to euros 2014 (€) using implied conversion factors based on the Purchasing Power Parity ratio in 2014.
      Hirth,
      • Hirth R.
      • Chernew M.E.
      • Miller E.
      • Fendrick a.M
      • Weissert W.G.
      Willingness to Pay for a Quality-adjusted Life Year: In Search of a Standard.
      , 2000
      Human capitalConstant valueUnited States, United Kingdom, Canada, France, Denmark$33,42133,843
      Revealed preferences/job risk$415,469420,72
      Revealed preferences/nonoccupational safety$352,362356,81
      Contingent valuation/WTP$331,666335,86
      Mason et al.
      • Mason H.
      • Jones-Lee M.
      • Donaldson C.
      Modelling the monetary value of a QALY: a new approach based on UK data.
      , 2009
      Contingent valuation/WTPConstant valueUnited Kingdom£70,89693,88
      Decreasing at age 18+ y£34,92546,25
      Decreasing at age 40+ y£67,47089,34
      Donaldson et al.

      Donaldson C, Baker R, Mason H, et al. European value of a quality adjusted life year. 2010. Available from: http://research.ncl.ac.uk/eurovaq/EuroVaQ_Final_Publishable_Report_and_Appendices.pdf. [Accessed November 15, 2014].

      , 2010
      Contingent valuation/WTPConstant valueEngland€102,373106,595
      Decreasing at age 18+ y€50,52452,607
      Decreasing at age 40+ y€77,88481,096
      Contingent valuation/WTPConstant valuethe Netherlands€180,295187,731
      Decreasing at age 18+ y€55,27457,553
      Decreasing at age 40+ y€122,598127,654
      Revealed preferences/job riskConstant valueSpain€178,527185,890
      Decreasing at age 18+ y€92,48896,302
      Decreasing at age 40+ y€171,476178,548
      Contingent valuation/WTPConstant valueSweden€110,961115,537
      Decreasing at age 18+ y€50,71252,803
      Decreasing at age 40+ y€168,152175,087
      Human capitalConstant valueDenmark€79,89283,187
      Decreasing at age 18+ y€32,75434,104
      Decreasing at age 40+ y€87,75291,371
      Revealed preferences/job riskConstant valueHungary€112,234116,863
      Decreasing at age 18+ y€51,14553,254
      Decreasing at age 40+ y€104,290108,591
      Abellán,

      Abellán Perpiñán JM, Martínez Pérez JE, Méndez Martínez I, Sánchez Martinez FI, Pinto-Prades JL, Robles Zurita JA. El Valor Monetario de Una Víctima No Mortal Y Del Año de Vida Ajustado Por Calidad En España. Dirección General de Tráfico. 2011. Available from http://www.dgt.es/Galerias/seguridad-vial/investigacion/estudios-e-informes/2011/SPAD1A_-.-ESTIMACION-EN-EL-CONTEXTO-DE-LOS-ACCIDENTES-DE-TRAFICO_INFORME-PARA-WEB.pdf. [Accessed January 15, 2015].

      , 2011
      Contingent valuation/WTPConstant valueSpain€53,58654,408
      VSL, value of statistical life; WTP, willingness to pay.
      low asterisk The values were updated and converted to euros 2014 (€) using implied conversion factors based on the Purchasing Power Parity ratio in 2014.
      As expected, studies that rely on estimations of the VSL would result in different estimates depending on the approach taken to value a life and the assumptions regarding the rate at which the VSL changes with life expectancy. It is also worth noting that the sole focus on mortality is likely to yield to estimations of the value of a QALY that are considerably higher as compared with those centered on valuation of QOL improvements.

       Supply-Side Empirical Research

      With respect to the work on the supply side, the opportunity cost of health care interventions could arguably be best identified by the use of “league tables” [
      • Gold M.R.
      • Siegel J.E.
      • Russell L.B.
      • Weinstein M.C.
      Cost-effectiveness in health and medicine.
      ]. However, this approach requires comprehensive and comparable information on the cost per QALY of all potential interventions, as well as the infrequent scenario of starting completely anew the composition of the health service package provided by a health care system [
      • Eichler H.-G.
      • Kong S.X.
      • Gerth W.C.
      • et al.
      Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge?.
      ]. In the most common case in which new technologies, on a one-by-one basis, are assessed in terms of their cost-effectiveness, a second-best solution is to identify the intervention (or combination of interventions) that needs to be cancelled to generate the additional resources required to implement the new one [
      • Birch S.
      • Gafni A.
      The biggest bang for the buck or bigger bucks for the bang: the fallacy of the cost-effectiveness threshold.
      ]. However, some have argued that we do not know what activities will be displaced when imposing new costs to the health care system [
      • Towse A.
      Should NICE’s threshold range for cost per QALY be raised? Yes.
      ], and it is also likely that different areas within a system disinvest on different services [
      • Appleby J.
      • Devlin N.
      • Parkin D.
      • et al.
      Searching for cost effectiveness thresholds in the NHS.
      ]. The empirical literature on the supply side has as a result focused on two proxy approaches: 1) the observation of past investment decisions (four studies) and 2) the empirical estimation of the marginal cost of a QALY (five studies).

       Past funding decisions

      This approach consists of identifying a set of decisions about health care investment and comparing them with their cost per QALY. The idea is that if many decisions are observed it might be possible to infer the cost per QALY under which favorable decisions are made, that is, the implicit cost-effectiveness threshold.
      The studies using this method are summarized in Table 3. Only one study from Australia was able to infer a range of €32,000 to €58,000 [
      • George B.
      • Harris A.
      • Mitchell A.
      Cost-effectiveness analysis and the consistency of decision making: evidence from pharmaceutical reimbursement in Australia (1991 to 1996).
      ]. The rest of the studies did not find enough consistency between funding decisions to infer a threshold.
      Table 3Characteristics and findings of studies applying the observation of past funding decisions approach
      StudyCountryData on funding decisionsNumber of comparisonsReported thresholdThreshold €2014
      George et al.
      • George B.
      • Harris A.
      • Mitchell A.
      Cost-effectiveness analysis and the consistency of decision making: evidence from pharmaceutical reimbursement in Australia (1991 to 1996).
      , 2001
      AustraliaPharmaceutical Benefits Advisory Committee35 submissions$42,000–$76,00032,000–58,000
      Devlin,
      • Devlin N.
      • Parkin D.
      Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis.
      , 2004
      EnglandNICE Guidance and Technology Appraisals33 decisions NineNA
      Appleby et al.
      • Appleby J.
      • Devlin N.
      • Parkin D.
      • et al.
      Searching for cost effectiveness thresholds in the NHS.
      , 2009
      EnglandSix local NHS purchasers and 18 providerstechnologiesNA
      Chambers,
      • Chambers J.D.
      • Neumann P.J.
      • Buxton M.J.
      Does Medicare have an implicit cost-effectiveness threshold?.
      , 2010
      United StatesNational coverage determinations from Centers for Medicare & Medicaid Services64 decisionsNA
      NA, not applicable/available; NHS, National Health Service; NICE, National Institute for Health and Care Excellence.
      These failed attempts highlight the difficulties of such approach. This methodology requires complete transparency regarding information on funding decisions, about the criteria taken into account to arrive at such decisions, and the explicit role of cost-effectiveness in that decision-making process. Moreover, there is one conceptual criticism to use this approach, which relates to the view that it is not necessarily desirable for current decisions to use the same decision rule as for previous ones. This is especially the case when there is no evidence that the decisions made in the past have led to improvement in efficiency, but even if so, some authors claim that as the budget and efficiency of the technologies change, the threshold should be regularly adjusted to reflect these variations [
      • Mccabe C.
      • Claxton K.
      • Culyer A.J.
      Cost-effectiveness threshold: what it is and what that means.
      ].

       Effect of expenditure

      This set of studies aims to estimate the cost per QALY at which the system currently operates, which would reflect the average opportunity cost. The general approach consists of three main steps: 1) modeling health outcomes, normally a measure of mortality, against health expenditure and other control variables to estimate the health expenditure elasticity; 2) to translate this effect into an effect expressed in LY gain; and 3) to then adjust this impact to account for QOL to approximate the result to the marginal cost of a QALY. The adjustment for QOL has been applied using increasingly sophisticated means: 1) assuming additional LYs are in perfect health; 2) applying the QOL norm of the general population by age and sex; 3) applying disease-specific QOL values to the diseases under study; and 4) also inferring the effect of health expenditure on QOL, not just mortality.
      Table 4 summarizes the characteristics and estimates of the studies using this approach. Some studies have used time-series analysis, reporting results on the cost per LY between €11,000 and €13,000. The most recent study that used cross-sectional disease-specific data found a central estimate of €14,141 in England [
      • Claxton K.
      • Martin S.
      • Soares M.
      • et al.
      Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold.
      ]. These values are at the low side of most of the estimates provided by studies focusing on the demand side.
      Table 4Characteristics and findings of studies estimating the impact of health expenditure on health outcomes
      StudyRegression methodQALY adjustmentDisease areaReported thresholdThreshold €2014
      Lichtengber,
      • Lichtenberg FR.
      Sources of U.S. longevity increase, 1960–2001.
      , 2004 United States
      Time series 1960–1997NATotal expenditure$11,00010,390
      Puig-Junoy,
      • Puig-Junoy J
      • Merino-Castelló A.
      Productividad marginal del gasto e innovaciones sanitarias. Resultados empíricos y lecciones para España.
      , 2004 Spain
      Time series 1960–2001NATotal expenditure€9,329–€11,07611,341–13,465
      Martin,
      • Martin S
      • Rice N
      • Smith PC.
      Does health care spending improve health outcomes? Evidence from English programme budgeting data.
      , 2008 England
      Instrumental variableUtilities scores by ICD-10 codes from HODaR projectCancer£19,07024,549
      Circulatory diseases£11,96015,395
      Martin,
      • Martin S
      • Rice N
      • Smith P.
      Comparing costs and outcomes across programmes of health care.
      , 2011 England
      Instrumental variableUtilities scores by ICD-10 codes from HODaR projectCancer£21,02127,048
      Cardiovascular£12,59316,210
      Respiratory£13,25613,256
      Gastrointestinal£30,40039,133
      Diabetes£47,06960,591
      Claxton,
      • Claxton K.
      • Martin S.
      • Soares M.
      • et al.
      Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold.
      , 2015
      Instrumental variableUtilities scores by ICD-10 codes from HODaR project and from the Medical Expenditure Panel23 program budget categories£12,93614,141
      ICD-10, International Classification of Diseases, Tenth Revision; HODaR, Health Outcomes Data Repository; NA, not applicable/available.
      The most significant caveats of these studies relate to data limitations and the endogeneity existing between expenditure and health. With respect to data availability, one main constraint is related to health outcomes. Current studies have to rely on mortality information and are unable to estimate the impact of expenditure on QOL alone. Claxton et al. [
      • Claxton K.
      • Martin S.
      • Soares M.
      • et al.
      Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold.
      ] partially address this limitation by inferring the effect on QOL as proportional to the effect estimated on mortality. Another major methodological difficulty is to account for the endogeneity expected between health spending and health outcomes in a population. These studies carried out sophisticated econometric analyses using time-series models or applying an instrumental variable approach. The former methods require the use of a long period of data and thus are not likely to reflect the current situation and precludes the estimation to be regularly updated, whereas the latter techniques necessarily rely on the validity of the instruments used.

      Conclusions

      In this study, we have tried to identify the critical issues around the estimation of the cost-effectiveness threshold, considering conceptual, methodological, and technical aspects. We are aware of previous reviews that have exclusively focused on WTP for QALY studies [
      • Ryen L.
      • Svensson M.
      The willingness to pay for a quality adjusted life year: a review of the empirical literature.
      ,
      • Nimdet K.
      • Chaiyakunapruk N.
      • Vichansavakul K.
      • Ngorsuraches S.
      A systematic review of studies eliciting willingness-to-pay per quality-adjusted life year: does it justify CE threshold?.
      ]. In this article, we have accounted for a broader body of literature, and in particular this comprehensive review allowed us to compare demand-side and supply-side estimates of the cost-effectiveness threshold. Nevertheless, we acknowledge that some of the methodologies we described have been the focus of extensive research not included in this review, such as the vast literature on the VSL or the impact of health spending on health outcomes. The aim of this article was not to review the extensive body of research in these areas, but its applications to arrive at a cost-effectiveness threshold.
      As mentioned in the Introduction, some authors have suggested that a framework incorporating both perspectives is to be preferred [
      • Baker R.
      • Chilton S.
      • Donaldson C.
      • et al.
      Searchers vs surveyors in estimating the monetary value of a QALY: resolving a nasty dilemma for NICE.
      ]. These authors, however, did not indicate how such a combined framework would be articulated, but deferred this debate to when more is known about the gap between the estimates obtained under each of these approaches.
      From this review, we are capable of drawing some patterns indicating that many of the estimates based on the value that society places on health gains tend to be at the high end of the distribution of results, whereas the estimates of the marginal cost of a QALY are found to be generally lower. This suggests that some interventions with positive social net benefits, as informed by individuals’ preferences, might not be an appropriate use of resources under current budgets constraints.
      Therefore, one may conclude that when faced with allocation decisions that imply health services displacements, the threshold ought to represent the opportunity costs of that investment. The supply perspective would in this situation better capture the decision-making context and provide the appropriate information to assess gains and losses. Nevertheless, although fixed budgets and displacements are likely to define many decision-making situations, this is not necessarily the case in every instance. Budgets might not be fixed, especially when new tax revenue becomes available for the health care system. Decision makers would then need to decide on how to allocate additional funds that become available with no service displacement involved. Society’s consumption value of health gains would arguably better reflect the strengths of preferences of those members of society affected by these decisions. Any given system will face a mixture of such situations and thus information on both society’s valuation of health gains and the opportunity cost of health investment is likely to be required.
      Although we might be close to resolving the conceptual “dilemma,” significant challenges still remain to capture both society’s valuation of health improvements and the opportunity cost of health care funding decisions. Most work has emphasized the level of uncertainty around the estimates provided, which suggest that efforts should be placed in characterizing such uncertainty, with recent attempts in both supply-side [
      • Claxton K.
      • Martin S.
      • Soares M.
      • et al.
      Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold.
      ] and demand-side [
      • Bobinac A
      • van Exel J
      • Rutten FFH
      • Brouwer WBF.
      The value of a QALY: individual willingness to pay for health gains under risk.
      ] studies. Promising new approaches also include the use of cross-country information [

      Ochalek J, Lomas J, Claxton K. Cost Per DALY averted thresholds for low-and middle-income countries: evidence from cross country data. 2015. Available from: http://www.idsihealth.org/wp-content/uploads/2016/01/CHERP122_cost_DALY_LMIC_threshold.pdf. [Accessed February 2, 2016].

      ]. Finally, as highlighted by this review, careful consideration should be taken to appropriately use the information provided by the societal value of health gains and the opportunity cost of health investments to guide the right decisions.

      Acknowledgment

      Laura Vallejo-Torres acknowledges the Improving Biomedical Research in the Canary Islands (IMBRAIN) project (FP7-REGPOT-2012-CT2012-31637-IMBRAIN), funded under the 7th Framework Programme (Capacities).
      Source of financial support: This work was undertaken in the framework of the activities run by the Network of Health Technology Assessment Agencies funded by the Ministry of Health, Social Services and Equality. The work was funded under the collaboration agreement between Carlos III Health Institute, an autonomous organization of the Ministry of Economics and Competitiveness, and the Canary Islands Foundation of Research and Health (FUNCIS).

      Supplementary Materials

      References

      1. National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013. Available from: https://www.nice.org.uk/article/pmg9/resources/non-guidance-guide-to-the-methods-of-technology-appraisal-2013-pdf. [Accessed January 11, 2015].

        • Kaplan R.M.
        • Bush J.W.
        Health-related quality of life measurement for evaluation research and policy analysis.
        Health Psychol. 1982; 1: 61-80
        • Laupacis A.
        • Feeny D.
        • Detsky A.S.
        • Tugwell P.X.
        How attractive does a new technology have to be to warrant adoption and utilization? Tentative guidelines for using clinical and economic evaluations.
        CMAJ. 1992; 146: 473-481
        • Bridges J.F.P.
        • Onukwugha E.
        • Mullins C.D.
        Healthcare rationing by proxy: cost-effectiveness analysis and the misuse of the $50,000 threshold in the US.
        Pharmacoeconomics. 2010; 28: 175-184
        • Neumann P.J.
        • Cohen J.T.
        • Weinstein M.C.
        Updating cost-effectiveness--the curious resilience of the $50,000-per-QALY threshold.
        N Engl J Med. 2014; 371: 796-797
      2. Socialstyrelsen. Nationella riktlinjer för prostatacancersjukvård—Medicinskt och hälsoekonomiskt faktadokument. Available from: Disponible en: http://www.wolterskluwer.se/produkt/9789185483075. [Accessed January 15, 2015].

        • Ryen L.
        • Svensson M.
        The willingness to pay for a quality adjusted life year: a review of the empirical literature.
        Health Econ. 2015; 24: 1289-1301
        • Bobinac A.
        • Van Exel N.J.A.
        • Rutten F.F.H.
        • Brouwer W.B.F.
        Willingness to pay for a quality-adjusted life-year: the individual perspective.
        Value Health. 2010; 13: 1046-1055
      3. World Health Organization. Cost effectiveness and strategic planning (WHO-CHOICE). 2005. Available from: http://www.who.int/choice/costs/CER_levels/en/. [Accessed December 4, 2014].

        • Birch S.
        • Gafni A.
        Changing the problem to fit the solution: Johannesson and Weinstein’s (mis) application of economics to real world problems.
        J Health Econ. 1993; 12: 469-476
        • Birch S.
        • Gafni A.
        The biggest bang for the buck or bigger bucks for the bang: the fallacy of the cost-effectiveness threshold.
        J Health Serv Res Policy. 2006; 11: 46-51
        • Gafni A.
        • Birch S.
        Incremental cost-effectiveness ratios (ICERs): the silence of the lambda.
        Soc Sci Med. 2006; 62: 2091-2100
        • Mccabe C.
        • Claxton K.
        • Culyer A.J.
        Cost-effectiveness threshold: what it is and what that means.
        Pharmacoeconomics. 2008; 26: 733-744
        • Eichler H.-G.
        • Kong S.X.
        • Gerth W.C.
        • et al.
        Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge?.
        Value Health. 2004; 7: 518-528
        • Phelps C.E.
        • Mushlin A.I.
        On the (near) equivalence of cost-effectiveness and cost-benefit analyses.
        Int J Technol Assess Health Care. 1991; 7: 12-21
        • Baker R.
        • Chilton S.
        • Donaldson C.
        • et al.
        Searchers vs surveyors in estimating the monetary value of a QALY: resolving a nasty dilemma for NICE.
        Health Econ Policy Law. 2011; 6: 435-447
        • Claxton K.
        • Martin S.
        • Soares M.
        • et al.
        Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold.
        Health Technol Assess. 2015; 19: 1-504
        • Culyer A.
        • McCabe C.
        • Briggs A.
        • et al.
        Searching for a threshold, not setting one: the role of the National Institute for Health and Clinical Excellence.
        J Health Serv Res Policy. 2007; 12: 56-58
        • Mason H.
        • Jones-Lee M.
        • Donaldson C.
        Modelling the monetary value of a QALY: a new approach based on UK data.
        Health Econ. 2009; 18: 933-950
      4. Donaldson C, Baker R, Mason H, et al. European value of a quality adjusted life year. 2010. Available from: http://research.ncl.ac.uk/eurovaq/EuroVaQ_Final_Publishable_Report_and_Appendices.pdf. [Accessed November 15, 2014].

        • Blumenschein K.
        • Johannesson M.
        Relationship between quality of life instruments, health state utilities, and willingness to pay in patients with asthma.
        Ann Allergy Asthma Immunol. 1998; 80: 189-194
        • Zethraeus N.
        Willingness to pay for hormone replacement therapy.
        Health Econ. 1998; 7: 31-38
        • Cunningham S.J.
        • Hunt N.
        Relationship between utility values and willingness to pay in patients undergoing orthognathic treatment.
        Community Dent Health. 2000; 17: 92-96
        • Gyrd-Hansen D.
        Willingness to pay for a QALY.
        Health Econ. 2003; 12: 1049-1060
        • Byrne M.M.
        • O’malley K.
        • Suarez-Almazor M.E.
        Willingness to pay per quality-adjusted life year in a study of knee osteoarthritis.
        Med Decis Making. 2005; 25: 655-666
        • King J.T.
        • Tsevat J.
        • Lave J.R.
        • Roberts M.S.
        Willingness to pay for a quality-adjusted life year: implications for societal health care resource allocation.
        Med Decis Making. 2005; 25: 667-677
        • Pinto Prades J.L.
        • Martinez Perez J.
        Estimacion del valor monetario de los anos de vida ajustados por calidad: Estimaciones preliminares.
        Ekonomiaz. 2005; I: 192-209
        • Lieu Ta
        • Ray G.T.
        • Ortega-Sanchez I.R.
        • Kleinman K.
        • Rusinak D.
        • Prosser La
        Willingness to pay for a QALY based on community member and patient preferences for temporary health states associated with herpes zoster.
        Pharmacoeconomics. 2009; 27: 1005-1016
        • Pinto-Prades J.L.
        • Loomes G.
        • Brey R.
        Trying to estimate a monetary value for the QALY.
        J Health Econ. 2009; 28: 553-562
        • Shiroiwa T
        • Sung Y
        • Fukuda T
        • Lang H
        • Bae S-C
        • Tsutani K.
        International survey on willingness-to-pay (WTP) for one additional QALY gained: what is the threshold of cost effectiveness?.
        Health Econ. 2010; 19: 422-437
        • Baker R
        • Bateman I
        • Donaldson C
        • et al.
        Weighting and valuing quality-adjusted life-years using stated preference methods: preliminary results from the Social Value of a QALY Project.
        Health Technol Assess. 2010; 14: 1-162
        • Donaldson C
        • Baker R
        • Mason H
        • et al.
        The social value of a QALY: raising the bar or barring the raise?.
        BMC Health Serv Res. 2011; 11: 8
        • Zhao F
        • Yue M
        • Yang H
        • Wang T
        • Wu J-H
        • Li S.
        Willingness to pay per quality-adjusted life year: is one threshold enough for decision-making?: results from a study in patients with chronic prostatitis.
        Med Care. 2011; 49: 267-272
        • Soini E.
        Contingent Valuation of Eight New Treatments: What is the Clinician’s and Politician’s Willingness to Pay?.
        Open Complement Med J. 2012; 4: 1-11
        • Shiroiwa T
        • Igarashi A
        • Fukuda T
        • Ikeda S.
        WTP for a QALY and health states: More money for severer health states?.
        Cost Eff Resour Alloc. 2013; 11: 22
        • Bobinac A
        • van Exel NJA
        • Rutten FFH
        • Brouwer WBF
        Valuing qaly gains by applying a societal perspective.
        Health Econ. 2013; 22: 1272-1281
        • Thavorncharoensap M
        • Teerawattananon Y
        • Natanant S
        • Kulpeng W
        • Yothasamut J
        • Werayingyong P.
        Estimating the willingness to pay for a quality-adjusted life year in Thailand: does the context of health gain matter?.
        Clinicoecon Outcomes Res. 2013; 5: 29-36
        • Robinson A
        • Gyrd-Hansen D
        • Bacon P
        • et al.
        Estimatinga WTP-based value of a QALY: the “chained” approach.
        Soc Sci Med. 2013; 92: 92-104
        • Pennington M
        • Baker R
        • Brouwer W
        • et al.
        Comparing WTP values of different types of QALY gain elicited from the general public.
        Health Econ. 2013; 24: 280-293
        • Bobinac A
        • van Exel J
        • Rutten FFH
        • Brouwer WBF.
        The value of a QALY: individual willingness to pay for health gains under risk.
        Pharmacoeconomics. 2014; 32: 75-86
        • Martín-Fernández J
        • Polentinos-Castro E
        • del Cura-González MI
        • et al.
        Willingness to pay for a quality-adjusted life year: an evaluation of attitudes towards risk and preferences.
        BMC Health Serv Res. 2014; 14: 287
        • Bobinac A.
        • van Exel N.J.A.
        • Rutten F.F.H.
        • Brouwer W.B.F.
        Get more, pay more? An elaborate test of construct validity of willingness to pay per QALY estimates obtained through contingent valuation.
        J Health Econ. 2012; 31: 158-168
        • Gyrd-Hansen D.
        • Kjaer T.
        Disentangling WTP per QALY data: different analytical approaches, different answers.
        Health Econ. 2012; 21: 222-237
        • Haninger K.
        • Hammitt J.K.
        Diminishing willingness to pay per quality-adjusted life year: valuing acute foodborne illness.
        Risk Anal. 2011; 31: 1363-1380
        • Gold M.R.
        • Siegel J.E.
        • Russell L.B.
        • Weinstein M.C.
        Cost-effectiveness in health and medicine.
        BMJ. 1997; 315 (689–689)
        • Hirth R.
        • Chernew M.E.
        • Miller E.
        • Fendrick a.M
        • Weissert W.G.
        Willingness to Pay for a Quality-adjusted Life Year: In Search of a Standard.
        Med Decis Mak. 2000; 20: 332-342
      5. Abellán Perpiñán JM, Martínez Pérez JE, Méndez Martínez I, Sánchez Martinez FI, Pinto-Prades JL, Robles Zurita JA. El Valor Monetario de Una Víctima No Mortal Y Del Año de Vida Ajustado Por Calidad En España. Dirección General de Tráfico. 2011. Available from http://www.dgt.es/Galerias/seguridad-vial/investigacion/estudios-e-informes/2011/SPAD1A_-.-ESTIMACION-EN-EL-CONTEXTO-DE-LOS-ACCIDENTES-DE-TRAFICO_INFORME-PARA-WEB.pdf. [Accessed January 15, 2015].

        • Towse A.
        Should NICE’s threshold range for cost per QALY be raised? Yes.
        BMJ. 2009; 338: b181
        • Appleby J.
        • Devlin N.
        • Parkin D.
        • et al.
        Searching for cost effectiveness thresholds in the NHS.
        Health Policy. 2009; 91: 239-245
        • George B.
        • Harris A.
        • Mitchell A.
        Cost-effectiveness analysis and the consistency of decision making: evidence from pharmaceutical reimbursement in Australia (1991 to 1996).
        Pharmacoeconomics. 2001; 19: 1103-1109
        • Devlin N.
        • Parkin D.
        Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis.
        Health Econ. 2004; 13: 437-452
        • Chambers J.D.
        • Neumann P.J.
        • Buxton M.J.
        Does Medicare have an implicit cost-effectiveness threshold?.
        Med Decis Making. 2010; 30: E14-E27
        • Nimdet K.
        • Chaiyakunapruk N.
        • Vichansavakul K.
        • Ngorsuraches S.
        A systematic review of studies eliciting willingness-to-pay per quality-adjusted life year: does it justify CE threshold?.
        PLoS One. 2015; 10: e0122760
        • Lichtenberg FR.
        Sources of U.S. longevity increase, 1960–2001.
        Q Rev Econ Financ. 2004; 44: 369-389
        • Puig-Junoy J
        • Merino-Castelló A.
        Productividad marginal del gasto e innovaciones sanitarias. Resultados empíricos y lecciones para España.
        in: Masson ¿Más Recursos Para La Salud?2004: 133-156
        • Martin S
        • Rice N
        • Smith PC.
        Does health care spending improve health outcomes? Evidence from English programme budgeting data.
        J Health Econ. 2008; 27: 826-842
        • Martin S
        • Rice N
        • Smith P.
        Comparing costs and outcomes across programmes of health care.
        Health Econ. 2012; 21: 316-337
      6. Ochalek J, Lomas J, Claxton K. Cost Per DALY averted thresholds for low-and middle-income countries: evidence from cross country data. 2015. Available from: http://www.idsihealth.org/wp-content/uploads/2016/01/CHERP122_cost_DALY_LMIC_threshold.pdf. [Accessed February 2, 2016].