Accounting for Heterogeneity in Resource Allocation Decisions: Methods and Practice in UK Cancer Technology Appraisals

Published:April 27, 2021DOI:


      • The availability of novel, more efficacious cancer therapies is increasing, resulting in significant treatment effect heterogeneity and complicated treatment and disease pathways. Technology appraisals (TAs) evaluate clinical and economic evidence to inform reimbursement decisions and resource allocation. Through critical appraisal of UK cancer TAs, we identify areas where considerations of heterogeneity can be improved. We focus on 3 cancer sites: colorectal, lung, and ovarian cancer, encompassing variation in screening, diagnostic and treatments pathways.
      • All TAs in this review used decision analytic modeling. The majority used partitioned survival models and evaluated aggregate outcomes of clinical trial populations. Only 2 models explicitly considered real-world patient heterogeneity in disease progression estimates. Moreover, predetermined subgroup analyses contained within the clinical studies that informed the TAs were rarely exploited in economic analyses.
      • This review highlights a paucity of information relating to the assessment of heterogeneity in colorectal, lung, and ovarian cancer TAs. We conclude that future cancer TAs should consider more flexible modeling approaches and apply real-world data to explore heterogeneity within their economic analyses, especially if the complexity of treatment and disease pathways is to be reflected.



      The availability of novel, more efficacious and expensive cancer therapies is increasing, resulting in significant treatment effect heterogeneity and complicated treatment and disease pathways. The aim of this study is to review the extent to which UK cancer technology appraisals (TAs) consider the impact of patient and treatment effect heterogeneity.


      A systematic search of National Institute for Health and Care Excellence TAs of colorectal, lung and ovarian cancer was undertaken for the period up to April 2020. For each TA, the pivotal clinical studies and economic evaluations were reviewed for considerations of patient and treatment effect heterogeneity. The study critically reviews the use of subgroup analysis and real-world translation in economic evaluations, alongside specific attributes of the economic modeling framework.


      The search identified 49 TAs including 49 economic models. In total, 804 subgroup analyses were reported across 69 clinical studies. The most common stratification factors were age, gender, and Eastern Cooperative Oncology Group performance score, with 15% (119 of 804) of analyses demonstrating significantly different clinical outcomes to the main population; economic subgroup analyses were undertaken in only 17 TAs. All economic models were cohort-level with the majority described as partitioned survival models (39) or Markov/semi-Markov models. The impact of real-world heterogeneity on disease progression estimates was only explored in 2 models.


      The ability of current modeling approaches to capture patient and treatment effect heterogeneity is constrained by their limited flexibility and simplistic nature. This study highlights a need for the use of more sophisticated modeling methods that enable greater consideration of real-world heterogeneity.


      To read this article in full you will need to make a payment
      Subscribe to Value in Health
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. Public Health England. Health profile for England: 2019 In: UK government; 2019.

        • Smittenaar C.R.
        • Petersen K.A.
        • Stewart K.
        • Moitt N.
        Cancer incidence and mortality projections in the UK until 2035.
        Br J Cancer. 2016; 115: 1147-1155
        • Cancer Research UK
        Cancer incidence statistics.
        • Hawkes N.
        Cancer survival data emphasise importance of early diagnosis.
        BMJ. 2019; 364: l408
        • Maddams J.
        • Utley M.
        • Moller H.
        Projections of cancer prevalence in the United Kingdom, 2010-2040.
        Br J Cancer. 2012; 107: 1195-1202
        • Macmillan cancer support
        Statistics fact sheet.
        • Pfizer
        Cancer costs: a ripple effect analysis of cancer’s wider impact.
        • Arnold M.
        • Rutherford M.J.
        • Bardot A.
        • et al.
        Progress in cancer survival, mortality, and incidence in seven high-income countries 1995-2014 (ICBP SURVMARK-2): a population-based study.
        Lancet Oncol. 2019; 20: 1493-1505
        • Allemani C.
        • Matsuda T.
        • Di Carlo V.
        • et al.
        Global surveillance of trends in cancer survival 2000-14 (CONCORD-3): analysis of individual records for 37 513 025 patients diagnosed with one of 18 cancers from 322 population-based registries in 71 countries.
        Lancet. 2018; 391: 1023-1075
        • Falzone L.
        • Salomone S.
        • Libra M.
        Evolution of cancer pharmacological treatments at the turn of the third millennium.
        Front Pharmacol. 2018; 9: 1300
        • National Institute for Health and Care Excellence
        Breast screening.
        (Published 2017. Accessed October 24, 2019)
        • National Institute for Health and Care Excellence
        Bowel screening.
        (Published 2019. Accessed October 24, 2019)
        • National Institute for Health and Care Excellence
        Cervical screening.
        (Published 2017. Accessed October 24, 2019)
      2. NHS to rollout lung cancer scanning trucks across the country [press release]. 2019
        • National Health Service
        The NHS Long Term Plan.
        • Varadhan R.
        • SJ
        Estimation and reporting of heterogeneity of treatment effects.
        in: Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. Agency for Healthcare Research and Quality (US), Rockville (MD)2013
        • Yuan M.
        • Huang L.-L.
        • Chen J.-H.
        • Wu J.
        • Xu Q.
        The emerging treatment landscape of targeted therapy in non-small-cell lung cancer.
        Signal Transduction and Targeted Therapy. 2019; 4: 61
        • Xin Yu J.
        • Hubbard-Lucey V.M.
        • Tang J.
        The global pipeline of cell therapies for cancer.
        Nat Rev Drug Discov. 2019; 18: 821-822
        • Sculpher M.
        Subgroups and heterogeneity in cost-effectiveness analysis.
        Pharmacoeconomics. 2008; 26: 799-806
        • Grutters J.P.
        • Sculpher M.
        • Briggs A.H.
        • et al.
        Acknowledging patient heterogeneity in economic evaluation: a systematic literature review.
        Pharmacoeconomics. 2013; 31: 111-123
        • Ramaekers B.L.T.
        • Joore M.A.
        • Grutters J.P.C.
        How should we deal with patient heterogeneity in economic evaluation: a systematic review of national pharmacoeconomic guidelines.
        Value Health. 2013; 16: 855-862
        • Cookson R.
        • Propper C.
        • Asaria M.
        • Raine R.
        Socio-economic inequalities in health care in England.
        Fiscal Studies. 2016; 37: 371-403
        • Foster H.M.E.
        • Celis-Morales C.A.
        • Nicholl B.I.
        • et al.
        The effect of socioeconomic deprivation on the association between an extended measurement of unhealthy lifestyle factors and health outcomes: a prospective analysis of the UK Biobank cohort.
        The Lancet Public Health. 2018; 3: e576-e585
      3. Macmillan cancer support.
        Health Inequalities: Time to Talk. 2019
        • Fowler H.
        • Belot A.
        • Ellis L.
        • et al.
        Comorbidity prevalence among cancer patients: a population-based cohort study of four cancers.
        BMC Cancer. 2020; 20: 2
      4. Public Health England. Health profile for England: 2018. In: UK government; 2018.

        • Exarchakou A.
        • Rachet B.
        • Belot A.
        • Maringe C.
        • Coleman M.P.
        Impact of national cancer policies on cancer survival trends and socioeconomic inequalities in England, 1996-2013: population based study.
        BMJ. 2018; 360: k764
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6e1000097
        • Espinoza M.A.
        • Manca A.
        • Claxton K.
        • Sculpher M.J.
        The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application.
        Med Decis Making. 2014; 34: 951-964
        • Brennan A.
        • Chick S.E.
        • Davies R.
        A taxonomy of model structures for economic evaluation of health technologies.
        Health Econ. 2006; 15: 1295-1310
        • Briggs A.D.M.
        • Wolstenholme J.
        • Blakely T.
        • Scarborough P.
        Choosing an epidemiological model structure for the economic evaluation of non-communicable disease public health interventions.
        Popul Health Metr. 2016; 14 (17-17)
        • Bullement A.
        • Cranmer H.L.
        • Shields G.E.
        A review of recent decision-analytic models used to evaluate the economic value of cancer treatments.
        Appl Health Econ Health Policy. 2019; 17: 771-780
        • Sonnenberg F.A.
        • Beck J.R.
        Markov models in medical decision making: a practical guide.
        Medical Decision Making. 1993; 13: 322-338
        • Abner E.L.
        • Charnigo R.J.
        • Kryscio R.J.
        Markov chains and semi-Markov models in time-to-event analysis.
        J Biom Biostat. 2013; Suppl 1: 19522
        • Geifman N.
        • Butte A.J.
        Do cancer clinical trial populations truly represent cancer patients? A comparison of open clinical trials to the cancer genome atlas.
        Pac Symp Biocomput. 2016; 21: 309-320
        • Unger J.M.
        • Barlow W.E.
        • Martin D.P.
        • et al.
        Comparison of survival outcomes among cancer patients treated in and out of clinical trials.
        J Natl Cancer Inst. 2014; 106
        • Mitchell A.P.
        • Harrison M.R.
        • George D.J.
        • Abernethy A.P.
        • Walker M.S.
        • Hirsch B.R.
        Clinical trial subjects compared to “real world” patients: generalizability of renal cell carcinoma trials.
        J Clin Oncol. 2014; 32: 6510
        • Cui L.
        • Hung H.M.
        • Wang S.J.
        • Tsong Y.
        Issues related to subgroup analysis in clinical trials.
        J Biopharm Stat. 2002; 12: 347-358
        • Cook D.I.
        • Gebski V.J.
        • Keech A.C.
        Subgroup analysis in clinical trials.
        Med J Aust. 2004; 180: 289-291
        • Grouin J.M.
        • Coste M.
        • Lewis J.
        Subgroup analyses in randomized clinical trials: statistical and regulatory issues.
        J Biopharm Stat. 2005; 15: 869-882
        • NICE NIfHaCE
        Nice DSU technical support document 19: partitioned survival analysis for decision modeling in health care: a critical review.
        Date accessed: February 6, 2017
        • Davis S.
        • Stevenson M.
        • Tappenden P.
        • Wailoo A.
        NICE decision support unit technical support documents.
        in: NICE DSU Technical Support Document 15: Cost-Effectiveness Modeling Using Patient-Level Simulation. National Institute for Health and Care Excellence (NICE), London2014
        • Cichosz S.L.
        • Johansen M.D.
        • Hejlesen O.
        Toward big data analytics: review of predictive models in management of diabetes and its complications.
        J Diabetes Sci Technol. 2016; 10: 27-34
        • Collins G.S.
        • Mallett S.
        • Omar O.
        • Yu L.-M.
        Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting.
        BMC Medicine. 2011; 9: 103
        • Noble D.
        • Mathur R.
        • Dent T.
        • Meads C.
        • Greenhalgh T.
        Risk models and scores for type 2 diabetes: systematic review.
        BMJ. 2011; 343: d7163
        • Tangri N.
        • Kitsios G.D.
        • Inker L.A.
        • et al.
        Risk prediction models for patients with chronic kidney disease: a systematic review.
        Ann Intern Med. 2013; 158: 596-603
        • Ramspek C.L.
        • de Jong Y.
        • Dekker F.W.
        • van Diepen M.
        Towards the best kidney failure prediction tool: a systematic review and selection aid.
        Nephrol Dial Transplant. 2020; 35: 1527-1538
        • Di Tanna G.L.
        • Wirtz H.
        • Burrows K.L.
        • Globe G.
        Evaluating risk prediction models for adults with heart failure: a systematic literature review.
        PLoS One. 2020; 15e0224135
        • Damen J.A.
        • Hooft L.
        • Schuit E.
        • et al.
        Prediction models for cardiovascular disease risk in the general population: systematic review.
        BMJ. 2016; 353: i2416
        • Muller P.
        • Walters S.
        • Coleman M.P.
        • Woods L.
        Which indicators of early cancer diagnosis from population-based data sources are associated with short-term mortality and survival?.
        Cancer Epidemiol. 2018; 56: 161-170
        • Kolovos S.
        • Nador G.
        • Kishore B.
        • et al.
        Unplanned admissions for patients with myeloma in the UK: low frequency but high costs.
        J Bone Oncol. 2019; 17: 100243
        • Laudicella M.
        • Walsh B.
        • Burns E.
        • Smith P.C.
        Cost of care for cancer patients in England: evidence from population-based patient-level data.
        Br J Cancer. 2016; 114: 1286-1292
        • McConnell H.
        • White R.
        • Maher J.
        Categorising cancers to enable tailored care planning through a secondary analysis of cancer registration data in the UK.
        BMJ Open. 2017; 7e016797
        • Ward S.E.
        • Holmes G.R.
        • Ring A.
        • et al.
        Adjuvant chemotherapy for breast cancer in older women: an analysis of retrospective English cancer registration data.
        Clin Oncol. 2019; 31: 444-452
        • Nguyen A.
        • Yoshida M.
        • Goodarzi H.
        • Tavazoie S.F.
        Highly variable cancer subpopulations that exhibit enhanced transcriptome variability and metastatic fitness.
        Nature Communications. 2016; 7: 11246
        • Clinical Practice Research Datalink
        Clinical practice research datalink dataset.
        Date accessed: September 28, 2020
        • NHS Digital
        Hospital episode statistics (HES).
        Date accessed: September 28, 2020
        • Public Health England
        Cancer registry dataset.
        Date accessed: September 28, 2020
        • UK Biobank
        Biobank Dataset.
        Date accessed: September 28, 2020
        • D'Agostino R.B.
        • Kwan H.
        Measuring effectiveness. What to expect without a randomized control group.
        Med Care. 1995; 33: As95-As105
        • Lakdawalla D.N.
        • Shafrin J.
        • Hou N.
        • et al.
        Predicting real-world effectiveness of cancer therapies using overall survival and progression-free survival from clinical trials: empirical evidence for the ASCO value framework.
        Value Health. 2017; 20: 866-875
        • Signorovitch J.E.
        • Wu E.Q.
        • Yu A.P.
        • et al.
        Comparative effectiveness without head-to-head trials.
        PharmacoEconomics. 2010; 28: 935-945
        • Caro J.J.
        • Ishak K.J.
        No Head-to-head trial? simulate the missing arms.
        PharmacoEconomics. 2010; 28: 957-967