Advertisement

Accounting for Heterogeneity in Resource Allocation Decisions: Methods and Practice in UK Cancer Technology Appraisals

Open ArchivePublished:April 27, 2021DOI:https://doi.org/10.1016/j.jval.2020.12.022

      Highlights

      • The availability of novel, more efficacious cancer therapies is increasing, resulting in significant treatment effect heterogeneity and complicated treatment and disease pathways. Technology appraisals (TAs) evaluate clinical and economic evidence to inform reimbursement decisions and resource allocation. Through critical appraisal of UK cancer TAs, we identify areas where considerations of heterogeneity can be improved. We focus on 3 cancer sites: colorectal, lung, and ovarian cancer, encompassing variation in screening, diagnostic and treatments pathways.
      • All TAs in this review used decision analytic modeling. The majority used partitioned survival models and evaluated aggregate outcomes of clinical trial populations. Only 2 models explicitly considered real-world patient heterogeneity in disease progression estimates. Moreover, predetermined subgroup analyses contained within the clinical studies that informed the TAs were rarely exploited in economic analyses.
      • This review highlights a paucity of information relating to the assessment of heterogeneity in colorectal, lung, and ovarian cancer TAs. We conclude that future cancer TAs should consider more flexible modeling approaches and apply real-world data to explore heterogeneity within their economic analyses, especially if the complexity of treatment and disease pathways is to be reflected.

      Abstract

      Objectives

      The availability of novel, more efficacious and expensive cancer therapies is increasing, resulting in significant treatment effect heterogeneity and complicated treatment and disease pathways. The aim of this study is to review the extent to which UK cancer technology appraisals (TAs) consider the impact of patient and treatment effect heterogeneity.

      Methods

      A systematic search of National Institute for Health and Care Excellence TAs of colorectal, lung and ovarian cancer was undertaken for the period up to April 2020. For each TA, the pivotal clinical studies and economic evaluations were reviewed for considerations of patient and treatment effect heterogeneity. The study critically reviews the use of subgroup analysis and real-world translation in economic evaluations, alongside specific attributes of the economic modeling framework.

      Results

      The search identified 49 TAs including 49 economic models. In total, 804 subgroup analyses were reported across 69 clinical studies. The most common stratification factors were age, gender, and Eastern Cooperative Oncology Group performance score, with 15% (119 of 804) of analyses demonstrating significantly different clinical outcomes to the main population; economic subgroup analyses were undertaken in only 17 TAs. All economic models were cohort-level with the majority described as partitioned survival models (39) or Markov/semi-Markov models. The impact of real-world heterogeneity on disease progression estimates was only explored in 2 models.

      Conclusion

      The ability of current modeling approaches to capture patient and treatment effect heterogeneity is constrained by their limited flexibility and simplistic nature. This study highlights a need for the use of more sophisticated modeling methods that enable greater consideration of real-world heterogeneity.

      Keywords

      Introduction

      Cancer represents a significant healthcare burden in the United Kingdom, being the leading cause of morbidity and mortality.

      Public Health England. Health profile for England: 2019 In: UK government; 2019.

      Between 2015 and 2017, an estimated 2.5 million people were living with cancer in the United Kingdom, with an estimated annual incidence of 367 000 and, despite general improvements in population health, incidence and prevalence are predicted to increase.
      • Smittenaar C.R.
      • Petersen K.A.
      • Stewart K.
      • Moitt N.
      Cancer incidence and mortality projections in the UK until 2035.
      Cancer Research UK
      Cancer incidence statistics.
      • Hawkes N.
      Cancer survival data emphasise importance of early diagnosis.
      • Maddams J.
      • Utley M.
      • Moller H.
      Projections of cancer prevalence in the United Kingdom, 2010-2040.
      Macmillan cancer support
      Statistics fact sheet.
      Consequently, the economic burden of cancer is high and is estimated to account for 5% of total UK medical expenditure.
      Pfizer
      Cancer costs: a ripple effect analysis of cancer’s wider impact.
      Nevertheless, while the United Kingdom falls behind other high-income countries, in recent years there has been improvement in mortality rates across most cancers, driven by an ever-evolving therapeutic landscape and earlier diagnoses.
      • Arnold M.
      • Rutherford M.J.
      • Bardot A.
      • et al.
      Progress in cancer survival, mortality, and incidence in seven high-income countries 1995-2014 (ICBP SURVMARK-2): a population-based study.
      • Allemani C.
      • Matsuda T.
      • Di Carlo V.
      • et al.
      Global surveillance of trends in cancer survival 2000-14 (CONCORD-3): analysis of individual records for 37 513 025 patients diagnosed with one of 18 cancers from 322 population-based registries in 71 countries.
      • Falzone L.
      • Salomone S.
      • Libra M.
      Evolution of cancer pharmacological treatments at the turn of the third millennium.
      The introduction of several nationwide screening policies, the emergence of targeted therapies and an increasing focus on personalized care have all contributed to such improvements.
      National Institute for Health and Care Excellence
      Breast screening.
      National Institute for Health and Care Excellence
      Bowel screening.
      National Institute for Health and Care Excellence
      Cervical screening.
      National Health Service
      The NHS Long Term Plan.
      These changes have ushered in the potential for significant treatment outcome variability, compounded by inherent increases in patient and treatment effect heterogeneity.
      Patient heterogeneity typically refers to the variability of particular characteristics (eg, age, sex, etc) among patients in a given population, while treatment effect heterogeneity refers to the nonrandom, explainable variability in the direction and magnitude of treatment effects for individuals within a population.
      • Varadhan R.
      • SJ
      Estimation and reporting of heterogeneity of treatment effects.
      Treatment effect heterogeneity can be measured in relative or absolute terms and patient heterogeneity may conventionally be represented by variation in outcomes under the status quo, while treatment effect heterogeneity would be operationalized as the variation in the difference in outcomes between the new treatment and the status quo.
      Measures of patient and treatment effect heterogeneity seem particularly applicable to a disease area such as cancer, where the treatment landscape is rapidly evolving, and the availability of novel and more efficacious therapies is increasing. This is even more relevant when considering that newer cancer therapies are often targeted to specific patient groups, such as those with particular gene mutations or treatment and clinical histories. These targeted treatment recommendations arise from the significant patient and treatment effect heterogeneity observed among patients with cancer, naturally resulting in complicated treatment and disease pathways.
      • Yuan M.
      • Huang L.-L.
      • Chen J.-H.
      • Wu J.
      • Xu Q.
      The emerging treatment landscape of targeted therapy in non-small-cell lung cancer.
      ,
      • Xin Yu J.
      • Hubbard-Lucey V.M.
      • Tang J.
      The global pipeline of cell therapies for cancer.
      However, there remains a lack of formal guidance on how to incorporate such effects into economic evaluations.
      • Sculpher M.
      Subgroups and heterogeneity in cost-effectiveness analysis.
      • Grutters J.P.
      • Sculpher M.
      • Briggs A.H.
      • et al.
      Acknowledging patient heterogeneity in economic evaluation: a systematic literature review.
      • Ramaekers B.L.T.
      • Joore M.A.
      • Grutters J.P.C.
      How should we deal with patient heterogeneity in economic evaluation: a systematic review of national pharmacoeconomic guidelines.
      Indeed, reimbursement decisions are typically made based on average population-level results of clinical and economic evaluations, which potentially conceal important sources of outcome variability, particularly within large clinically heterogeneous populations.
      Linked to these issues are the growing concerns related to inequalities across socioeconomic groups, particularly with respect to cancer survival.
      • Cookson R.
      • Propper C.
      • Asaria M.
      • Raine R.
      Socio-economic inequalities in health care in England.
      ,
      • Foster H.M.E.
      • Celis-Morales C.A.
      • Nicholl B.I.
      • et al.
      The effect of socioeconomic deprivation on the association between an extended measurement of unhealthy lifestyle factors and health outcomes: a prospective analysis of the UK Biobank cohort.
      People in the most income-deprived areas in England are more likely to have their cancer diagnosed at a later stage, present with more comorbidities and observe different treatment pathways to those in less deprived areas, and perhaps as a consequence, observe lower life expectancy.
      Macmillan cancer support.
      ,
      • Fowler H.
      • Belot A.
      • Ellis L.
      • et al.
      Comorbidity prevalence among cancer patients: a population-based cohort study of four cancers.
      Furthermore, while survival rates improve there is little evidence that inequalities in cancer survival have narrowed.

      Public Health England. Health profile for England: 2018. In: UK government; 2018.

      ,
      • Exarchakou A.
      • Rachet B.
      • Belot A.
      • Maringe C.
      • Coleman M.P.
      Impact of national cancer policies on cancer survival trends and socioeconomic inequalities in England, 1996-2013: population based study.
      Knowledge about variation in patient outcomes and their association with clinical and socioeconomic characteristics would enable efficient and equitable healthcare resource allocation.
      The objective of this study is to review the extent to which UK cancer TAs consider the impact of patient and treatment effect heterogeneity and to evaluate the suitability of current modeling approaches with respect to their ability to capture such heterogeneity. Through critical appraisal, this review aims to identify areas where the consideration of patient and treatment effect heterogeneity may be improved, and to move toward recommendations on best practice for future economic evaluations.

      Methods

      A search of published National Institute for Health and Care Excellence (NICE) cancer TAs was undertaken. Focus was given to 3 cancer sites: colorectal, lung, and ovarian cancer, to encompass a range of screening, diagnostic and treatment practices. Full details of the search are provided in see Appendix 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.12.022. In brief, the review was undertaken according to best practices as described by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines.
      • Moher D.
      • Liberati A.
      • Tetzlaff J.
      • Altman D.G.
      Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
      Searches were conducted on the April 12, 2020; no date restrictions were applied. For each TA, the clinical studies describing the effectiveness of the intervention under assessment, and any associated economic analyses, were retrieved and reviewed.
      Within the context of economic evaluation specifically, this review explores the use of subgroup analyses and real-world translation, alongside specific attributes of the underlying economic modeling frameworks. Each component is critically reviewed from the perspective of their ability to incorporate patient and treatment effect heterogeneity.

      Subgroup Analysis

      Randomized clinical trials often assess the impact of treatments in specific groups of patients through predefined subgroup or stratification factor analyses. Subgroup analysis is also a common approach used to explore heterogeneity implications in cost-effectiveness analyses. Espinoza et al develops a general framework to guide the use of subgroup cost-effectiveness analysis for decision making in a collectively funded health system.
      • Espinoza M.A.
      • Manca A.
      • Claxton K.
      • Sculpher M.J.
      The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application.
      With this framework in mind, we consider to what extent TAs have included subgroup analysis in clinical and economic sections of the submission.
      Patient and treatment effect heterogeneity were initially explored by extracting data relating to the presentation of subgroup analyses in the pivotal clinical studies. Subgroup analyses undertaken within the clinical studies that presented treatment effect hazard ratios (HR) for either progression-free survival (PFS) or overall survival (OS) were recorded, alongside subgroup stratifications. In addition, we recorded the number of subgroup analyses where the HR in the subgroup population significantly differed from the HR in the intention-to-treat, or overall population. Here, a significant difference is defined by an opposing effect in each population, for example, instances where the HR is greater than one favoring the comparator in the subgroup analysis population, whilst the HR is lower than one favoring the intervention in the overall population.

      Economic Modeling

      Economic modeling in TAs is used to estimate lifetime clinical and economic outcomes associated with a particular treatment, where direct experimental or observational data are unavailable or incomplete. Modeling frameworks provide a natural environment to assess the impact of patient and treatment effect heterogeneity and their associated uncertainty. The ability of models to incorporate such aspects can be highly dependent on their structural form and the statistical analysis used to manipulate and evaluate the underlying data.
      We draw on Brennan et al and Briggs et al and describe a modified taxonomy of models (Appendix 2 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.12.022) to critically review the ability of identified models to incorporate patient and treatment effect heterogeneity.
      • Brennan A.
      • Chick S.E.
      • Davies R.
      A taxonomy of model structures for economic evaluation of health technologies.
      ,
      • Briggs A.D.M.
      • Wolstenholme J.
      • Blakely T.
      • Scarborough P.
      Choosing an epidemiological model structure for the economic evaluation of non-communicable disease public health interventions.
      Partitioned survival models (PSM) and Markov models are the most common approaches to modeling cost-effectiveness in cancer.
      • Bullement A.
      • Cranmer H.L.
      • Shields G.E.
      A review of recent decision-analytic models used to evaluate the economic value of cancer treatments.
      Both are typically cohort-level and predict outcomes based on the average patient and treatment effect in a population. A PSM follows a cohort as they move between a set of exhaustive and mutually exclusive health states, relying on the use of independent survival functions to estimate state occupancy. Similarly, a Markov model follows a cohort as they move between exhaustive and mutually exclusive health states, but relies on static, cyclic transition rates. These Markov transitions enable the cohort to move back into health states that have already been visited. However, to incorporate time-dependency of transitions through relaxation of the Markov assumption, the use of tunnel states or semi-Markov models is required.
      • Sonnenberg F.A.
      • Beck J.R.
      Markov models in medical decision making: a practical guide.
      ,
      • Abner E.L.
      • Charnigo R.J.
      • Kryscio R.J.
      Markov chains and semi-Markov models in time-to-event analysis.
      Patient-level models are an alternative to cohort-level models and estimate outcomes for each individual patient, enabling individual patient histories to be recorded and the ability to capture (first order) heterogeneity in the patient population. Patient-level models require more data than cohort models, and their ability to capture patient histories therefore comes at a cost which may or may not be necessary for solving a decision problem. Although it is suggested that patient-level models are the preferred choice for incorporating heterogeneity considerations due to their inherent flexibility, heterogeneity may be incorporated in PSMs and Markov models using extra health states to stratify patients by clinical or treatment characteristics.
      • Brennan A.
      • Chick S.E.
      • Davies R.
      A taxonomy of model structures for economic evaluation of health technologies.
      ,
      • Briggs A.D.M.
      • Wolstenholme J.
      • Blakely T.
      • Scarborough P.
      Choosing an epidemiological model structure for the economic evaluation of non-communicable disease public health interventions.
      The following model components are therefore reviewed and appraised:
      • Modeled population
      • Model type
      • Health states
      • Health state transitions and their derivation
      • Treatment pathway and its influence on outcomes

      Real-World Translation

      Trial populations often differ from those they are deemed to represent in routine clinical practice, with trial participants often being younger and healthier.
      • Geifman N.
      • Butte A.J.
      Do cancer clinical trial populations truly represent cancer patients? A comparison of open clinical trials to the cancer genome atlas.
      • Unger J.M.
      • Barlow W.E.
      • Martin D.P.
      • et al.
      Comparison of survival outcomes among cancer patients treated in and out of clinical trials.
      • Mitchell A.P.
      • Harrison M.R.
      • George D.J.
      • Abernethy A.P.
      • Walker M.S.
      • Hirsch B.R.
      Clinical trial subjects compared to “real world” patients: generalizability of renal cell carcinoma trials.
      Trials undertaken in different regions or at different times can also differ significantly with respect to the patients they recruit and treatment management. These differences are particularly important for establishing the external validity of economic findings, with subgroup analysis a natural first test.
      For example, in a trial that has recruited patients younger than those observed in routine clinical practice, and where the intervention demonstrates reduced effectiveness in the elderly subgroup, showing the generalizability of the trial findings to the proportions of the elderly found in routine clinical practice is akin to extending the heterogeneity of the trial subgroup to the overall clinical population. We explored the TAs acknowledgement of differences between trial populations and routine clinical practice and their approaches to real-world data translation.
      Firstly, we extracted patient characteristic data for age, sex, Eastern Cooperative Oncology performance status (ECOG-PS) and ethnicity from the pivotal clinical studies. Where multiple clinical studies were included for a single TA, the range of results was presented and discussed. Secondly, for clinical studies with a National Clinical Trial identification number we reviewed the exclusion criteria described on the ClinicalTrials.gov website. The extent to which exclusion criteria would reduce the comparability between trial and routine practice populations was discussed. Finally, the TA submissions were reviewed for explicit acknowledgements of differences between trial and routine clinical practice, and economic analyses were reviewed for analytical methods that accounted for these differences.

      Results

      Included Studies

      A total of 49 TAs, published between 2003 and 2020, were included in the review; 38 evaluated a targeted therapy (Fig. 1). The included TAs were dominated by lung (L) cancer appraisals (32/49), of which 31 were for non-small cell lung cancer; there were 8 colorectal (C) and 9 ovarian (O) cancer TAs. Details of excluded studies are presented in Appendix 3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.12.022, alongside an overview of each included TA (Fig.1).
      The clinical evidence across all TAs was informed by a total of 94 (C: 22; L: 55; O: 17) clinical studies. Among the TAs, a total of 49 (C: 9; L: 31; O: 9) cost-effectiveness models were available for review. A total of 41 cost-effectiveness analyses were undertaken by the submitting pharmaceutical company, with 8 undertaken by academic review groups.

      Subgroup Analysis

      Subgroup analyses assessing either PFS or OS were reported for 72 (C: 13; L: 44; O: 15) of the clinical studies in either the clinical section of the TA or in the main clinical study publication cited in the TA. A total of 804 subgroup analyses were described among these 72 clinical studies. The most common stratification factors were age (62 studies), sex (46 studies), and ECOG-PS (50 studies). Across all reported clinical subgroup analyses, 14.8% (119/804) observed results that differed to those of the overall population. Figure 2 contrasts the number of subgroup analyses presented as clinical evidence to the number of subgroup analyses undertaken within economic evaluations. Subgroup analysis in the economic evaluations was only conducted in 17 TAs. In 8 TAs the conclusions from at least one economic subgroup analysis differed to those of the main population, based on cost-effectiveness criteria described by the analysis authors. The most common subgroups included histology (5 lung cancer studies) and mutation status (8 lung cancer studies; Fig. 2).
      Figure thumbnail gr2
      Figure 2Overview of subgroup analyses presented in clinical and economic submissions. The number in brackets next to the TA number indicates how many clinical studies reported at least some information relating to the use of subgroup analyses. A number of subgroup analysis and results were unavailable for review either due to absence or redaction.

      Economic Modeling

      Table 1 describes the structures of the 49 cost-effectiveness models. The majority of models were described as PSMs (total: 39; C: 5; L: 27; O: 7), Markov models (total: 5; C: 3; L: 2; O: 0), or semi-Markov models (total: 4; C: 0; L: 2; O: 2). Figure 3 describes the health states included in each of the models. Health states of partitioned survival models typically reflected PFS, progression and death (37 models), with 6 of these models including response- or treatment-based substates. In contrast, Markov and semi-Markov structures described a range of health states reflecting treatment and clinical status. All models were cohort-level, and given the majority included between 2 and 4 health states only, there was little consideration of individual patient clinical heterogeneity or variability within the model structures themselves (Fig. 3).
      Table 1Overview of model structures.
      Model typeModeled health statesModeled health state transitions
      Underlined transitions denote those that are modeled with different rates across each treatment arm.
      Outcomes informing health state transitionsPFS or TTD analysisOS analysisTechnology Assessment
      PSMAlive; deathAlive -> deathOS-Piecewise parametric survival models
      Same survival model form chosen for each treatment arm.
      TA190
      On treatment; off treatment; deathOn treatment -> Off treatment; On treatment -> Death; off treatment -> deathTTD; OSPiecewise parametric survival models
      Same survival model form chosen for each treatment arm.
      Mixture-cure model
      Same survival model form chosen for each treatment arm.
      TA520
      Progression-free; progression; deathProgression-free -> progression; progression-free -> death; progression -> deathPFS; OSKM data
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      Parametric survival models
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      TA227
      Multivariable parametric model with treatment covariateMultivariable parametric model with treatment covariateTA406
      Multivariable parametric models
      Same survival model form chosen for each treatment arm.
      TA403
      Multivariable parametric model
      Same survival model form chosen for each treatment arm.
      Multivariable parametric model
      Same survival model form chosen for each treatment arm.
      TA529
      Multivariable parametric model
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      Multivariable parametric model
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      TA192
      Parametric survival model with treatment covariateParametric survival model with treatment covariateTA310
      Parametric survival models
      Same survival model form chosen for each treatment arm.
      TA118
      Piecewise parametric survival model with treatment covariateTA621
      Parametric survival models
      Same survival model form chosen for each treatment arm.
      Parametric survival models
      Same survival model form chosen for each treatment arm.
      TA184, TA242, TA285, TA307, TA347, TA389, TA395, TA405, TA416, TA484, TA528, TA536, TA611
      Piecewise parametric survival models
      Same survival model form chosen for each treatment arm.
      TA428
      Parametric survival models
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      Parametric survival models
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      TA500, TA571, TA584, TA595
      Piecewise parametric survival models
      Same survival model form chosen for each treatment arm.
      Parametric model with treatment covariateTA284
      Piecewise parametric survival models
      Same survival model form chosen for each treatment arm.
      TA212, TA374, TA402, TA411, TA531, TA557, TA600
      Piecewise parametric survival models
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      TA598
      Spline model
      Same survival model form chosen for each treatment arm.
      Spline model
      Same survival model form chosen for each treatment arm.
      TA620
      Spline model with treatment covariateParametric survival model with treatment covariateTA483
      Model typeModeled health statesModeled health state transitionsOutcomes informing health state transitionsNondeath transitionsDeath transitionsTechnology assessment
      Markov model1st line, 2nd line, 3rd line, postresection, death1st line -> post resection; 1st line -> 2nd line; 1st line -> death; post resection -> death; 2nd line -> 3rd line; 2nd line -> death; 3rd line -> deathResection rate, PFS, ToT, OSParametric survival models
      Same survival model form chosen for each treatment arm.
      Parametric survival models
      Same survival model form chosen for each treatment arm.
      TA439
      Alive without relapse, alive with relapse, deathAlive without relapse -> alive with relapse; alive without relapse -> death; alive with relapse -> deathDFS, PPS, ACMParametric survival models
      Same survival model form chosen for each treatment arm.
      Mixture of exponential transition rates and life tablesTA100
      PFS: 1st line, PFS: no drug, PFS: post successful resection, PD: post successful resection, 2nd line: FOLFOX/FOLFIRI, 3rd line: BSC, deathPFS-1st line -> PFS-no drug; PFS-1st line -> PFS-post successful resection; PFS-1st line -> death; PFS-no drug -> 2nd line-FOLFOX/FOLFIRI; PFS-no drug -> death; PFS-post successful resection -> PD-post successful resection; PFS-post successful resection -> Death; 2nd line-FOLFOX/FOLFIRI -> 3rd line-BSC; 2nd line-FOLFOX/FOLFIRI -> death; 3rd line-bsc -> deathResection rate, PFS, ToT, OSMixture of parametric survival models and exponential ratesMixture of parametric survival models and exponential ratesTA439
      Response, stable disease, progressive disease, deathStable disease -> response; stable disease -> progression; response -> progression; progression -> deathResponse, PFS, OSExponential transition ratesExponential transition ratesTA124, TA181
      Semi-Markov modelProgression-free, first subsequent treatment, second subsequent treatment, deathPFS -> First subsequent treatment; PFS -> death; first subsequent treatment -> second subsequent treatment; first subsequent treatment -> death; second subsequent treatment -> deathToT, OSMultivariable parametric survival models with or without treatment coefficientsMultivariable parametric survival models with or without treatment coefficientsTA381
      Progression-free, progressed, deathProgression-free -> progression; progression-free -> death; progression -> deathPFS, PPS, OSPiecewise parametric survival models
      Same survival model form chosen for each treatment arm.
      Exponential transition ratesTA284
      Parametric survival models
      Same survival model form chosen for each treatment arm.
      Exponential transition ratesTA578
      Progression-free -> progression; progression-free -> death; progression -> deathPFS, PPS, OSPiecewise parametric survival models
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      Exponential transition ratesTA258
      Accounting exercise--OS--TA61
      ACM indicates all-cause mortality; DFS, disease-free survival; OS: overall survival; PFS, progression-free survival; PPS, postprogression survival; PSM, partitioned survival model; TA, technology appraisal; ToT, time on treatment; TTD, time to discontinuation.
      TA192, TA402 and TA411: Model structure believed to be incorrectly described as Markov model in submission.
      TA118: Model structure not described but assumed to be PSM based on description of parameters.
      TA192 and TA258: PFS health state stratified in to 2 substates (“treatment response” and “stable disease”) based on proportions at model initiation.
      TA242: Some (but not all) comparator survival estimates informed by survival ratios applied to parametric curves of other arms (holding shape parameters constant).
      TA212, TA307 and TA611: PFS health state stratified in to 2 substates (“on treatment” and “post-treatment”) using parametric survival models.
      TA374: Two populations modeled where piecewise spline models used for one population and piecewise parametric models used for one population.
      TA381: Unclear whether single models were used for certain transitions (ie, treatment independent transitions) as the submission contains contradictory statements; the external review group report states that apart from time to first event, all other transitions were set the same for each treatment arm.
      TA411: PFS health state stratified in to 3 substates (“on induction treatment,” “off treatment,” “receiving maintenance treatment”) using parametric survival models.
      TA484: TTD used as proxy for PFS.
      TA528: Model states that mean survival estimates are used (therefore not strictly a PSM as an area-under-the-curve approach not utilized); however, parametric survival curves are used to assess PFS and OS so it has been included in the PSM section.
      TA536: PFS stratified in to 2 substates (“patients with brain metastases” and “patients without brain metastases”) although it is unclear how patients are stratified.
      Underlined transitions denote those that are modeled with different rates across each treatment arm.
      Same survival model form chosen for each treatment arm.
      Survival models were only produced for one arm with indirect treatment comparison results used to inform disease progression in other arms.
      With respect to treatment heterogeneity, 42 (C: 4; L: 31; O: 7) economic analyses used data from a single clinical study relating to the first modeled line of treatment only, relying on either clinical expert opinion or validation against published studies with longer-term follow-up to justify extrapolation choices. Appendix 4 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.12.022 demonstrates the growing importance of accurate clinical extrapolation; across the 72 clinical studies for which information on the maturity of clinical data was available, 46% (33 of 72) of studies had observed events in <50% of patients at the time of analysis (17% [12 of 72] had observed events in <25% of patients). Furthermore, there appears to be no discernible relationship between the length of follow-up of the clinical studies and the choice of modeling structure, with lung cancer studies, as expected due to their comparatively lower survival rate, observing the shortest periods of follow-up on average.
      The modeling of treatment pathways is described in Table 2, alongside additional context with respect to the modeled population. Despite real-world potential for multiple subsequent therapies across many of the reviewed indications, 20% (10 of 49) of models did not include subsequent therapy at all and 71% (35 of 49) included only one explicit subsequent line of therapy (not including best supportive care). Of those models that included the impact of subsequent therapies, this impact was limited to cost accrual in 77% (30 of 39) of models and to cost accrual and utility values in 18% (7 of 39) of models; subsequent therapy impacted disease progression, cost accrual and utility values in the remaining 2 models (Tables 1 and 2).
      Table 2Overview of modeled treatment pathways.
      PopulationNo. of subsequent therapies modeled
      Subsequent therapy is defined as either targeted therapy, chemotherapy, surgery, or radiotherapy and does not capture the modeling of best supportive care.
      Impact of subsequent therapyMethod for estimating time on initial treatmentTA
      CancerPreviously treatedStage
      For NSCLC, “advanced or metastatic” includes “recurrent disease” in TA347 and, for ovarian cancer, “advanced” includes one appraisal that looked at stage III/IV disease (TA284).
      Mutation-specific
      CRCNoDukes stage CNo1Cost onlyMean treatment durationTA100
      MetastaticNoNone-Explicit number of cycles capped by OSTA61
      ToT KM curveTA212
      Yes1Cost, utility, disease progressionInitial treatment modeled with own health stateTA439 (external review group)
      2Cost, utility, disease progressionInitial treatment modeled with own health stateTA439 (company)
      YesMetastaticNoNone-Treatment to progression or mean treatment durationTA242
      No1Cost onlyMean treatment durationTA307
      No1Cost onlyTreatment to progressionTA405
      Yes2Cost onlyMean treatment durationTA118
      NSCLCBothAdvanced or metastaticYes1Cost and utilityTTD modeled independently using KM dataTA529
      Cost onlyParametric ToT modelTA584
      NoAdvanced or metastaticNo1Cost onlyCyclic discontinuation rate capped by specific no. of cyclesTA181
      Parametric ToT modelTA557
      ToT KM curveTA411
      Treatment to progressionTA600
      YesNone-Parametric ToT modelTA258
      1Cost and utilityMean treatment duration beyond progressionTA406, TA621
      Treatment to progressionTA310
      Cost onlyParametric ToT modelTA500
      Treatment to progressionTA192, TA531, TA536
      2Cost onlyTreatment to progressionTA595
      YesAdvanced or metastaticNoNone-Cyclic discontinuation rate capped by specific number of cyclesTA124
      Treatment to progressionTA374
      1Cost and utilityParametric ToT modelTA520
      Treatment to progressionTA483, TA484
      Cost onlyCyclic discontinuation rate capped by progressionTA347
      Parametric ToT modelTA402
      ToT KM curveTA578
      Treatment to progressionTA190, TA227, TA403
      YesNoneCost onlyIndependent mean duration beyond progressionTA571
      1Cost onlyTreatment to progressionTA416, TA428
      AnyYesNone-Treatment to progressionTA395
      SCLCYesRelapsedNoNone-Specific number of treatment cyclesTA184
      Ovarian cancerAnyAnyNo---TA55
      NoAdvancedNoNoneCost onlyMean treatment durationTA284 (Model 1)
      1Cost onlyMean treatment durationTA285
      TA284 (Model 2)
      YesAnyNo1Cost onlyTreatment to progressionTA389
      AdvancedYes1Cost onlyParametric ToT modelTA598
      High-gradeNo1Cost onlyParametric ToT modelTA528, TA611
      Yes1Cost onlyParametric ToT modelTA620
      2Cost onlyInitial treatment modeled with own health stateTA381
      CRC indicates colorectal cancer; KM, Kaplan-Meier; NSCLC, non-small cell lung cancer; SCLC, small cell lung cancer; TA, technology appraisal; ToT, time on treatment.
      TA284 (Model 2), TA416 and TA528: Average number of subsequent therapies received was greater than 1 but modeled within one subsequent line of therapy.
      TA307: Despite including substates within the progression-free health state describing treatment status (“on treatment” versus “off treatment”), treatment costs were modeled based on a mean treatment duration.
      TA406: Only a proportion of patients were assumed to receive therapy postprogression; remaining patients ceased treatment at progression.
      TA439: Patients could also receive curative resection after first line treatment, independent of other treatment lines.
      For NSCLC, “advanced or metastatic” includes “recurrent disease” in TA347 and, for ovarian cancer, “advanced” includes one appraisal that looked at stage III/IV disease (TA284).
      Subsequent therapy is defined as either targeted therapy, chemotherapy, surgery, or radiotherapy and does not capture the modeling of best supportive care.

      Real-World Translation

      The majority of TAs (32 of 49) acknowledged differences between the patient characteristics or treatment pathways used in the clinical studies and routine clinical practice. Furthermore, within individual TAs, patient heterogeneity was particularly noticeable in those that included more than one pivotal clinical study (Appendix 5 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.12.022). In such TAs, where data were reported, the average range of median ages was 4.9 years (C: 4.9; L: 5.7; O: 3.0), with average ranges of 16% (C: 12%; L: 18%), 10% (C: 5%; L: 11%; O: 14%), and 21% (C: 31%; L: 20%; O: 12%) for the proportion of patients that were male, had an ECOG-PS of 0 or 1 and were white or Caucasian, respectively; the largest ranges in any single TA were 18.8 years (C: 18.5; L: 18.8; O: 3.1), 65% (C: 29%; L: 65%), 57% (C: 11%; L: 57%; O: 37%), and 99% (C: 58%; L: 99%; O: 13%), respectively.
      Figure 4 describes the most common exclusion criteria used by the clinical studies and gives an overall impression of the selective nature of clinical trials and how the trial populations might differ from those found in routine clinical practice. A total of 73 of 94 clinical studies described exclusion criteria. The most common criteria not related to the treatment indication (eg, treatment history, histology, mutation status, etc) were a history of other malignancies (35 of 73 studies) and a history of cardiac problems (31 of 73). Such exclusion criteria would likely ostracize a significant proportion of cancer patients in UK clinical practice that are expected to have comorbid conditions (Fig. 4).
      • Fowler H.
      • Belot A.
      • Ellis L.
      • et al.
      Comorbidity prevalence among cancer patients: a population-based cohort study of four cancers.
      Figure thumbnail gr4
      Figure 4Exclusion criteria categories reported by the clinical studies (limited to exclusion criteria reported in at least 3 clinical studies).
      Finally, few TAs attempted to investigate the impact of clinical heterogeneity through disease progression modeling, with only 5 models (all evaluating targeted therapies), including clinical covariates within their estimation of PFS and OS disease progression estimates (Table 1). Although not exclusive to these TAs, patients in the clinical studies associated with 4 out of the 5 TAs were systematically different to the routine clinical practice patients they were representing with respect to their ethnic origin. Of these, 2 TAs (TA406 and TA529) used methods to account for differences between trial and clinical practice populations. These lung cancer TAs generated disease progression survival models that included clinical covariates based on data from the clinical study. Subsequently, these survival models were used to predict clinical outcomes for the cost-effectiveness model at covariate values corresponding to those observed in published studies deemed representative of UK clinical practice. In both cases, the following clinical covariates were included in the survival models: race (Asian/non-Asian), ECOG-PS (0 or 1/2), brain metastases at baseline (yes/no), age (≥65/<65 years), sex, smoking status (never smoked/former or current smoker), and adenocarcinoma at baseline (yes/no).

      Discussion

      This is the first review to consider patient and treatment effect heterogeneity in UK TAs of colorectal, lung, and ovarian cancer. The review highlighted that whilst many clinical studies undertook subgroup analyses, only a small number of economic evaluations considered these subgroups further in modeling analyses. Although, such findings must be caveated with the potential for publication bias and the under-reporting of negative results in economic submissions. This lack of representation in economic analyses was notable as several clinical subgroup analyses presented results contradicting the overall population findings, although statistical significance was rare. In addition, it was common to find a significant and positive treatment effect in the overall population analysis with subgroup analyses failing to demonstrate the same effect (or achieve significance). Statistical significance was not a focus of this study due to inconsistent definitions across TAs, underreporting of results (eg, commercial-in-confidence redaction) and small sample sizes. Subgroup analyses provide evidence for improved allocation of healthcare resources, with the potential to tailor reimbursement recommendations to specific patient groups where evidence for effectiveness is either very strong or very weak. Guidance is available on when to apply subgroup analysis in cost-effectiveness evaluation, with such analyses continuing to be a preferred first step to evaluate patient and treatment heterogeneity.
      • Espinoza M.A.
      • Manca A.
      • Claxton K.
      • Sculpher M.J.
      The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application.
      ,
      • Cui L.
      • Hung H.M.
      • Wang S.J.
      • Tsong Y.
      Issues related to subgroup analysis in clinical trials.
      • Cook D.I.
      • Gebski V.J.
      • Keech A.C.
      Subgroup analysis in clinical trials.
      • Grouin J.M.
      • Coste M.
      • Lewis J.
      Subgroup analyses in randomized clinical trials: statistical and regulatory issues.
      Review of the economic models further showed that the ability to incorporate heterogeneity in the economic evaluations was hampered by an overreliance on relatively simplistic cohort-based modeling structures. It was found for example, that most economic models used univariable disease progression estimates and represented disease progression through just 3 health states. Of particular note is what may be perceived as a systemic reliance on PSMs to demonstrate the economic impact of new cancer therapies. PSMs are designed for use with near complete clinical data and relatively simplistic treatment and disease pathways.
      NICE NIfHaCE
      Nice DSU technical support document 19: partitioned survival analysis for decision modeling in health care: a critical review.
      Inherently, models with simple structures lack flexibility and therefore do not lend themselves to the modeling of heterogeneity, particularly those of patient characteristics and treatment effects.
      The NICE Decision Support Unit technical documentation suggests patient-level simulation should be considered when the number of categories required to define patient groups with homogeneous outcomes becomes large.
      • Davis S.
      • Stevenson M.
      • Tappenden P.
      • Wailoo A.
      NICE decision support unit technical support documents.
      Patient-level simulation is also advocated for consideration when the likelihood of future events (eg, death) are dependent on the time since previous events (eg, disease progression). Notably, these criteria are true of certain TAs in this review, with the latter being particularly relevant to cancers for which curative treatment is available (colorectal and ovarian cancer) and those where disease progression is particularly influential over patient prognosis (lung cancer). However, patient-level models often have greater computational requirements, with respect to the data required, the time taken to run analyses and the complexity of such analyses. As such, trade-offs between analyst time, computation time, and the requirements of the decision problem may be required. To justify final model selection, a checklist approach could be used to characterize the decision problem, the data, computational limitations, and other relevant issues. This approach offers several advantages over algorithmic model selection, including the ability to summarize strengths and weaknesses of modeling approaches within the context of the decision problem aiding critical appraisal of model choice, and the avoidance of prescriptive decisions that create the illusion that only one model type suits a particular decision problem.
      The review additionally identified weaknesses in the methods used to extrapolate clinical endpoints to policy-relevant time horizons. Extrapolations rarely considered clinical mechanisms for estimating disease progression, and instead relied predominantly on within-trial statistical goodness-of-fit output, visual inspection, and comparison to historical data. A potential solution is to use risk equations to aid in the extrapolation of outcomes beyond the trial phase using clinical and treatment history data. This approach is commonly applied in other disease areas such as diabetes, cardiovascular disease, and chronic kidney disease.
      • Cichosz S.L.
      • Johansen M.D.
      • Hejlesen O.
      Toward big data analytics: review of predictive models in management of diabetes and its complications.
      • Collins G.S.
      • Mallett S.
      • Omar O.
      • Yu L.-M.
      Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting.
      • Noble D.
      • Mathur R.
      • Dent T.
      • Meads C.
      • Greenhalgh T.
      Risk models and scores for type 2 diabetes: systematic review.
      • Tangri N.
      • Kitsios G.D.
      • Inker L.A.
      • et al.
      Risk prediction models for patients with chronic kidney disease: a systematic review.
      • Ramspek C.L.
      • de Jong Y.
      • Dekker F.W.
      • van Diepen M.
      Towards the best kidney failure prediction tool: a systematic review and selection aid.
      • Di Tanna G.L.
      • Wirtz H.
      • Burrows K.L.
      • Globe G.
      Evaluating risk prediction models for adults with heart failure: a systematic literature review.
      • Damen J.A.
      • Hooft L.
      • Schuit E.
      • et al.
      Prediction models for cardiovascular disease risk in the general population: systematic review.
      The derivation of these risk equations is typically undertaken from large real-world observational datasets and may also assist in alleviating concerns over the real-world applicability of outcome extrapolation. These methods may have previously been overlooked in cancer due to the potential for low quality of recording of data, the propensity for cancer treatments to fundamentally change the course of disease and for the prevalence of highly unique cancer subpopulations defined by genetic variation.
      • Muller P.
      • Walters S.
      • Coleman M.P.
      • Woods L.
      Which indicators of early cancer diagnosis from population-based data sources are associated with short-term mortality and survival?.
      • Kolovos S.
      • Nador G.
      • Kishore B.
      • et al.
      Unplanned admissions for patients with myeloma in the UK: low frequency but high costs.
      • Laudicella M.
      • Walsh B.
      • Burns E.
      • Smith P.C.
      Cost of care for cancer patients in England: evidence from population-based patient-level data.
      • McConnell H.
      • White R.
      • Maher J.
      Categorising cancers to enable tailored care planning through a secondary analysis of cancer registration data in the UK.
      • Ward S.E.
      • Holmes G.R.
      • Ring A.
      • et al.
      Adjuvant chemotherapy for breast cancer in older women: an analysis of retrospective English cancer registration data.
      • Nguyen A.
      • Yoshida M.
      • Goodarzi H.
      • Tavazoie S.F.
      Highly variable cancer subpopulations that exhibit enhanced transcriptome variability and metastatic fitness.
      However, national comprehensive clinical practice datasets have improved in both quality and coverage over recent years.
      Clinical Practice Research Datalink
      Clinical practice research datalink dataset.
      NHS Digital
      Hospital episode statistics (HES).
      Public Health England
      Cancer registry dataset.
      UK Biobank
      Biobank Dataset.
      Combining risk equations with more flexible and sophisticated modeling methods will provide greater consideration, and understanding, of real-world patient and treatment effect heterogeneity, and go some way to addressing historical limitations.
      Finally, while appraisals acknowledged differences between the clinical studies from which their evidence was based and routine clinical practice, few summarized these differences quantitatively. Clearly there are tensions between the representativeness of clinical trials and the necessity of a trial to have homogeneous groups of patients to enable comparison between groups.
      • D'Agostino R.B.
      • Kwan H.
      Measuring effectiveness. What to expect without a randomized control group.
      However, homogeneity does not need to come at the expense of the natural heterogeneity observed in the population of interest, which may become the case when extensive exclusion criteria are applied. Addressing such issues is not straightforward, with patient safety, ethical issues, and sample size considerations at the forefront of concern. As a relatively simple and practical initial step, we suggest that trial investigators could improve reporting by making available more evidence on clinical outcomes of stratification subgroups, alongside encouraging access to individual patient data (IPD) for research. Subsequently, we would advocate the addition of a more explicit and structured comparison of routine clinical practice and trial patient populations within TAs. Such a comparison might take the form of a quantitative side-by-side summary of the clinical and demographic characteristics of patients from both groups, based on relevant UK clinical practice datasets and the clinical studies informing the appraisal. We would encourage the adoption of such an approach as standard practice within TAs to provide relevant parties with a transparent overview of both the relevance and the extent of any differences.
      Furthermore, very few TAs used methods to adjust cancer outcomes to account for differences between trial and routine clinical practice, even in the most recent TAs. This is particularly relevant given observed differences between cancer outcomes in these settings, particularly amongst PFS and OS outcomes,
      • Lakdawalla D.N.
      • Shafrin J.
      • Hou N.
      • et al.
      Predicting real-world effectiveness of cancer therapies using overall survival and progression-free survival from clinical trials: empirical evidence for the ASCO value framework.
      and the need for policymakers to understand these differences to inform policy recommendations and guidelines. Two TAs used a form of simulated treatment comparison (STC), generating survival models that included clinical covariates based on IPD from the clinical study, and subsequently using these to predict clinical outcomes for the cost-effectiveness model at covariate values deemed representative of UK clinical practice. Methods such as STC and matching-adjusted indirect comparison, aim to reduce bias in treatment comparisons by using IPD from the clinical studies to provide indicative estimates of the likely outcomes in different settings, and may be used to address the above concerns.
      • Signorovitch J.E.
      • Wu E.Q.
      • Yu A.P.
      • et al.
      Comparative effectiveness without head-to-head trials.
      ,
      • Caro J.J.
      • Ishak K.J.
      No Head-to-head trial? simulate the missing arms.
      Matching-adjusted indirect comparison adjusts average population-level outcomes by applying weights to IPD from the clinical study, using larger weights for patients that more closely match those of routine clinical practice, while STC uses regression equations to adjust estimates.
      Clearly these suggestions should acknowledge the current constraints of the NICE review process, which is subject to strict timelines. For example, IPD needed for patient-level simulation or risk equation development may not be available to researchers. This raises the questions of how NICE should resource future TAs to enable them to better incorporate heterogeneity and related equity concerns. Another limitation of the review is the pragmatic decision to consider 3 cancer sites. Additional research is required before we can generalize across all cancers and across economic evaluations of cancer outside the remit of NICE TAs.

      Conclusion

      This study highlights a relative paucity of information relating to the assessment of heterogeneity in UK cancer TAs and identifies a mostly unjustified reliance on relatively simplistic modeling frameworks. If heterogeneity considerations are to be included in TA frameworks, and the complexity of treatment and disease pathways reflected in economic analyses, there is a requirement to embrace more flexible modeling approaches and to further research real-world heterogeneity.

      Article and Author Information

      Author Contributions: Concept and design: Ward, Medina-Lara, Mujica-Mota, Spencer
      Acquisition of data: Ward, Medina-Lara, Mujica-Mota
      Analysis and interpretation of data: Ward
      Drafting of the manuscript: Ward, Medina-Lara, Mujica-Mota, Spencer
      Critical revision of the paper for important intellectual content: Ward, Medina-Lara, Mujica-Mota, Spencer
      Provision of study materials or patients: Ward, Medina-Lara, Mujica-Mota
      Obtaining funding: Spencer
      Administrative, technical, or logistic support: Ward
      Supervision: Medina-Lara, Mujica-Mota, Spencer
      Conflict of Interest Disclosures: Mr Ward and Dr Spencer reported receiving grants from the Dennis and Mereille Gillings Foundation during the conduct of the study. No other disclosures were reported.
      Funding/Support: This research has been supported by a PhD studentship grant for Mr Ward from the Dennis and Mereille Gillings Foundation.
      Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

      Acknowledgment

      This research arises from the Dennis and Mireille Gillings Foundation funded ERICA trial that explores the use of electronic risk assessments to help identify possible cancers in primary care. Thomas Ward is a postgraduate researcher and Anne Spencer is a co-applicant on this trial. This research is linked to the CanTest Multi-institution Collaborative, which is funded by Cancer Research UK (C8640/A23385), of which Mr Ward is an affiliated postgraduate researcher and Dr Spencer is senior faculty. We would like to thank Willie Hamilton, MD, Professor of Primary Care Diagnostics, University of Exeter, Exeter, England, UK, for his continued support and clinical advice on the choice of cancer areas as principal investigator of the ERICA trial.

      Supplemental Material

      References

      1. Public Health England. Health profile for England: 2019 In: UK government; 2019.

        • Smittenaar C.R.
        • Petersen K.A.
        • Stewart K.
        • Moitt N.
        Cancer incidence and mortality projections in the UK until 2035.
        Br J Cancer. 2016; 115: 1147-1155
        • Cancer Research UK
        Cancer incidence statistics.
        • Hawkes N.
        Cancer survival data emphasise importance of early diagnosis.
        BMJ. 2019; 364: l408
        • Maddams J.
        • Utley M.
        • Moller H.
        Projections of cancer prevalence in the United Kingdom, 2010-2040.
        Br J Cancer. 2012; 107: 1195-1202
        • Macmillan cancer support
        Statistics fact sheet.
        2019
        • Pfizer
        Cancer costs: a ripple effect analysis of cancer’s wider impact.
        2020
        • Arnold M.
        • Rutherford M.J.
        • Bardot A.
        • et al.
        Progress in cancer survival, mortality, and incidence in seven high-income countries 1995-2014 (ICBP SURVMARK-2): a population-based study.
        Lancet Oncol. 2019; 20: 1493-1505
        • Allemani C.
        • Matsuda T.
        • Di Carlo V.
        • et al.
        Global surveillance of trends in cancer survival 2000-14 (CONCORD-3): analysis of individual records for 37 513 025 patients diagnosed with one of 18 cancers from 322 population-based registries in 71 countries.
        Lancet. 2018; 391: 1023-1075
        • Falzone L.
        • Salomone S.
        • Libra M.
        Evolution of cancer pharmacological treatments at the turn of the third millennium.
        Front Pharmacol. 2018; 9: 1300
        • National Institute for Health and Care Excellence
        Breast screening.
        (Published 2017. Accessed October 24, 2019)
        • National Institute for Health and Care Excellence
        Bowel screening.
        (Published 2019. Accessed October 24, 2019)
        • National Institute for Health and Care Excellence
        Cervical screening.
        (Published 2017. Accessed October 24, 2019)
      2. NHS to rollout lung cancer scanning trucks across the country [press release]. 2019
        • National Health Service
        The NHS Long Term Plan.
        2019
        • Varadhan R.
        • SJ
        Estimation and reporting of heterogeneity of treatment effects.
        in: Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. Agency for Healthcare Research and Quality (US), Rockville (MD)2013
        • Yuan M.
        • Huang L.-L.
        • Chen J.-H.
        • Wu J.
        • Xu Q.
        The emerging treatment landscape of targeted therapy in non-small-cell lung cancer.
        Signal Transduction and Targeted Therapy. 2019; 4: 61
        • Xin Yu J.
        • Hubbard-Lucey V.M.
        • Tang J.
        The global pipeline of cell therapies for cancer.
        Nat Rev Drug Discov. 2019; 18: 821-822
        • Sculpher M.
        Subgroups and heterogeneity in cost-effectiveness analysis.
        Pharmacoeconomics. 2008; 26: 799-806
        • Grutters J.P.
        • Sculpher M.
        • Briggs A.H.
        • et al.
        Acknowledging patient heterogeneity in economic evaluation: a systematic literature review.
        Pharmacoeconomics. 2013; 31: 111-123
        • Ramaekers B.L.T.
        • Joore M.A.
        • Grutters J.P.C.
        How should we deal with patient heterogeneity in economic evaluation: a systematic review of national pharmacoeconomic guidelines.
        Value Health. 2013; 16: 855-862
        • Cookson R.
        • Propper C.
        • Asaria M.
        • Raine R.
        Socio-economic inequalities in health care in England.
        Fiscal Studies. 2016; 37: 371-403
        • Foster H.M.E.
        • Celis-Morales C.A.
        • Nicholl B.I.
        • et al.
        The effect of socioeconomic deprivation on the association between an extended measurement of unhealthy lifestyle factors and health outcomes: a prospective analysis of the UK Biobank cohort.
        The Lancet Public Health. 2018; 3: e576-e585
      3. Macmillan cancer support.
        Health Inequalities: Time to Talk. 2019
        • Fowler H.
        • Belot A.
        • Ellis L.
        • et al.
        Comorbidity prevalence among cancer patients: a population-based cohort study of four cancers.
        BMC Cancer. 2020; 20: 2
      4. Public Health England. Health profile for England: 2018. In: UK government; 2018.

        • Exarchakou A.
        • Rachet B.
        • Belot A.
        • Maringe C.
        • Coleman M.P.
        Impact of national cancer policies on cancer survival trends and socioeconomic inequalities in England, 1996-2013: population based study.
        BMJ. 2018; 360: k764
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6e1000097
        • Espinoza M.A.
        • Manca A.
        • Claxton K.
        • Sculpher M.J.
        The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application.
        Med Decis Making. 2014; 34: 951-964
        • Brennan A.
        • Chick S.E.
        • Davies R.
        A taxonomy of model structures for economic evaluation of health technologies.
        Health Econ. 2006; 15: 1295-1310
        • Briggs A.D.M.
        • Wolstenholme J.
        • Blakely T.
        • Scarborough P.
        Choosing an epidemiological model structure for the economic evaluation of non-communicable disease public health interventions.
        Popul Health Metr. 2016; 14 (17-17)
        • Bullement A.
        • Cranmer H.L.
        • Shields G.E.
        A review of recent decision-analytic models used to evaluate the economic value of cancer treatments.
        Appl Health Econ Health Policy. 2019; 17: 771-780
        • Sonnenberg F.A.
        • Beck J.R.
        Markov models in medical decision making: a practical guide.
        Medical Decision Making. 1993; 13: 322-338
        • Abner E.L.
        • Charnigo R.J.
        • Kryscio R.J.
        Markov chains and semi-Markov models in time-to-event analysis.
        J Biom Biostat. 2013; Suppl 1: 19522
        • Geifman N.
        • Butte A.J.
        Do cancer clinical trial populations truly represent cancer patients? A comparison of open clinical trials to the cancer genome atlas.
        Pac Symp Biocomput. 2016; 21: 309-320
        • Unger J.M.
        • Barlow W.E.
        • Martin D.P.
        • et al.
        Comparison of survival outcomes among cancer patients treated in and out of clinical trials.
        J Natl Cancer Inst. 2014; 106
        • Mitchell A.P.
        • Harrison M.R.
        • George D.J.
        • Abernethy A.P.
        • Walker M.S.
        • Hirsch B.R.
        Clinical trial subjects compared to “real world” patients: generalizability of renal cell carcinoma trials.
        J Clin Oncol. 2014; 32: 6510
        • Cui L.
        • Hung H.M.
        • Wang S.J.
        • Tsong Y.
        Issues related to subgroup analysis in clinical trials.
        J Biopharm Stat. 2002; 12: 347-358
        • Cook D.I.
        • Gebski V.J.
        • Keech A.C.
        Subgroup analysis in clinical trials.
        Med J Aust. 2004; 180: 289-291
        • Grouin J.M.
        • Coste M.
        • Lewis J.
        Subgroup analyses in randomized clinical trials: statistical and regulatory issues.
        J Biopharm Stat. 2005; 15: 869-882
        • NICE NIfHaCE
        Nice DSU technical support document 19: partitioned survival analysis for decision modeling in health care: a critical review.
        http://nicedsu.org.uk/
        Date accessed: February 6, 2017
        • Davis S.
        • Stevenson M.
        • Tappenden P.
        • Wailoo A.
        NICE decision support unit technical support documents.
        in: NICE DSU Technical Support Document 15: Cost-Effectiveness Modeling Using Patient-Level Simulation. National Institute for Health and Care Excellence (NICE), London2014
        • Cichosz S.L.
        • Johansen M.D.
        • Hejlesen O.
        Toward big data analytics: review of predictive models in management of diabetes and its complications.
        J Diabetes Sci Technol. 2016; 10: 27-34
        • Collins G.S.
        • Mallett S.
        • Omar O.
        • Yu L.-M.
        Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting.
        BMC Medicine. 2011; 9: 103
        • Noble D.
        • Mathur R.
        • Dent T.
        • Meads C.
        • Greenhalgh T.
        Risk models and scores for type 2 diabetes: systematic review.
        BMJ. 2011; 343: d7163
        • Tangri N.
        • Kitsios G.D.
        • Inker L.A.
        • et al.
        Risk prediction models for patients with chronic kidney disease: a systematic review.
        Ann Intern Med. 2013; 158: 596-603
        • Ramspek C.L.
        • de Jong Y.
        • Dekker F.W.
        • van Diepen M.
        Towards the best kidney failure prediction tool: a systematic review and selection aid.
        Nephrol Dial Transplant. 2020; 35: 1527-1538
        • Di Tanna G.L.
        • Wirtz H.
        • Burrows K.L.
        • Globe G.
        Evaluating risk prediction models for adults with heart failure: a systematic literature review.
        PLoS One. 2020; 15e0224135
        • Damen J.A.
        • Hooft L.
        • Schuit E.
        • et al.
        Prediction models for cardiovascular disease risk in the general population: systematic review.
        BMJ. 2016; 353: i2416
        • Muller P.
        • Walters S.
        • Coleman M.P.
        • Woods L.
        Which indicators of early cancer diagnosis from population-based data sources are associated with short-term mortality and survival?.
        Cancer Epidemiol. 2018; 56: 161-170
        • Kolovos S.
        • Nador G.
        • Kishore B.
        • et al.
        Unplanned admissions for patients with myeloma in the UK: low frequency but high costs.
        J Bone Oncol. 2019; 17: 100243
        • Laudicella M.
        • Walsh B.
        • Burns E.
        • Smith P.C.
        Cost of care for cancer patients in England: evidence from population-based patient-level data.
        Br J Cancer. 2016; 114: 1286-1292
        • McConnell H.
        • White R.
        • Maher J.
        Categorising cancers to enable tailored care planning through a secondary analysis of cancer registration data in the UK.
        BMJ Open. 2017; 7e016797
        • Ward S.E.
        • Holmes G.R.
        • Ring A.
        • et al.
        Adjuvant chemotherapy for breast cancer in older women: an analysis of retrospective English cancer registration data.
        Clin Oncol. 2019; 31: 444-452
        • Nguyen A.
        • Yoshida M.
        • Goodarzi H.
        • Tavazoie S.F.
        Highly variable cancer subpopulations that exhibit enhanced transcriptome variability and metastatic fitness.
        Nature Communications. 2016; 7: 11246
        • Clinical Practice Research Datalink
        Clinical practice research datalink dataset.
        https://www.cprd.com/
        Date accessed: September 28, 2020
        • NHS Digital
        Hospital episode statistics (HES).
        https://digital.nhs.uk/
        Date accessed: September 28, 2020
        • Public Health England
        Cancer registry dataset.
        https://www.cancerdata.nhs.uk/
        Date accessed: September 28, 2020
        • UK Biobank
        Biobank Dataset.
        https://www.ukbiobank.ac.uk/
        Date accessed: September 28, 2020
        • D'Agostino R.B.
        • Kwan H.
        Measuring effectiveness. What to expect without a randomized control group.
        Med Care. 1995; 33: As95-As105
        • Lakdawalla D.N.
        • Shafrin J.
        • Hou N.
        • et al.
        Predicting real-world effectiveness of cancer therapies using overall survival and progression-free survival from clinical trials: empirical evidence for the ASCO value framework.
        Value Health. 2017; 20: 866-875
        • Signorovitch J.E.
        • Wu E.Q.
        • Yu A.P.
        • et al.
        Comparative effectiveness without head-to-head trials.
        PharmacoEconomics. 2010; 28: 935-945
        • Caro J.J.
        • Ishak K.J.
        No Head-to-head trial? simulate the missing arms.
        PharmacoEconomics. 2010; 28: 957-967