Advertisement

Key Considerations in the Health Technology Assessment of Advanced Therapy Medicinal Products in Scotland, The Netherlands, and England

  • Renske M.T. ten Ham
    Correspondence
    Correspondence: Renske M.T. ten Ham, PhD, Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Universiteitsweg 99, Utrecht, The Netherlands 3584 CG.
    Affiliations
    Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht, The Netherlands

    Health Economics and Health Technology Assessment, Institute of Health and Wellbeing, University of Glasgow, Glasgow, Scotland, UK
    Search for articles by this author
  • Geert W.J. Frederix
    Affiliations
    Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht, The Netherlands

    Julius Centre for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
    Search for articles by this author
  • Olivia Wu
    Affiliations
    Health Economics and Health Technology Assessment, Institute of Health and Wellbeing, University of Glasgow, Glasgow, Scotland, UK
    Search for articles by this author
  • Wim Goettsch
    Affiliations
    Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht, The Netherlands

    National Health Care Institute, Diemen, The Netherlands
    Search for articles by this author
  • Hubert G.M. Leufkens
    Affiliations
    Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht, The Netherlands

    Lygature, Utrecht, The Netherlands
    Search for articles by this author
  • Olaf H. Klungel
    Affiliations
    Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht, The Netherlands
    Search for articles by this author
  • Jarno Hoekman
    Affiliations
    Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht, The Netherlands

    Innovation Studies, Copernicus Institute of Sustainable Development, Utrecht University, Utrecht, The Netherlands
    Search for articles by this author
Open AccessPublished:December 10, 2021DOI:https://doi.org/10.1016/j.jval.2021.09.012

      Highlights

      • Advanced therapy medicinal products (ATMPs) are highly innovative therapies. Their high costs and uncertain value claims raise concerns among health technology assessment bodies and payers, yet little is known about the experiences with health technology assessment of ATMPs.
      • Key considerations underlying the reimbursement recommendations of ATMPs were identified and assessed in Scotland, The Netherlands, and England using the EUnetHTA Core Model.
      • This study provides insights in supporting and opposing considerations for individual products and on a member state level. Besides the therapeutic effectiveness and cost-effectiveness domains, the social, ethical, and legal domains seem to bare considerable weight.

      Abstract

      Objectives

      Advanced therapy medicinal products (ATMPs) are highly innovative therapies. Their costs and uncertain value claims have raised concerns among health technology assessment (HTA) bodies and payers. Little is known about how underlying considerations in HTA of ATMPs shape assessment and reimbursement recommendations. We aim to identify and assess key considerations that played a role in HTA of ATMPs underlying reimbursement recommendations.

      Methods

      A review of HTA reports was conducted of all authorized ATMPs in Scotland, The Netherlands, and England. Considerations were extracted and categorized into EUnetHTA Core Model domains. Per jurisdiction, considerations were aggregated and key considerations identified (defined as occurring in >1/assessment per jurisdiction). A narrative analysis was conducted comparing key considerations between jurisdictions and different reimbursement recommendations.

      Results

      We identified 15 ATMPs and 18 HTA reports. In The Netherlands and England most key considerations were identified in clinical effectiveness (EFF) and cost- and economic effectiveness (ECO) domains. In Scotland, the social aspects domain yielded most key considerations, followed by ECO and EFF. More uncertainty in evidence and assessment outcomes was accepted when orphan or end-of-life criteria were applied. A higher percentage of considerations supporting recommendations were identified for products with positive recommendations compared with restricted and negative recommendations.

      Conclusions

      This is the first empirical review of HTA’s using the EUnetHTA Core Model to identify and structure key considerations retrospectively. It provides insights in supporting and opposing considerations for reimbursement of individual products and differences between jurisdictions. Besides the EFF and ECO domain, the social, ethical, and legal domains seem to bear considerable weight in assessment of ATMPs.

      Keywords

      Introduction

      Advances in biomedical science have resulted in translation of gene- and cell-based technologies into authorized therapies with a positive benefit-risk balance.
      • Eder C.
      • Wild C.
      Technology forecast : advanced therapies in late clinical research, EMA approval or clinical application via hospital exemption.
      In the European Union (EU), the diverse group of gene therapy medicinal products, cell therapy medicinal products, tissue engineered products, and combined advanced therapies are regulated as medicinal products from 2007 onward and formally defined as advanced therapy medicinal products (ATMPs).
      The European Parliament, the Council of the European Union. Regulation (EC) no 1394/2007 of the European Parliament and of the council.
      ATMPs are expected to provide opportunities for previously untreatable diseases with current development efforts targeting patients with high unmet medical need and orphan indications in particular. As of June 2020, 15 ATMPs have received marketing authorization (MA) in the EU with full development pipelines suggesting that more ATMPs will reach the market in the next few years.
      • Eder C.
      • Wild C.
      Technology forecast : advanced therapies in late clinical research, EMA approval or clinical application via hospital exemption.
      ,
      CAT monthly report of application procedures, guidelines and related documents on advanced therapies - June 2020 meeting. European Medicines Agency.
      After MA, ATMPs are subjected to formal health technology assessment (HTA) in individual member states to be considered for reimbursement. For access to these treatments, patients in Europe are dependent on inclusion of ATMPs in public healthcare funding. Nevertheless, HTA bodies and payers have expressed concerns about how to assess and appraise (relative) effectiveness, cost-effectiveness, and affordability of ATMPs.
      • Eder C.
      • Wild C.
      Technology forecast : advanced therapies in late clinical research, EMA approval or clinical application via hospital exemption.
      In particular, the novel and uncertain value claims in combination with high (upfront) payments are deemed challenging.
      • Hanna E.
      • Tavella F.
      • Rémuzat C.
      • Auquier P.
      • Toumi M.
      Market access of Atmps: overview and expected challenges.
      • Morrow D.
      • Ussi A.
      • Migliaccio G.
      Addressing pressing needs in the development of advanced therapies.
      • Ermisch M.
      • Bucsics A.
      • Bonanno P.V.
      • et al.
      Payers’ views of the changes arising through the possible adoption of adaptive pathways.
      In addition, time horizons of the sustained (curative) value claims often exceed available clinical evidence,
      • Coyle D.
      • Durand-Zaleski I.
      • Farrington J.
      • et al.
      HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies.
      resulting in extrapolation of evidence to model treatment benefits. Although such extrapolations are not new, considerable uncertainty is added by limited experience with interpreting data and assumptions regarding retreatment, treatment waning, and disease progression. In addition, several ATMPs has been authorized through expedited pathways, which leads to availability of less comprehensive data at the time of authorization and HTA compared with more conventional medicines.
      • Vreman R.A.
      • Bouvy J.C.
      • Bloem L.T.
      • et al.
      Weighing of evidence by health technology assessment bodies: retrospective study of reimbursement recommendations for conditionally approved drugs.
      ,
      • Vreman R.A.
      • Naci H.
      • Goettsch W.G.
      • et al.
      Decision making under uncertainty: comparing regulatory and health technology assessment reviews of medicines in the United States and Europe.
      Finally, ATMPs are administered in single or short-term courses while claiming curative or sustained benefits. Consequently, treatment cannot be discontinued when expected benefits are not accomplished, and cost of the unsuccessful treatment cannot be recouped.
      • Towse A.
      • Fenwick E.
      Uncertainty and cures: discontinuation, irreversibility, and outcomes-based payments: what is different about a one-off treatment?.
      Available literature on HTA of ATMPs provides several studies on how components of HTA frameworks can be adapted to increase the fit with the specific characteristics of ATMPs.
      • Coyle D.
      • Durand-Zaleski I.
      • Farrington J.
      • et al.
      HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies.
      ,
      • Angelis A.
      • Naci H.
      • Hackshaw A.
      Recalibrating health technology assessment methods for cell and gene therapies.
      Studies recommend introduction of novel value elements, adjustments to cost-effectiveness methodology, lowering of budget impact, structural weighing of ethical considerations, and several ways to address evidentiary uncertainties in the assessment.
      • Coyle D.
      • Durand-Zaleski I.
      • Farrington J.
      • et al.
      HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies.
      ,
      • Angelis A.
      • Naci H.
      • Hackshaw A.
      Recalibrating health technology assessment methods for cell and gene therapies.
      • ten Ham R.M.T.
      • Klungel O.H.
      • Leufkens H.G.M.
      • Frederix G.W.J.
      A review of methodological considerations for economic evaluations of gene therapies and their application in literature.
      • Petrou P.
      Is it a chimera? A systematic review of the economic evaluations of CAR-T cell therapy.
      • Rozier P.
      • Maria A.
      • Goulabchand R.
      • Jorgensen C.
      • Guilpain P.
      • Noël D.
      Mesenchymal stem cells in systemic sclerosis: allogenic or autologous approaches for therapeutic use?.
      • Gonçalves E.
      Advanced therapy medicinal products: value judgement and ethical evaluation in health technology assessment.
      Novel payment models are also proposed to address uncertainties around (sustained) effectiveness in combination with high upfront costs.
      • Carr D.R.
      • Bradshaw S.E.
      Gene therapies: the challenge of super-high-cost treatments and how to pay for them.
      • Jørgensen J.
      • Kefalas P.
      Annuity payments can increase patient access to innovative cell and gene therapies under England’s net budget impact test.
      • Barlow J.
      • Bens C.
      • Brent T.
      • et al.
      AMCP Partnership Forum. Designing benefits and payment models for innovative high-investment medications.
      Coyle et al
      • Coyle D.
      • Durand-Zaleski I.
      • Farrington J.
      • et al.
      HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies.
      and Angelis et al
      • Angelis A.
      • Naci H.
      • Hackshaw A.
      Recalibrating health technology assessment methods for cell and gene therapies.
      take a wider perspective and propose adaption of HTA methods and outline policy options to improve HTA of ATMPs. Several studies also describe challenges and strategies in obtaining reimbursement from a developer perspective,
      • Morrow D.
      • Ussi A.
      • Migliaccio G.
      Addressing pressing needs in the development of advanced therapies.
      ,
      • Jørgensen J.
      • Kefalas P.
      Reimbursement of licensed cell and gene therapies across the major European healthcare markets.
      ,
      • Hanna E.
      • Rémuzat C.
      • Auquier P.
      • Toumi M.
      Advanced therapy medicinal products: current and future perspectives.
      yet, to the best of our knowledge, studies providing empirical insights into current practices of HTA of ATMPs in EU member states are currently not available. Providing insights into ATMP assessment practices can contribute to the learning process on how to conduct HTA of these innovative products.
      Therefore, in this study, we conducted a review of HTA reports of authorized ATMPs in 3 EU countries (Scotland, The Netherlands, and England). The aim of this review is to identify and assess key considerations in the HTA underlying the reimbursement recommendations. We also examined whether considerations differed between different reimbursement recommendation types.

      Methods

      Study Design

      A narrative review was conducted of HTA reports of authorized ATMPs in Scotland, The Netherlands, and England. Included were all ATMPs that received centralized MA by the European Medicines Agency (EMA) up until June 1, 2020.
      CAT monthly report of application procedures, guidelines and related documents on advanced therapies - June 2020 meeting. European Medicines Agency.
      From the HTA reports, considerations were extracted, and key considerations identified. A consideration was defined as “a value judgment of the HTA body on the presented dossier by the developer.” This definition includes considerations that may contribute to a positive recommendation and issues or concerns that may contribute to a restricted or negative recommendation.

      Medicinal Products and Jurisdictions

      ATMPs were identified via a search of the EMA’s Committee of Advanced Therapy (CAT) monthly reports from March 2009 (first available public report after committee establishment) to June 2020.
      CAT monthly report of application procedures, guidelines and related documents on advanced therapies - June 2020 meeting. European Medicines Agency.
      CAT reports provide a summary of all ATMP-related regulatory activity in the EU, including MA opinions. Products that received a positive draft MA opinion were included. Excluded were products with a negative opinion, withdrawals, and ongoing assessments.
      In line with previous research,
      • Vreman R.A.
      • Bouvy J.C.
      • Bloem L.T.
      • et al.
      Weighing of evidence by health technology assessment bodies: retrospective study of reimbursement recommendations for conditionally approved drugs.
      eligibility of HTA bodies was assessed by applying the following inclusion criteria:
      • 1.
        HTA body is linked to a European jurisdiction.
      • 2.
        HTA jurisdiction is part of the EU at time of data collection (June 2020).
      • 3.
        The HTA body is the primary institute with legal remits within the jurisdiction.
      • 4.
        The HTA body systematically published HTA reports in the public domain.
      • 5.
        The published reports are written in a language understood by the researchers (ie, English or Dutch).
      This resulted in the inclusion of 3 HTA bodies: Scottish Medicines Consortium (SMC) from Scotland, National Health Care Institute (ZIN) from The Netherlands, and the National Institute for Health and Care Excellence (NICE) from England.
      HTA reports were retrieved via the HTA body websites by searching for the products brand and generic name.

      Bloem LT, Vreman RA, van Oirschot S, et al. The role of regulator-imposed post-approval studies in health technology assessments for conditionally approved drugs [published online October 27, 2020]. Int J Heal Policy Manag. https://doi.org/10.34172/ijhpm.2020.198.

      If products were authorized for multiple indications, reports for each indication were included.
      • Vreman R.A.
      • Bouvy J.C.
      • Bloem L.T.
      • et al.
      Weighing of evidence by health technology assessment bodies: retrospective study of reimbursement recommendations for conditionally approved drugs.
      In addition, only reports describing the outcome of the initial assessment were included.
      • Lipska I.
      • Hoekman J.
      • McAuslane N.
      • Leufkens H.G.M.
      • Hövels A.M.
      Does conditional approval for new oncology drugs in Europe lead to differences in health technology assessment decisions?.
      This resulted in the exclusion of reports describing resubmissions, withdrawals, or updates of ongoing assessment. This resulted in the inclusion of one HTA report per indication per jurisdiction. More specifically, the following are HTA reports: SMC, detailed advice document; ZIN, final recommendation document (in Dutch); and NICE, final appraisal document. In line with the inclusion criteria, the SMC detailed advice document and ZIN final recommendation document were the most recent and elaborate systematically public available reports describing the HTA and reimbursement recommendation. In England, more documents are publicly available, such as the scoping article and committee articles. The final appraisal document was included in this study because it was found that this was the most comprehensive document containing the scope, previously published updates, and committee discussions.

      Data Extraction

      Data were extracted using a predefined data extraction form constructed in Microsoft Excel (Microsoft Corporation, 2018, Redmond, Washington). This form included a (1) product section (eg, HTA report number, proprietary name, generic name and indication), (2) a considerations section, and (3) the reimbursement recommendation. In line with previous research, reimbursement recommendations were classified as positive, restricted (positive with conditions), or negative.
      • Vreman R.A.
      • Bouvy J.C.
      • Bloem L.T.
      • et al.
      Weighing of evidence by health technology assessment bodies: retrospective study of reimbursement recommendations for conditionally approved drugs.
      ,
      • Kleijnen S.
      • Lipska I.
      • Leonardo Alves T.
      • et al.
      Relative effectiveness assessments of oncology medicines for pricing and reimbursement decisions in European countries.
      Per report, considerations were identified and extracted. Duplicates were removed. Next, considerations were categorized into domains corresponding with an existing framework: the EUnetHTA JA2 – HTA core modelv3.0
      EUnetHTA joint action. HTA core model handbook.
      (hereafter referred to as EUnetHTA Core Model). The EUnetHTA Core Model is a methodological framework for creating and sharing of HTA information in the European context.
      EUnetHTA joint action. HTA core model handbook.
      • Allen N.
      • Liberti L.
      • Walker S.R.
      • Salek S.
      A comparison of reimbursement recommendations by European HTA agencies: is there opportunity for further alignment?.
      • Allen N.
      • Walker S.R.
      • Liberti L.
      • Salek S.
      Health technology assessment (HTA) case studies: factors influencing divergent HTA reimbursement recommendations in Australia, Canada, England, and Scotland.
      The model distinguishes the following domains in HTA: health problem and current use of technology (CUR), description and technical characteristics (TEC), clinical effectiveness (EFF), safety (SAF), cost- and economic effectiveness (ECO), ethical analysis (ETH), organizational aspects (ORG), patient and social aspects (SOC), and legal aspects (LEG). An “other (OTH)” domain was added to capture any considerations not covered by the predefined domains. More information about the EUnetHTA Core Model, domain definitions, and a description on how to categorize information (here considerations) into domains is described in detail elsewhere.
      EUnetHTA joint action. HTA core model handbook.
      , Figure 1 provides a schematic overview of the applied data extraction form and domains.
      Figure thumbnail gr1
      Figure 1Schematic overview of data extraction form and domain based on the EUnetHTA Core Model (JA2 – HTA core modelv3.0.).
      EUnetHTA joint action. HTA core model handbook.
      CUR indicates health problem and current use of technology; ECO, cost- and economic effectiveness; EFF, clinical effectiveness; ETH, ethical analysis; HTA, health technology assessment; LEG, legal aspects; ORG, organizational aspects; OTH, other; REC, reimbursement recommendation; SAF, safety; SOC, patient and social aspects; TEC, description and technical characteristics.

      Data Analysis

      Per jurisdiction, the extracted considerations were aggregated on a domain level, and indication-, disease-, and product-specific terminology generalized (eg, intervention, disease, survival, standard of care). Each categorized consideration was labeled as a supporting (pro) or opposing (contra) consideration. This resulted in the exclusion of neutral or factual statements. Domain categories and prolabels/contralabels were mutually exclusive. Using thematic content analysis, similar considerations were grouped together and their frequency of occurrence per jurisdiction was counted.
      • Le P.Y.H.
      • De Abreu Lourenco R.
      • Haas M.
      • van der Linden N.
      Key considerations in reimbursement decision-making for multiple sclerosis drugs in Australia.
      ,
      In a next step, key considerations were identified. A consideration was considered key if it was mentioned more than once in 2 different HTA reports within the same jurisdiction. Consequently, considerations that were identified once within a jurisdiction were deemed not key, but idiosyncratic or product or disease specific.
      A narrative analysis was then performed comparing key considerations between jurisdictions.
      • Le P.Y.H.
      • De Abreu Lourenco R.
      • Haas M.
      • van der Linden N.
      Key considerations in reimbursement decision-making for multiple sclerosis drugs in Australia.
      ,
      In addition, differences in key considerations between ATMPs that received a positive, restricted, and negative recommendations were described.
      Data extraction and analysis were conducted by one author (RTH). A second author (JH) validated data extraction and analysis by processing a sample of HTA reports (∼25%) including different types of products (eg, gene therapy medicinal product, cell therapy medicinal product), different jurisdictions, and different recommendations. This sample also included a product assessed in all 3 jurisdictions. Inconsistencies in data extraction and analysis were discussed until consensus was reached in accordance with the Cochrane Handbook for Systematic Reviews of Interventions.
      The Cochrane Collaboration:7.6.5
      Reliability and reaching consensus.
      Learnings from these discussions were applied to the analysis of the remaining data set.

      Results

      Identified ATMPs and HTA Reports

      The search of CAT monthly reports yielded 15 ATMPs for 16 indications (see Figure 2). After initial centralized MA, 4 ATMPs were withdrawn from the market by the developer and one product was suspended by the EMA.
      CAT monthly report of application procedures, guidelines and related documents on advanced therapies - June 2020 meeting. European Medicines Agency.
      In total, 18 HTA report were identified (Scotland, n = 5; The Netherlands, n = 5; England, n = 8). Of these, 3 issued a positive recommendation (Scotland, n = 0; The Netherlands, n = 1; England, n = 2), 10 a restricted recommendation (Scotland, n = 3; The Netherlands, n = 2; England, n = 5), and 5 a negative recommendation (Scotland, n = 2; The Netherlands, n = 2; England, n = 1). Figure 2 shows that 6 ATMPs were not assessed by any of the included HTA bodies, 5 ATMPs by one included HTA body, and 3 in all jurisdictions. A detailed overview of the included reports per HTA body is provided in Appendix Table 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.09.012.
      Figure thumbnail gr2
      Figure 2Identified health technology assessment reports and initial reimbursement recommendations of authorized advanced therapy medicinal products in Europe (June 2020). Negative recommendations (orange), Restricted recommendations (light green) and Positive recommendations (green).

      Identification of (Key) Consideration

      Figure 3 visualizes the data extraction and analysis process leading to the identification of key considerations. In total, 557 considerations (Scotland, n = 196; The Netherlands, n = 153; England, n = 208) were extracted from which 188 were identified as key (Scotland, n = 69; The Netherlands, n = 57; England, n = 65). A comprehensive list of key considerations per jurisdiction is provided as a supplement to this research in Appendix Tables 1 (Scotland), 2 (The Netherlands), and 3 (England) in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.09.012.
      Figure thumbnail gr3
      Figure 3Flow diagram of data extraction and analysis of (key) considerations. ∼ Data extraction steps were conducted on a product level.
      ATMP indicates advanced therapy medicinal product; CUR, health problem and current use of technology; ECO, cost- and economic effectiveness; EFF, clinical effectiveness; ETH, ethical analysis; LEG, legal aspects; NICE, National Institute for Health and Care Excellence; ORG, organizational aspects; SAF, safety; SMC, Scottish Medicines Consortium; SOC, patient and social aspects; TEC, description and technical characteristics; ZIN, National Health Care Institute.
      A visualization of key considerations per jurisdiction ordered by supporting (pro) and opposing (contra) considerations is shown in Figure 4. In The Netherlands and England, the EFF domain yielded most key considerations (both 37%). In Scotland, the EFF domain was considered less often (19%) than SOC (SOC domain, 26%) and ECO aspects (ECO domain, 22%).
      Figure thumbnail gr4
      Figure 4Distribution of identified key proconsideration and contraconsideration aggregated on a jurisdiction level.
      CUR indicates health problem and current use of technology; ECO, cost- and economic effectiveness; EFF, clinical effectiveness; ETH, ethical analysis; LEG, legal aspects; NICE, National Institute for Health and Care Excellence; ORG, organizational aspects; OTH, other; SAF, safety; SMC, Scottish Medicines Consortium; SOC, patient and social aspects; ZIN, National Health Care Institute.
      In the EFF domain, the key considerations supporting reimbursement recommendations were most often related to (relative) effectiveness (See Appendix Tables 1-3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.09.012). More specifically, in The Netherlands, a distinction was made between statistically significant and clinically relevant improvements. In England, demonstration of EFF together with incremental relative EFF was mentioned. In Scotland, key considerations entailed demonstration of significant benefit, often linked to participation in society (ie, patients resuming work, self-care, and social activities). A supporting key consideration mentioned by all jurisdictions expected clinical data as part of MA conditions or (ongoing) clinical trials. The contraconsiderations in the EFF domain concerned evidentiary uncertainties. Described uncertainties were due to single-arm studies (Scotland and England) and a lack of direct comparative data (The Netherlands and England). The Dutch reports also mentioned uncertainty of clinical effect because of low data quality, whereas English reports described the lack of comparative data as challenging in the same context.
      The ECO domain covered 19%, 16%, and 25% of key considerations for Scotland, The Netherlands, and England, respectively (Fig. 4). In Scotland, 2 supporting considerations were identified both concerning the presence of sensitivity analyses. Most Scottish ECO considerations were opposing reimbursement (13 of 15) describing uncertainties following from clinical data (primary outcome in the cost-effectiveness analysis differs from trial, use of proxy data), assumptions (cure and survival assumptions), and methods (indirect comparison, extrapolation, insufficient robust analysis, utility measurement). The SMC was the only HTA body that described service implications and financial risk because of high upfront cost. In The Netherlands, supporting considerations (4 of 9) in the ECO domain described expected limited budget impact. The opposing considerations (5 of 9) discussed expected high budget impact, insufficient methodological quality of analyses, and uncertainty associated with cure assumptions.
      Albeit small, England was the only jurisdiction where more supporting than opposing key considerations were identified in the ECO domain (9 of 16). Appreciation was explicitly expressed with compliance to NICE guidance and when developers provided additional data or additional (sensitivity) analyses after consultation. In addition, overlap in developer and external review group approaches was supporting reimbursement recommendations. In the assessment of Holoclar and Strimvelis, it explicitly reads that more uncertainty was accepted in the cost-effectiveness analysis given the small patient sample size. Opposing key considerations (7 of 16) showed similarities with other jurisdictions, detailing higher than considered plausible incremental cost-effectiveness ratios (ICERs), and uncertainties in survival extrapolation, cost, and cure assumptions. It is of note that drug price and budget impact information was not disclosed in Scottish and English HTA reports because of confidentiality agreements.
      The SOC domain yielded most key considerations in Scotland (26%). One reason for this is that all Scottish HTA reports included a section describing considerations from Patient and Clinical Engagement meetings
      Patient and Clinician Engagement (PACE) meeting overview. Healthcare Improvement Sctoland.
      (see Appendix Table 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.09.012). A Patient and Clinical Engagement meeting can be requested by a submitting party for medicines treating (ultra-)orphan and end-of-life indications. Its aim is to give patient groups and clinicians a stronger voice in SMC decision making. The supporting key considerations in the Scottish assessments included impact of the intervention and standard of care, the patient and carer perspective, physical and mental wellbeing, and participation in society (see Appendix Table 2 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.09.012). Two key considerations opposing a positive reimbursement recommendation were unknown long-term effect and significant initial monitoring after treatment. In the Dutch HTAs, 12% of key considerations were identified in the SOC domain (see Appendix Table 3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.09.012). Few reports included discussions from the Insured Package Advisory Committee (Adviescommissie Pakket). The Insured Package Advisory Committee supports the Dutch HTA body in considerations that may affect the society. The supporting key considerations included ease of use, single administration, and halting of disease progression. For 4 products, ease of use explicitly was described to be attributed less weight with increasing disease severity; in 2 products, ease of use was considered irrelevant because of the life-threatening nature of the disease. In England, 5 of the 6 key considerations (9%) describe the impact of the condition on various aspects of patients’ life (see Appendix Table 4 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.09.012). No opposing key considerations in the SOC domain were identified.
      Key considerations related to LEG domain, CUR domain, and ETH domain overlapped and were often observed simultaneously, determining specific conditions applied in the assessment. These considerations mainly revolved around orphan designations and the presence of unmet medical need. In Scotland, orphan medicinal products were described to be eligible for SMC orphan criteria and assessment under the orphan framework. This allowed for greater uncertainty in the economic domain. In The Netherlands, mentioning an unmet medical need and high disease burden in CUR domain was often associated with mention of an orphan indication (ETH domain). In England, a similar observation was made where unmet medical need was described alongside limited treatment options (CUR domain). Two ATMPs in England were classified as ultraorphan conditions (ETH domain), which seemed to have implications for acceptance of uncertainty in the assessment (LEG domain). Scottish and English reports also mentioned the use of end-of-life criteria, but consequences of the application of end-of-life criteria could not be derived from the HTA reports.
      An additional section in 3 English reports mentions that ICER estimates are highly uncertain and often higher than what NICE considers acceptable. Therefore, these products could not be recommended for use in the National Health Service. Nevertheless, the HTA body recommends that the uncertainties can be addressed with additional data collection. Therefore, these products were recommended for use under the Cancer Drug Fund (CDF), which is an interim-funded managed entry agreement.
      Appraisal and funding of cancer drugs from July 2016 (including the new Cancer Drugs Fund) A new deal for patients, taxpayers and industry. NHS England.
      The 3 products received a restricted reimbursement recommendation. To add, whether the use of the CDF would also have been granted to non- or different oncologic agents or to oncologic agents that did not meet end-of-life criteria could not be derived from this sample.
      The ORG domain covered 6%, 7%, and 6% of total key considerations in Scotland, The Netherlands, and England, respectively. These considerations were all opposing positive recommendations for reimbursement. In The Netherlands, limited experience and administration in specialized centers were emphasized, whereas in Scotland, a need for appropriate centers with experienced staff was described. In England, a need was expressed for staff training to treat and handle adverse events, and only allowing administration in specialist centers was considered.

      Positive, Restricted, and Negative Recommendations

      In the Dutch reports, the percentage of supporting key considerations resulting in a positive, restricted, and negative recommendation was 53%, 52%, and 41%, respectively (Fig. 5). In the English reports, these percentages were 77%, 51%, and 43%. In Scotland, no positive recommendations were identified and supporting key considerations for restricted and negative recommendations were similar (55% vs 53%). Observations suggest that a higher percentage of supporting considerations do not necessarily result in a positive recommendation and vice versa.
      Figure thumbnail gr5
      Figure 5Key considerations stratified by reimbursement recommendation. In Scotland, no positive recommendations were identified.
      CUR indicates health problem and current use of technology; ECO, cost- and economic effectiveness; EFF, clinical effectiveness; ETH, ethical analysis; LEG, legal aspects; NICE, National Institute for Health and Care Excellence; ORG, organizational aspects; OTH, other; SAF, safety; SMC, Scottish Medicines Consortium; SOC, patient and social aspects; ZIN, National Health Care Institute.

      Role of Uncertainties and Mitigating Factors

      Uncertainties and how to address them were important themes in several domains. Therefore, we separately discuss findings on uncertainties in this section and link it to reimbursement recommendations. HTAs in which identified uncertainties were addressed by the developer seemed more likely to receive a positive or restricted recommendation. Opposing key considerations seemed to be weighed less if the issue was addressed or demonstrated little impact (see Appendix Tables 1-3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.09.012). In the reports, opposing considerations were addressed via quantification (additional sensitivity analyses), providing context for choice of methods (no or little data available), or inclusion of alternative (eg, historic, real-world, literature) evidence. In addition, if addressed uncertainties demonstrated high impact on the ICER, additional measures were put in place or requested. Examples were risk mitigation plans, additional trials, trainings, and development of materials for clinicians and patients. The additional clinical trials could be company initiated or those mandated by the EMA (eg, conditional MA or postauthorization safety studies). Other recommended studies by the HTA bodies included setup of registries or addition of country-specific effectiveness and quality of life measures to existing trials or registries.
      Not all uncertainties could be addressed at time of the assessments. For example, a lack of long-term effectiveness and safety evidence or retreatment data were described as highly uncertain. Merely the prospect of additional clinical evidence seemed to somewhat contribute to the acceptance of uncertainties regarding long-term effectiveness, safety, and retreatment. Addressing residuary uncertainties, in some cases, were found to appear in the conditions of restricted recommendations.

      Discussion

      In this review, key considerations were identified and assessed in HTA reports of ATMPs in 3 European (EU) jurisdictions: Scotland, The Netherlands, and England. The considerations were categorized into EUnetHTA Core Model domains.
      EUnetHTA joint action. HTA core model handbook.
      In The Netherlands and England, most key considerations were identified in the EFF and ECO domains (The Netherlands, 37% and 16%; England, 37% and 25%, respectively). In Scotland, the SOC domain yielded most key considerations (26%), followed by the ECO and EFF domain (22% and 19%). We also observed that the LEG, CUR, and ETH domains imposed assessment conditions via orphan or end-of-life criteria, which allowed for acceptance of more uncertainty in decision making. Finally, a trend was observed in the ratio between supporting and opposing key considerations, with a higher percentage supporting key considerations identified in positive recommendations and lower percentages in restricted and negative recommendations. Nevertheless, a higher percentage supporting key considerations did not always lead to a positive recommendation, suggesting that different weights may be attributed to individual key considerations. Based on the narrative analysis, a reason for this observation could be that considerations and domains are given different weight in the assessment. In addition, individual key considerations could be weighed more (clinically relevant effect vs statistically significant effect) or different under circumstances. An example of the latter was described previously in the case of insignificance of ease of use in the treatment of life-threatening indications. Unfortunately, the study sample was too small to test, rank, and substantiate these hypotheses.
      Although previous studies have explored several specific considerations in HTA reports, these studies did not include ATMPs.
      • Vreman R.A.
      • Bouvy J.C.
      • Bloem L.T.
      • et al.
      Weighing of evidence by health technology assessment bodies: retrospective study of reimbursement recommendations for conditionally approved drugs.
      ,
      • Le P.Y.H.
      • De Abreu Lourenco R.
      • Haas M.
      • van der Linden N.
      Key considerations in reimbursement decision-making for multiple sclerosis drugs in Australia.
      Nevertheless, similarities are observed between our findings and research addressing HTA of orphan products and conditional MA. First, Vreman et al
      • Vreman R.A.
      • Bouvy J.C.
      • Bloem L.T.
      • et al.
      Weighing of evidence by health technology assessment bodies: retrospective study of reimbursement recommendations for conditionally approved drugs.
      found that, based on similar evidence, HTA agencies within different jurisdictions formulate different recommendations. This has been described previously and is said to be caused by application of different HTA frameworks and conditions.
      • Kleijnen S.
      • Lipska I.
      • Leonardo Alves T.
      • et al.
      Relative effectiveness assessments of oncology medicines for pricing and reimbursement decisions in European countries.
      ,
      • Allen N.
      • Liberti L.
      • Walker S.R.
      • Salek S.
      A comparison of reimbursement recommendations by European HTA agencies: is there opportunity for further alignment?.
      Despite our small sample, which is due to the low number of authorized products that have undergone HTA, different recommendations for the same product were indeed observed between jurisdictions. In addition, applications of country-specific conditions (eg, end-of-life or orphan criteria) affected weighing of evidence in specific domains. Next, a previous study also suggested that demonstrated statistical or clinical significant benefit of orphan products may positively drive assessment of relative effectiveness in HTA.
      • Vreman R.A.
      • de Ruijter A.S.
      • Zawada A.
      • et al.
      Assessment of significant benefit for orphan medicinal products by European regulators may support subsequent relative effectiveness assessments by health technology assessment organizations.
      This is in line with our observation that more uncertainty was accepted in the assessment when (unmet) medical need was demonstrated.
      In literature, it is argued that ATMPs are perceived as considerably different from more established medicinal products, such as small molecules or biologicals.
      • Buckland K.F.
      • Bobby Gaspar H.
      Gene and cell therapy for children - new medicines, new challenges?.
      ,
      • Coppens D.G.M.
      Regulating gene and cell-based therapies as medicinal products. Utrecht University Repositor.
      In line, recent literature has questioned whether existing HTA frameworks are fit to assess one-off expensive therapies.
      • Coyle D.
      • Durand-Zaleski I.
      • Farrington J.
      • et al.
      HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies.
      ,
      • Angelis A.
      • Naci H.
      • Hackshaw A.
      Recalibrating health technology assessment methods for cell and gene therapies.
      Although out of the primary scope of this study, the specific technical characteristics and supply chain challenges were to some extent reflected in the key considerations regarding the technical and organizational domain. Upfront and high prices are often mentioned as a main challenge in HTA.
      • Hanna E.
      • Tavella F.
      • Rémuzat C.
      • Auquier P.
      • Toumi M.
      Market access of Atmps: overview and expected challenges.
      ,
      • Morrow D.
      • Ussi A.
      • Migliaccio G.
      Addressing pressing needs in the development of advanced therapies.
      ,
      • Coyle D.
      • Durand-Zaleski I.
      • Farrington J.
      • et al.
      HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies.
      , In our research, only Scottish reports described service implications and financial risk because of high upfront cost in the ECO domain. In other domains, identified key considerations show overlap with challenges previously described in the assessment of orphan products and nonrandomized evidence.
      • Kanters T.A.
      Health technology assessment of orphan drugs: the example of Pompe disease.
      • Griffiths E.A.
      • Macaulay R.
      • Vadlamudi N.K.
      • Uddin J.
      • Samuels E.R.
      The role of noncomparative evidence in health technology assessment decisions.
      • Phillippo D.M.
      • Dias S.
      • Elsada A.
      • Ades A.E.
      • Welton N.J.
      Population adjustment methods for indirect comparisons: a review of National Institute for Health and Care Excellence technology appraisals.
      In line with our results, the NICE mentioned in the HTA of Holoclar a collaboration with the Centre for Reviews and Dissemination and Centre for Health Economics, University of York.
      • Hettle R.
      • Corbett M.
      • Hinde S.
      • et al.
      The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal.
      This collaboration yielded a report that investigated whether NICE’s assessment and appraisal methods were fit for purpose for regenerative medicines and cell therapies. It concluded that the appraisal methods and decision framework was largely applicable but individual elements may need adjustment. Several studies describe similar findings.
      • Coyle D.
      • Durand-Zaleski I.
      • Farrington J.
      • et al.
      HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies.
      ,
      • Angelis A.
      • Naci H.
      • Hackshaw A.
      Recalibrating health technology assessment methods for cell and gene therapies.
      ,
      • ten Ham R.M.T.
      • Klungel O.H.
      • Leufkens H.G.M.
      • Frederix G.W.J.
      A review of methodological considerations for economic evaluations of gene therapies and their application in literature.
      ,
      • Jørgensen J.
      • Kefalas P.
      Annuity payments can increase patient access to innovative cell and gene therapies under England’s net budget impact test.
      Therefore, an apparent misfit between the HTA framework and assessed ATMPs could not be derived from our results. This can be caused by the notion that this study was not designed to answer this research question, that the perceived misfit is not as evident in practice as portrayed in the literature, or that misfits experienced in assessment practices were not made explicit in the resulting HTA reports. More research is needed to inform this discussion.
      Payment models have also been discussed as solutions to address both high cost and evidentiary uncertainties.
      • Jørgensen J.
      • Kefalas P.
      Annuity payments can increase patient access to innovative cell and gene therapies under England’s net budget impact test.
      ,
      • Barlow J.
      • Bens C.
      • Brent T.
      • et al.
      AMCP Partnership Forum. Designing benefits and payment models for innovative high-investment medications.
      ,
      • Drummond M.
      • Towse A.
      Is rate of return pricing a useful approach when value-based pricing is not appropriate?.
      The use of such models (ie, annuity payment, outcome-based payment, or coverage with evidence development) was not widely observed in the included assessments.
      • Drummond M.
      • Towse A.
      Is rate of return pricing a useful approach when value-based pricing is not appropriate?.
      ,
      • Hanna E.
      • Toumi M.
      • Dussart C.
      • et al.
      Funding breakthrough therapies : a systematic review and recommendation.
      Application of the CDF in England (classified as a managed entry agreement with the commitment to re-evaluate clinical and cost-effectiveness every 3 years after further data collection) and the orphan drug agreement in The Netherlands (a form of Coverage with Evidence Generation combined with price reductions) can be seen as payment agreements allowing early patient access while additional data are collected. Application of novel and more advanced statistical methods was not observed, neither were novel value elements considered.
      • ten Ham R.M.T.
      • Klungel O.H.
      • Leufkens H.G.M.
      • Frederix G.W.J.
      A review of methodological considerations for economic evaluations of gene therapies and their application in literature.
      ,
      • Drummond M.F.
      • Neumann P.J.
      • Sullivan S.D.
      • et al.
      Analytic considerations in applying a general economic evaluation reference case to gene therapy.
      ,
      • Jönsson B.
      Economic evaluation of health care technologies.
      To the best of our knowledge, we are the first to use the EUnetHTA model to categorize key considerations in existing HTA reports retrospectively. Application of the EUnetHTA Core Model in HTA research is not new.
      • Vettoretto N.
      • Foglia E.
      • Ferrario L.
      • et al.
      Could fluorescence-guided surgery be an efficient and sustainable option? A SICE (Italian Society of Endoscopic Surgery) health technology assessment summary.
      • Radaelli G.
      • Lettieri E.
      • Masella C.
      • Merlino L.
      • Strada A.
      • Tringali M.
      Implementation of eunethta core model® in lombardia: the VTS framework.
      • Brenna E.
      • Polistena B.
      • Spandonaro F.
      The implementation of health technology assessment principles in public decisions concerning orphan drugs.
      Although technologies, jurisdictions, and purposes differ among studies, the use of similar terminology and definitions increases transferability and dissemination of HTA practices. In a previous research, Radaelli et al
      • Radaelli G.
      • Lettieri E.
      • Masella C.
      • Merlino L.
      • Strada A.
      • Tringali M.
      Implementation of eunethta core model® in lombardia: the VTS framework.
      described that issues identified based on the EUnetHTA workflow resembled questions that needed to be addressed in the assessment. Therefore, application of the model allows for breakdown of the assessment in smaller units that can be assigned to experts for further clarification.
      • Radaelli G.
      • Lettieri E.
      • Masella C.
      • Merlino L.
      • Strada A.
      • Tringali M.
      Implementation of eunethta core model® in lombardia: the VTS framework.
      In our research, the breakdown into smaller units provided insight in the content of discussions underlying the reimbursement recommendations. When categorizing considerations into the EUnetHTA domains, overlap in domain descriptions was observed, especially in the ETH, SOC, and CUR domains. This issue has previously been described in literature.
      • Facey K.
      • Ørtenblad L.
      • Scalzo A.L.
      Patient and social aspects domain.
      The OTH domain, which was added by the authors to capture considerations not captured by the EUnetHTA Core Model, yielded one key consideration in Scotland and none in The Netherlands and England. This may suggest that the EUnetHTA Core Model framework may be appropriate for the intended purpose of this research.
      Our research has several limitations. First, the included reports represent a written summary of the HTAs and discussions. We realize that it is likely that not all considerations were included in the reports. In addition, details may have been lost including, but not limited to, an understanding of the weight of individual arguments and issues encountered in assessment practices. Second, our definition of (key) considerations may be subject to discussion. A different definition may yield different or more/less key considerations. Third, it is important to realize that our approach is sensitive to time as health technology policies and assessment change over time. Nevertheless, the approach can also be used to make the effect of policy changes over time visible—for instance, by studying the effect of the introduction of the CDF in England on the importance of certain domains in assessment and the weights given to them. In addition, several ATMPs have been withdrawn from the market, making it highly unlikely that these products will undergo HTA in the near future and halting patient access. Fourth, the high incidence of CAR-T products (9 of 18) may have led to identification of key considerations reflecting benefits and hindrances that are biased toward these specific therapies and less representative of ATMPs in general. Therefore, the results of this study should be considered as a timely snapshot of assessments within a rapidly evolving field. Fifth, analysis and interpretation of qualitative research in general are sensitive to bias.
      This is a limitation inherent to the type of research. To decrease bias and increase external validity, data extraction and analysis were conducted by 2 authors. This approach is highly recommended and considered good research practice in qualitative research.
      ,
      • Leung L.
      Validity, reliability, and generalizability in qualitative research.
      Sixth, the included HTA bodies are located in Western Europe and known to be quite advanced and similar in their HTA approaches.
      • Allen N.
      • Liberti L.
      • Walker S.R.
      • Salek S.
      A comparison of reimbursement recommendations by European HTA agencies: is there opportunity for further alignment?.
      ,
      • Allen N.
      • Pichler F.
      • Wang T.
      • Patel S.
      • Salek S.
      Development of archetypes for non-ranking classification and comparison of European National Health Technology Assessment systems.
      Therefore, replication of this research in jurisdictions with less alike HTA processes is necessary and might incite adjustment of methodology. Seventh, our study does not include a comparator cohort, which has consequences for the interpretation of our results. For example, if alternative assessment conditions were not applied, would the reimbursement recommendations be different? In line, at the time this analysis was conducted, little empirical evidence was available to contextualize our findings. Although this can be seen as a limitation, it also demonstrates the novelty of the approach and products. It would be of interest to contextualize the findings in future research by replicating our approach in more jurisdictions and medicinal product groups. In addition, more in-depth qualitative research could be conducted via interview-based studies within HTA bodies. This may result in learnings on how HTA agencies accommodate novel technologies and address uncertainties in a rapidly changing context, contributing to assessment of so-called organizational learning and organizational readiness.
      • Uzkurt C.
      • Kumar R.
      • Ensari N.
      Assessing organizational readiness for innovation: an exploratory study on organizational characteristics of innovativeness.
      ,
      • Rashman L.
      • Withers E.
      • Hartley J.
      Organizational learning and knowledge in public service organizations: A systematic review of the literature.

      Conclusions

      This study used the EUnetHTA Core Model to identify and structure key consideration in HTA of ATMPs in 3 EU jurisdictions. We found some ATMP-specific considerations, but also observed that most identified key considerations were similar to known considerations for orphan medicines and conditional approved products. We found that considerations outside the common described effectiveness and cost-effectiveness domains, including ETH and LEG may bare considerable weight in formulation of reimbursement recommendations. In addition, specific criteria (eg, orphan or end-of-life) may alter assessment conditions. Additional research is needed to explain variation in considerations and recommendations in more detail and allow for comparison with other medicinal product groups and jurisdictions.

      Article and Author Information

      Author Contributions: Concept and design: ten Ham, Frederix, Wu, Hoekman
      Acquisition of data: ten Ham, Frederix, Hoekman
      Analysis and interpretations of data: ten Ham, Frederix, Wu, Goettsch, Hoekman
      Drafting of the manuscript: ten Ham, Frederix, Wu, Goettsch, Leufkens, Klungel, Hoekman
      Critical revision of paper for important intellectual content: ten Ham
      Statistical analysis: ten Ham, Hoekman
      Administrative, technical, or logistic support: Fredrix, Wu, Goettsch, Leufkens, Klungel
      Supervision: Fredrix, Wu, Leufkens, Klungel, Hoekman
      Conflict of Interest Disclosures: Dr. Wu is an editor of Value in Health and had no role in the peer review process of this article. No other disclosures were reported.
      Funding/Support: The authors received no financial support for this research.

      Acknowledgment

      The authors thank Parinaz Goodarzyfard for her contribution to the design of this study.

      Supplemental Material

      References

        • Eder C.
        • Wild C.
        Technology forecast : advanced therapies in late clinical research, EMA approval or clinical application via hospital exemption.
        J Mark Access Heal Policy. 2019; 71600939
      1. The European Parliament, the Council of the European Union. Regulation (EC) no 1394/2007 of the European Parliament and of the council.
        Off J Eur Union. 2007; 324: 121-138
      2. CAT monthly report of application procedures, guidelines and related documents on advanced therapies - June 2020 meeting. European Medicines Agency.
        (Accessed August 27, 2021)
        • Hanna E.
        • Tavella F.
        • Rémuzat C.
        • Auquier P.
        • Toumi M.
        Market access of Atmps: overview and expected challenges.
        Value Healh. 2015; 18: A518-A519
        • Morrow D.
        • Ussi A.
        • Migliaccio G.
        Addressing pressing needs in the development of advanced therapies.
        Front Bioeng Biotechnol. 2017; 5: 1-6
        • Ermisch M.
        • Bucsics A.
        • Bonanno P.V.
        • et al.
        Payers’ views of the changes arising through the possible adoption of adaptive pathways.
        Front Pharmacol. 2016; 7: 1-9
        • Coyle D.
        • Durand-Zaleski I.
        • Farrington J.
        • et al.
        HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies.
        Eur J Heal Econ. 2020; 2: 1421-1437
        • Vreman R.A.
        • Bouvy J.C.
        • Bloem L.T.
        • et al.
        Weighing of evidence by health technology assessment bodies: retrospective study of reimbursement recommendations for conditionally approved drugs.
        Clin Pharmacol Ther. 2018; 105: 684-691
        • Vreman R.A.
        • Naci H.
        • Goettsch W.G.
        • et al.
        Decision making under uncertainty: comparing regulatory and health technology assessment reviews of medicines in the United States and Europe.
        Clin Pharmacol Ther. 2020; 108: 350-357
        • Towse A.
        • Fenwick E.
        Uncertainty and cures: discontinuation, irreversibility, and outcomes-based payments: what is different about a one-off treatment?.
        Value Healh. 2019; 22: 677-683
        • Angelis A.
        • Naci H.
        • Hackshaw A.
        Recalibrating health technology assessment methods for cell and gene therapies.
        Pharmacoeconomics. 2020; 38: 1297-1308
        • ten Ham R.M.T.
        • Klungel O.H.
        • Leufkens H.G.M.
        • Frederix G.W.J.
        A review of methodological considerations for economic evaluations of gene therapies and their application in literature.
        Value Healh. 2020; 23: 1268-1280
        • Petrou P.
        Is it a chimera? A systematic review of the economic evaluations of CAR-T cell therapy.
        Expert Rev Pharmacoecon Outcomes Res. 2019; 19: 529-536
        • Rozier P.
        • Maria A.
        • Goulabchand R.
        • Jorgensen C.
        • Guilpain P.
        • Noël D.
        Mesenchymal stem cells in systemic sclerosis: allogenic or autologous approaches for therapeutic use?.
        Front Immunol. 2018; 9: 2938
        • Gonçalves E.
        Advanced therapy medicinal products: value judgement and ethical evaluation in health technology assessment.
        Eur J Heal Econ. 2020; 21: 311-320
        • Carr D.R.
        • Bradshaw S.E.
        Gene therapies: the challenge of super-high-cost treatments and how to pay for them.
        Regen Med. 2016; 11: 381-393
        • Jørgensen J.
        • Kefalas P.
        Annuity payments can increase patient access to innovative cell and gene therapies under England’s net budget impact test.
        J Mark Access Heal Policy. 2017; 51355203
        • Barlow J.
        • Bens C.
        • Brent T.
        • et al.
        AMCP Partnership Forum. Designing benefits and payment models for innovative high-investment medications.
        J Manag Care Spec Pharm. 2019; 25: 156-162
        • Jørgensen J.
        • Kefalas P.
        Reimbursement of licensed cell and gene therapies across the major European healthcare markets.
        J Mark Access Heal Policy. 2015; 3: 29321
        • Hanna E.
        • Rémuzat C.
        • Auquier P.
        • Toumi M.
        Advanced therapy medicinal products: current and future perspectives.
        J Mark Access Heal Policy. 2016; 4: 31036
      3. Bloem LT, Vreman RA, van Oirschot S, et al. The role of regulator-imposed post-approval studies in health technology assessments for conditionally approved drugs [published online October 27, 2020]. Int J Heal Policy Manag. https://doi.org/10.34172/ijhpm.2020.198.

        • Lipska I.
        • Hoekman J.
        • McAuslane N.
        • Leufkens H.G.M.
        • Hövels A.M.
        Does conditional approval for new oncology drugs in Europe lead to differences in health technology assessment decisions?.
        Clin Pharmacol Ther. 2015; 98: 489-491
        • Kleijnen S.
        • Lipska I.
        • Leonardo Alves T.
        • et al.
        Relative effectiveness assessments of oncology medicines for pricing and reimbursement decisions in European countries.
        Ann Oncol. 2016; 27: 1768-1775
      4. EUnetHTA joint action. HTA core model handbook.
        EUnetHTA JA. 2011; 2: 1-14
        • Allen N.
        • Liberti L.
        • Walker S.R.
        • Salek S.
        A comparison of reimbursement recommendations by European HTA agencies: is there opportunity for further alignment?.
        Front Pharmacol. 2017; 8: 384
        • Allen N.
        • Walker S.R.
        • Liberti L.
        • Salek S.
        Health technology assessment (HTA) case studies: factors influencing divergent HTA reimbursement recommendations in Australia, Canada, England, and Scotland.
        Value Healh. 2017; 20: 320-328
      5. The HTA Core Model® guiding principles on use. eunethta.
        (Accessed August 27, 2021)
        • Le P.Y.H.
        • De Abreu Lourenco R.
        • Haas M.
        • van der Linden N.
        Key considerations in reimbursement decision-making for multiple sclerosis drugs in Australia.
        Mult Scler Relat Disord. 2018; 25: 144-149
      6. Bailey A. Hutter I. Hutter I. Qualitative Research Methods. 1st ed. SAGE Publications, Inc, Thousands Oak, CA2010
        • The Cochrane Collaboration:7.6.5
        Reliability and reaching consensus.
        in: Higgens J. Green S. Cochran Handbook for Systematic Reviews of Interventions. Wiley-Blackwell, Hoboken, NJ2011: 694
        Version: Version 5
      7. Patient and Clinician Engagement (PACE) meeting overview. Healthcare Improvement Sctoland.
        (Accessed August 27, 2021)
      8. Appraisal and funding of cancer drugs from July 2016 (including the new Cancer Drugs Fund) A new deal for patients, taxpayers and industry. NHS England.
        • Vreman R.A.
        • de Ruijter A.S.
        • Zawada A.
        • et al.
        Assessment of significant benefit for orphan medicinal products by European regulators may support subsequent relative effectiveness assessments by health technology assessment organizations.
        Drug Discov Today. 2020; 25: 1223-1231
        • Buckland K.F.
        • Bobby Gaspar H.
        Gene and cell therapy for children - new medicines, new challenges?.
        Adv Drug Deliv Rev. 2014; 73: 162-169
        • Coppens D.G.M.
        Regulating gene and cell-based therapies as medicinal products. Utrecht University Repositor.
        (Accessed August 27, 2021)
        • Kanters T.A.
        Health technology assessment of orphan drugs: the example of Pompe disease.
        • Griffiths E.A.
        • Macaulay R.
        • Vadlamudi N.K.
        • Uddin J.
        • Samuels E.R.
        The role of noncomparative evidence in health technology assessment decisions.
        Value Healh. 2017; 20: 1245-1251
        • Phillippo D.M.
        • Dias S.
        • Elsada A.
        • Ades A.E.
        • Welton N.J.
        Population adjustment methods for indirect comparisons: a review of National Institute for Health and Care Excellence technology appraisals.
        Int J Technol Assess Health Care. 2019; 35: 221-228
        • Hettle R.
        • Corbett M.
        • Hinde S.
        • et al.
        The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal.
        Health Technol Assess (Rockv). 2017; 21: 1-204
        • Drummond M.
        • Towse A.
        Is rate of return pricing a useful approach when value-based pricing is not appropriate?.
        Eur J Heal Econ. 2019; 20: 945-948
        • Hanna E.
        • Toumi M.
        • Dussart C.
        • et al.
        Funding breakthrough therapies : a systematic review and recommendation.
        Health Policy (New York). 2018; 122: 217-229
        • Drummond M.F.
        • Neumann P.J.
        • Sullivan S.D.
        • et al.
        Analytic considerations in applying a general economic evaluation reference case to gene therapy.
        Value Healh. 2019; 22: 661-668
        • Jönsson B.
        Economic evaluation of health care technologies.
        Encycl Public Heal. 2008; 128 (316-316)
        • Vettoretto N.
        • Foglia E.
        • Ferrario L.
        • et al.
        Could fluorescence-guided surgery be an efficient and sustainable option? A SICE (Italian Society of Endoscopic Surgery) health technology assessment summary.
        Surg Endosc. 2020; 34: 3270-3284
        • Radaelli G.
        • Lettieri E.
        • Masella C.
        • Merlino L.
        • Strada A.
        • Tringali M.
        Implementation of eunethta core model® in lombardia: the VTS framework.
        Int J Technol Assess Health Care. 2014; 30: 105-112
        • Brenna E.
        • Polistena B.
        • Spandonaro F.
        The implementation of health technology assessment principles in public decisions concerning orphan drugs.
        Eur J Clin Pharmacol. 2020; 76: 755-764
        • Facey K.
        • Ørtenblad L.
        • Scalzo A.L.
        Patient and social aspects domain.
        EUnetHTA HTA Core Model v3.0. The Netherlands, Diemen2016: 411
        • Leung L.
        Validity, reliability, and generalizability in qualitative research.
        J Fam Med Prim Care. 2015; 4: 324-327
        • Allen N.
        • Pichler F.
        • Wang T.
        • Patel S.
        • Salek S.
        Development of archetypes for non-ranking classification and comparison of European National Health Technology Assessment systems.
        Health Policy (New York). 2013; 113: 305-312
        • Uzkurt C.
        • Kumar R.
        • Ensari N.
        Assessing organizational readiness for innovation: an exploratory study on organizational characteristics of innovativeness.
        Int J Innov Technol Manag. 2013; 101350018
        • Rashman L.
        • Withers E.
        • Hartley J.
        Organizational learning and knowledge in public service organizations: A systematic review of the literature.
        Int J Manag Rev. 2009; 11: 463-494