Advertisement

A Value Framework for the Assessment of Diagnostic Technologies: A Proposal Based on a Targeted Systematic Review and a Multistakeholder Deliberative Process in Latin America

Open AccessPublished:January 15, 2021DOI:https://doi.org/10.1016/j.jval.2020.11.008

      Highlights

      • Value frameworks are frequently used to improve transparency and assist healthcare decision-making processes by signaling the important dimensions to assess when evaluating different technologies. Very few are targeted to assess diagnostic technologies and their specificities or are representative of a multistakeholder perspective.
      • We propose a VF targeted broadly to diagnostic technologies, based on a targeted systematic review and a participatory process with key HTA stakeholders in Latin America. It is the first to be produced in a lower and middle income setting but can also be potentially useful in other contexts aimed to assist decision-making processes in this complex scenario.

      Abstract

      Objectives

      there are very few value frameworks (VFs) to assess health technologies that are focused on diagnostic tests; they usually do not reflect a multistakeholder process; and they are all developed in high-income countries. Our project performed a targeted systematic review, with the objective of proposing an evidence-based, up-to-date VF informed by a multinational multistakeholder group working in the health technology assessment (HTA) space.

      Methods

      (1) A targeted systematic review, with the aim to identify existing VFs and their dimensions; and (2) generation a VF proposal through a mixed-methods, qualitative-quantitative approach.

      Results

      From 73 citations identified, 20 met our inclusion criteria and served to provide the initial list of dimensions for our VF. An initial list of criteria and subcriteria for a preliminary VF was proposed. After a full-day deliberative face-to-face meeting with 30 relevant stakeholders from seven Latin American countries and the United Kingdom, the final VF was defined, consisting of 15 criteria: five “essential or core,” six highly relevant, three moderately relevant, and one of low relevance. Barriers and facilitators of value assessment of diagnostic technologies were also discussed.

      Conclusions

      We propose a VF oriented to diagnostic technologies based on a targeted systematic review and a participatory process with key HTA stakeholders. It is the first to be produced in a lower and middle income setting but can also be potentially useful in other contexts aimed to assist decision-making processes with these particularly complex health technologies.

      Introduction

      Diagnostic tests are a heterogeneous group of technologies with significant complexity in several relevant aspects or dimensions that decision makers need to assess when integrating their value.
      • Garrison L.P.
      • Towse A.
      A strategy to support efficient development and use of innovations in personalized medicine and precision medicine.
      ,
      • Lee D.W.
      • Neumann P.J.
      • Rizzo J.A.
      Understanding the medical and nonmedical value of diagnostic testing.
      Among the different aspects to bear in mind when assessing a diagnostic technology, some of the most important are not only the diagnostic ability of the test or the process it entails, but also its eventual ability to improve clinical decision making, the choice of treatments, and the final patient relevant outcomes. In this way, the test is usually considered within the entire patient care process or pathway. It is in this context that the value of a diagnostic test or diagnostic technology is reflected, through the value of the information generated for multiple aspects related to health and healthcare, such as the selection of a treatment, the determination of a state of health, the prognostic information for the patient or their offspring, the monitoring of a treatment, or the personalization of a therapeutic scheme.
      • Akhmetov I.
      • Bubnov R.V.
      Assessing value of innovative molecular diagnostic tests in the concept of predictive, preventive, and personalized medicine.
      • Garau M.
      • Towse A.
      • Garrison L.
      • Housman L.
      • Ossa D.
      Can and should value-based pricing be applied to molecular diagnostics?.
      National Institute for Health and Care Excellence
      Diagnostics Assessment Programme Manual.
      In a context where decision makers in health systems traditionally focus on drugs and prioritize dimensions such as efficacy, safety, and costs when evaluating a health technology to be covered or financed, diagnostic technologies impose significant challenges for assessment.
      • Smith A.F.
      • Shinkins B.
      • Hall P.S.
      • Hulme C.T.
      • Messenger M.P.
      Toward a framework for outcome-based analytical performance specifications: a methodology review of indirect methods for evaluating the impact of measurement uncertainty on clinical outcomes.
      They may be assessed in a rather partial or suboptimal way when using the same assessment criteria as drugs, with the consequent misjudgment of their value (either in a positive or a negative way) when deciding about their incorporation in a benefits package or healthcare system.
      • Wurcel V.
      • Cicchetti A.
      • Garrison L.
      • et al.
      The value of diagnostic information in personalised healthcare: a comprehensive concept to facilitate bringing this technology into healthcare systems.
      • Faulkner E.
      • Holtorf A.P.
      • Walton S.
      • et al.
      Being precise about precision medicine: what should value frameworks incorporate to address precision medicine? a report of the personalized precision medicine special interest group.
      • Garfield S.
      • Polisena J.
      • Spinner D.S.
      • et al.
      Health technology assessment for molecular diagnostics: practices, challenges, and recommendations from the medical devices and diagnostics special interest group.
      • Govaerts L.
      • Simoens S.
      • Van Dyck W.
      • Huys I.
      Health policy analysis shedding light on reimbursement policies of companion diagnostics in European countries.
      The coronavirus disease 2019 (COVID-19) outbreak challenges health systems from the first line of assistance to the ones that have to make strategic and coverage decisions.
      • Emanuel E.J.
      • Persad G.
      • Upshur R.
      • et al.
      Fair allocation of scarce medical resources in the time of Covid-19.
      The adoption of testing strategies implies the consideration of a broad spectrum of factors, many of which involve a high degree of uncertainty. In this particularly pressing example, a framework that makes visible the factors that decision makers have to consider in the assessment of a diagnostic technology can serve as a compass in the decision-making process. When writing this article, there is still not a specific treatment or vaccine, but there are several diagnostic tests.
      • Peeling R.W.
      • Wedderburn C.J.
      • Garcia P.J.
      • et al.
      Serology testing in the COVID-19 pandemic response.
      • Thanh Le T.
      • Andreadakis Z.
      • Kumar A.
      • et al.
      The COVID-19 vaccine development landscape.
      • Pascarella G.
      • Strumia A.
      • Piliego C.
      • et al.
      COVID-19 diagnosis and management: a comprehensive review.
      Which should each health system adopt? To whom? These are not easy questions to answer.
      Value frameworks (VFs) attempt to communicate in a transparent and explicit way the important dimensions for decision making, and they usually reflect the preferences or values of the different actors involved in their construction and use. They basically define what dimensions are the most important—and thus which are less so—when judging the value of a health intervention.
      • Mandelblatt J.S.
      • Ramsey S.D.
      • Lieu T.A.
      • Phelps C.E.
      Evaluating frameworks that provide value measures for health care interventions.
      ,
      • Neumann P.J.
      • Willke R.J.
      • Garrison L.P.
      A health economics approach to US value assessment frameworks—introduction: an ISPOR Special Task Force Report [1].
      There are numerous examples of “generic” VFs used for the evaluation of a wide range of health technologies, but most are designed for or apply more straightforwardly to drugs or other therapeutic technologies. Some have been internationally developed, although there are also regional value frameworks or some developed in a particular country, or even oriented to decision making under specific health conditions (ie, oncology) or specific settings (ie, patient-centered decisions).
      Avalere Health
      Patient-Perspective Value Framework (PPVF).
      • Rocha-Gonçalves F.
      • Borges M.
      • Redondo P.
      • Laranja-Pontes J.
      Health technology assessment and value: the cancer value label (CAVALA) methodology.
      Health Information and Quality Authority
      A guide to health technology assessment at HIQA.
      • Culyer A.J.
      • Bombard Y.
      An equity checklist: a framework for health technology assessments.
      • Schnipper L.E.
      • Davidson N.E.
      • Wollins D.S.
      • et al.
      American Society of Clinical Oncology statement: a conceptual framework to assess the value of cancer treatment options.
      A specific VF to evaluate a diagnostic technology, on the other hand, tries to focus on this particular type of technology and to reflect the value dimensions of the tests within the entire patient care pathway.
      There is no study to our knowledge that has (1) synthesized value frameworks that target diagnostic technologies and (2) used this information to derive a value framework with the participation of a broad audience of key stakeholders in a deliberative and integrated way, particularly in Latin America. Thus, our project performed a targeted systematic review, with the objective of proposing an evidence-based, up-to-date VF informed by a broad group of stakeholders working in the health technology assessment (HTA) space, which could help to guide and inform evidence-based decision making.

      Methods

      This project was conducted to delineate a value framework for diagnostic technologies useful especially in the context of the HTA space. It was led by a team of researchers from the Institute for Clinical Effectiveness and Health Policy (IECS) together with an advisory committee overseeing the project, composed of 1 representative of a Ministry of Health, 1 academic member, 2 leaders of HTA agencies, and 1 member of a chamber of diagnostic technology producers. A broader group of key stakeholder groups was also convened to participate from academia, technology users, patients, public and private healthcare decision makers, and HTA specialists.
      This study—carried on between February 2019 and February 2020—had 2 clearly defined stages, each with different methodological approaches: (1) targeted systematic review, with the aim to identify current VFs and their dimensions; and (2) design of a VF proposal through a mixed-methods, qualitative-quantitative approach. We summarize in Table 1 the process and the different stages involved. It was funded by an educational unrestricted grant from Roche Diagnostics. The sponsor had no role in the design, conduct, or reporting of the study. As all other participants involved, they reviewed the materials and gave feedback. This article is based on the work of the researchers, advisory committee, and stakeholders and on discussions held during the project. It is not based on a formal consensus methodology and, therefore, should not be interpreted as necessarily representative of the views of the participants or the organizations in which they work. The views expressed in this article are those of the authors and do not necessarily reflect the position of any other involved in the process.
      Table 1Stages of the project.
      ComponentsObjectivesMethodsTasks
      First stageLiterature reviewIdentify VFs oriented to diagnostic technologies

      Identify criteria and subcriteria of VFs
      Quantitative/QualitativeOverview of diagnostic technology VFs through literature targeted systematic review
      Initial list of criteriaCreate a draft of VFQualitativeResearchers consensus meeting
      Second stageCriteria categorizationCategorize criteria and subcriteria, hierarchically

      Identify new criteria from stakeholders’ survey
      Quantitative/QualitativeStakeholders’ online survey
      Validation of the proposed VFValidate criteria’s categorization

      New criteria validation
      Quantitative/QualitativeFace-to-face meeting (workshop) with stakeholders
      Exploratory researchIdentify barriers and facilitators of VF implementation

      Identify particular aspects of diagnostic technology vs drug assessments
      Quantitative/Qualitative
      Final version of VFGenerate a VF for diagnostic technologies assessmentQuantitative/QualitativeResearchers consensus meeting

      Synthesis of data, integration of comments and recommendations
      VF indicates value framework.

      First Stage: Targeted Systematic Literature Review and Identification of the Components of the Value Framework and Initial List Proposed

      Targeted literature review

      We did a scoping review of the literature to identify VFs targeted to the evaluation of diagnostic tests or adaptations of generic VFs for the evaluation of diagnostic tests. Inclusion criteria were publications that described a VF (defined as a set of criteria or attributes that assist in defining the global value of particular health technologies) targeted to assist in the decision-making process of diagnostic tests, published from January 2000 to July 2019.
      We searched MEDLINE, LILACs, and other generic sources (Google Scholar, Google, Tripdatabase) with the following search strategy in MEDLINE: (Value framework∗ [tiab] OR Assessment Framework∗ [tiab]) AND Diagnos∗ [tiab]); the following terms were searched in the rest of the databases: “Assessment Framework + Diagnostic,” “Value Framework + Diagnostic.” We additionally did manual and gray literature searches in the webpages of the main HTA agencies, academic institutions, public or civil society organisms, health ministries, governments, and scientific and patient societies, among others. The search was done in English and Spanish.
      We included all the articles and sites that described a VF for diagnostic technologies regardless of who had been the developers (eg, agencies of HTA, scientific organizations, independent researchers, producers’ chambers), their scope width within diagnostic technologies (ie, if they were designed for a special type of tests such as companion test or genetic test), their methodology, or their funding. Finally, data from full-text articles of the included VFs was extracted. Two independent researchers participated in the selection process and screening titles and abstracts; disagreement was resolved by consensus and an eventual participation of a third researcher.

      Identification of the components of the value framework

      Candidate criteria or dimensions to be considered in a VF were identified and extracted from the literature review. In this article we use interchangeably the words “criterion,” “dimension,” or “domain”: a particular aspect or area that is considered when assessing the value of a diagnostic technology that could be considered to inform a decision. We define subcriteria (or subdomains or subdimensions) as different components “nested” within a criterion or dimension in cases where a dimension includes several different aspects. An exhaustive set of criteria and subcriteria was generated—in a free text form—in an Excel template extracting the data of the literature review. One researcher extracted the data while the others independently checked the extracted templates for completeness and accuracy. An initial comprehensive list with criteria and subcriteria was constructed.

      Initial list of criteria proposed

      Our research team worked on the initial list eliminating redundant and consolidating overlapping criteria. A smaller number of criteria and subcriteria from the list that were judged to include all the key dimensions and subdimensions were kept. This process was done in a deliberative way to arrive at a consensus among our research team and the advisory committee in order to keep an extensive but reasonable list of criteria and subcriteria for a VF that is planned to be used in real-life settings.

      Second Stage: Generation of the Proposed Value Framework

      This value framework was developed to be used for assessing the value of any diagnostic technology as any technology or procedure that is used to confirm, exclude, or classify a health problem. It is intended to be used in the context of the decision making specifically targeted to HTA agencies; Ministries of Health; and other public or private healthcare payers and providers (ie, to decide whether to incorporate a new diagnostic technology into a coverage package, to disinvest in a diagnostic technology with current coverage or in the acquisition of a diagnostic technology by a health service provider). The number of criteria was not predefined a priori in order to include all relevant criteria that our heterogeneous stakholders’ group considered and to be as comprehensive as possible.

      Preliminary value framework proposed

      The next step consisted of a first round of consultation with the stakeholders. Through an online survey (performed in SurveyMonkey), the initial list of selected criteria and subcriteria were assessed. All participants judged the relevance of each criteria and subcriteria and proposed new criteria they judged were important in the value assessment of diagnostic technologies and were omitted in our initial list. They categorized those criteria and subcriteria according to an importance scale. Each respondent had to indicate (on a scale of 0 to 10) how important the criteria or subcriteria proposed was for them to be included in a value framework targeted to diagnostic technology. Text labels were also used to relate the 0 to 10 scale to importance adjectives (9-10, essential or key; 7-8, high importance; 5-6, medium importance; 1-4, low importance; 0, not a candidate to be part of the VF). A secondary objective of this survey was to find out the opinion of the respondents about the distinctive characteristics of diagnostic technologies—as opposed mainly to drugs—at the time of being assessed through a generic value framework.
      Finally, each criterion and subcriterion was assigned to a relevance category (essential or key, high importance, medium importance, low importance, and not a candidate to be part of the VF) according to the most voted relevance category for each criterion. After analyzing the results of the survey, we generated a preliminary version of the proposed VF with the criteria and their categorization.

      Face-to-face validation of the preliminary VF and exploratory qualitative research

      This second round consisted of a full-day face-to-face workshop with the stakeholders, including the advisory committee. The format of this meeting included presentations, small-group dynamics, and plenary discussions. Small group activities consisted in qualitative focus group discussions following a semistructured guideline. Focus group participants used the results of the survey and had to validate or recommend changes to the initial categorization of the criteria and subcriteria, and to provide their opinion regarding the appropriateness of the new criteria or subcriteria proposed during the survey (through an online survey performed during the meeting to categorize new criteria or subcriteria). Through plenary sessions, they identified potential barriers and facilitators for the application of VF, and they discussed key messages to consider and the particular differences encountered when assessing these technologies compared to drugs. In case of disagreement, consensus was reached through a deliberative process. All the face-to-face activities were coordinated and guided by the leading group of researchers.

      Final version of the proposed value framework

      The researchers’ group synthetized the data gathered in the workshop using notes and online surveys performed. They produced a final list as well as their final categorization of criteria and subcriteria (essential, high importance, medium importance, low importance, or not considered relevant) of the proposed value framework.

      Results

      First Stage: Targeted Systematic Literature Review and Identification of the Components of the Value Framework and Initial List Proposed

      Targeted systematic review

      A total of 73 citations were initially identified through MEDLINE. Sixty-eight studies were discarded based on the titles and abstracts owing to either not being a VF, or being a VF not targeted to diagnostic technologies, so only 5 were retrieved from this search stage; 15 were retrieved from other sources. This left a total of 20 studies that met our inclusion criteria and were subsequently abstracted.
      National Institute for Health and Care Excellence
      Diagnostics Assessment Programme Manual.
      ,
      • Wurcel V.
      • Cicchetti A.
      • Garrison L.
      • et al.
      The value of diagnostic information in personalised healthcare: a comprehensive concept to facilitate bringing this technology into healthcare systems.
      ,
      Medical Services Advisory Committee
      Guidelines for the assessment of diagnostic technologies.
      • Merlin T.
      • Farah C.
      • Schubert C.
      • Mitchell A.
      • Hiller J.E.
      • Ryan P.
      Assessing personalized medicines in Australia: a national framework for reviewing codependent technologies.
      • Teutsch S.M.
      • Bradley L.A.
      • Palomaki G.E.
      • et al.
      The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group.
      Institut national d’excellence en santé et services sociaux
      Assessment mechanism for medical biology tests at INESSS.
      Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen
      General methods: version 5.0.
      AdvaMed
      A framework for comprehensive assessment of the value of diagnostic tests.
      • Frueh F.W.
      • Quinn B.
      Molecular diagnostics clinical utility strategy: a six-part framework.
      • Palmetto G.B.A.
      Molecular Diagnostic Program (MolDX®) coverage, coding, and pricing standard requirements.
      • Bojke L.
      • Grigore B.
      • Jankovic D.
      • Peters J.
      • Soares M.
      • Stein K.
      Informing reimbursement decisions using cost-effectiveness modelling: a guide to the process of generating elicited priors to capture model uncertainties.
      • Mann G.
      • Squire S.B.
      • Bissell K.
      • et al.
      State of the Art. Beyond accuracy: creating a comprehensive evidence base for tuberculosis diagnostic tools.
      • Canestaro W.J.
      • Pritchard D.E.
      • Garrison L.P.
      • Dubois R.
      • Veenstra D.L.
      Improving the efficiency and quality of the value assessment process for companion diagnostic tests: the Companion test Assessment Tool (CAT).
      • Anonychuk A.
      • Beastall G.
      • Shorter S.
      • Kloss-Wolf R.
      • Neumann P.
      A framework for assessing the value of laboratory diagnostics.
      EUnetHTA
      HTA Core Model for Diagnostic Technologies v 1.0r Work Package 4.
      EuroGentest
      Instrument for quality self-assessment in provision of genetic counselling.
      • Calonge N.
      • Green N.S.
      • Rinaldo P.
      • et al.
      Committee report: method for evaluating conditions nominated for population-based screening of newborns and children.
      • Rosenkötter N.
      • Vondeling H.
      • Blancquaert I.
      • Mekel O.C.L.
      • Kristensen F.B.
      • Brand A.
      The contribution of health technology assessment, health needs assessment, and health impact assessment to the assessment and translation of technologies in the field of public health genomics.
      • Rousseau F.
      • Lindsay C.
      • Charland M.
      • et al.
      Development and description of GETT: a genetic testing evidence tracking tool: International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) - IFCC Sientific Division Committee on Molecular Diagnostics.
      The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart is shown in Figure 1.
      Figure thumbnail gr1
      Figure 1Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram.
      All these studies were published in English from 2000 to 2019. Seven VFs (or guidelines to use a preexistent generic value framework in the context of diagnostic technologies) were developed by HTA agencies, 5 by consultants commissioned by industry associations of diagnostic technology or technology producers, 4 reflected the work of independent researchers, and 4 were undertaken by academic or scientific associations. All the VFs identified were developed in high-income countries; 5 of them were from international organizations, 6 originated in European countries, and 9 in North America and Australia. None were generated in Latin American countries or other low- and middle-income countries. In the majority of cases (85%), it was a VF developed ad hoc for the evaluation of diagnostic tests; the rest (15%) were guides adapting the use of a preexisting generic VF to the evaluation of diagnostic technologies—for example, the European Network for Health Technology Assessment (EUnetHTA) Core Model or the National Institute for Health and Care Excellence (NICE) Diagnostic Assessment Program. Of the included VFs, 11 were developed for some special test subtype (eg, companion test, molecular diagnostic test, genetic test, tuberculosis-oriented test), whereas only 9 were designed for any type of diagnostic technology. There were 2 VFs specifically targeted to the technology reimbursement process, whereas the majority were oriented to assist in the assessment of value of these technologies by different stakeholders. Diagnostic VFs are identified and their key characteristics summarized in Table 2. In Appendix 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.11.008, a complementary and detailed description of their objectives and components is also available.
      Table 2Value frameworks identified.
      FrameworkCountryYearDevelopersSourceOriginal VFBroad spectrum of diagnostic technologiesSpecific diagnostic technology
      Medical Services Advisory Committee (MSAC)
      • Canestaro W.J.
      • Pritchard D.E.
      • Garrison L.P.
      • Dubois R.
      • Veenstra D.L.
      Improving the efficiency and quality of the value assessment process for companion diagnostic tests: the Companion test Assessment Tool (CAT).
      Australia2005HTA agencyWeb
      National Framework for Reviewing Codependent Technologies
      • Anonychuk A.
      • Beastall G.
      • Shorter S.
      • Kloss-Wolf R.
      • Neumann P.
      A framework for assessing the value of laboratory diagnostics.
      Australia2013HTA agencyManual searchingCodependent technologies (companion test)
      National Institute for Health and Care Excellence. Diagnostic Assessment Programme (NICE DAP)
      National Institute for Health and Care Excellence
      Diagnostics Assessment Programme Manual.
      United Kingdom2010HTA agencyPubMed
      Evaluation of Genomic Applications in Practice and Prevention (EGAPP)
      EUnetHTA
      HTA Core Model for Diagnostic Technologies v 1.0r Work Package 4.
      United States2004Government officeWebGenetic test
      Institut National déxcellence en santé et en services sociaux (INESS)
      EuroGentest
      Instrument for quality self-assessment in provision of genetic counselling.
      Canada (Québec)Not availableHTA agencyWeb
      Institute for Quality and Efficiency in Health Care (IQWIG)
      • Calonge N.
      • Green N.S.
      • Rinaldo P.
      • et al.
      Committee report: method for evaluating conditions nominated for population-based screening of newborns and children.
      Germany2017HTA agencyWeb
      The Advanced Medical Technology Association (AdvaMed)
      • Rosenkötter N.
      • Vondeling H.
      • Blancquaert I.
      • Mekel O.C.L.
      • Kristensen F.B.
      • Brand A.
      The contribution of health technology assessment, health needs assessment, and health impact assessment to the assessment and translation of technologies in the field of public health genomics.
      United States2017Technology producers and consultantsWeb
      Value of Diagnostic Information (VODI)
      • Wurcel V.
      • Cicchetti A.
      • Garrison L.
      • et al.
      The value of diagnostic information in personalised healthcare: a comprehensive concept to facilitate bringing this technology into healthcare systems.
      Europe2018Technology producers and consultantsManual searching
      Frueh and Quinn
      • Rousseau F.
      • Lindsay C.
      • Charland M.
      • et al.
      Development and description of GETT: a genetic testing evidence tracking tool: International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) - IFCC Sientific Division Committee on Molecular Diagnostics.
      United States2014Independent researchersPubMedMolecular test
      Palmetto MOLDXUnited States2011Technology producers and consultantsManual searchingMolecular test
      Bojke, Soares et al
      Medical Services Advisory Committee
      Guidelines for the assessment of diagnostic technologies.
      United Kindgom2018Independent researchersPubMed
      Impact assessment framework (IAF)
      • Merlin T.
      • Farah C.
      • Schubert C.
      • Mitchell A.
      • Hiller J.E.
      • Ryan P.
      Assessing personalized medicines in Australia: a national framework for reviewing codependent technologies.
      International2010Academic groupPubMedTuberculosis diagnostic test
      Companion test Assessment Tool (CAT)
      • Teutsch S.M.
      • Bradley L.A.
      • Palomaki G.E.
      • et al.
      The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group.
      International2015Independent researchersWebCompanion test
      Lee et al
      • Lee D.W.
      • Neumann P.J.
      • Rizzo J.A.
      Understanding the medical and nonmedical value of diagnostic testing.
      International2010Technology producers and researchersPubMed
      EUnetHTA
      Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen
      General methods: version 5.0.
      Europe2008HTA agencyWeb
      Blancquaert Evaluation Framework
      Institut national d’excellence en santé et services sociaux
      Assessment mechanism for medical biology tests at INESSS.
      Canada2006Independent researchersManual searchingGenetic test
      Advisory Committee on Heritable Disorders in Newborns and Children (ACHDNC)
      • Frueh F.W.
      • Quinn B.
      Molecular diagnostics clinical utility strategy: a six-part framework.
      United States2003Government officeManual searchingGenetic test
      Genetic testing Evidence Tracking Tool (GEET Evaluation Tool)
      • Bojke L.
      • Grigore B.
      • Jankovic D.
      • Peters J.
      • Soares M.
      • Stein K.
      Informing reimbursement decisions using cost-effectiveness modelling: a guide to the process of generating elicited priors to capture model uncertainties.
      International2009Scientific associationsManual searchingGenetic test
      EuroGentest Evaluation Model
      AdvaMed
      A framework for comprehensive assessment of the value of diagnostic tests.
      International2010Scientific associationsManual searchingGenetic test
      The United Kingdom Genetic Testing Network (UKGTN)
      • Mann G.
      • Squire S.B.
      • Bissell K.
      • et al.
      State of the Art. Beyond accuracy: creating a comprehensive evidence base for tuberculosis diagnostic tools.
      United Kingdom2002HTA agencyManual searchingGenetic test
      VF indicates value framework; HTA, health technology assessment.

      Value framework criteria and subcriteria identification

      We initially consolidated a unique list of criteria and subcriteria by eliminating duplicates and unifying overlapping ones. Criteria such as effectiveness or accuracy were present in most VFs, whereas equity, sustainability, ecologic impact, or ethical aspects were the least frequent criteria in the diagnostic VFs included.

      Initial list of criteria proposed

      An initial list consisting of 9 criteria and 21 subcriteria was identified from the 20 studies that met our inclusion criteria, along with their definitions. The criteria list is shown in Table 3 (preliminary value framework section), and their definitions and subcriteria are shown in Appendix 2 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.11.008.
      Table 3Preliminary value framework and final value framework proposed (and ratings).
      CriteriaPreliminary value frameworkCriteriaValue framework proposed
      First round (online survey)Second round (face validation)
      EssentialsHigh ImportanceModerate ImportanceLow ImportanceNon-importantEssentialsHigh ImportanceModerate ImportanceLow ImportanceNon-important
      Clinical Benefit and Test Performance88%8%4%0%0%Clinical Benefit and Test Performance
      Quality of Scientific Evidence84%12%4%0%0%Quality of Scientific Evidence
      Safety and Unintended Consequences75%21%4%0%0%Safety and Unintended Consequences
      Economic Aspects54%38%8%0%0%Economic Aspects
      Severity of the disease52%26%19%4%0%Severity of the disease
      Organizational Aspects and Feasibility17%71%12%0%0%Organizational Aspects and Feasibility
      Health priority within the health system25%58%17%0%0%Health priority within the health system
      Disease burden42%50%8%0%0%Disease burden
      Equity25%38%38%0%0%Equity
      Ethical and legal aspects19%48%22%11%0%Ethical and legal aspects
      Absence of alternative diagnostic technologies27%44%22%11%0%Absence of alternative diagnostic technologies
      Innovation7%41%33%7%11%Innovation▼▼
      Nonclinical benefits4%32%60%4%0%Nonclinical benefits
      Environmental impact19%26%37%11%7%Environmental impact
      Broader social impact4%26%41%22%7%Broader social impact
      Note. Those criteria that were suggested during the online survey by stakeholders are marked in bold. The gray background indicates the highest value reached in each category.
      ✓ indicates that the categorization of the criteria did not change between the first and second round, ▲, recategorization upgraded 1 level; ▼, recategorization downgraded 1 level; ▼▼, recategorization downgraded 2 levels.

      Second Stage: Proposed Value Framework Generation

      Preliminary value framework proposed

      The survey was answered by all stakeholders (N = 31), and criteria and subcriteria were classified according to their perceived importance. None of them ranked as “low importance” or as a candidate for exclusion. Equity had a tie in the number of responses of 2 neighboring categories, so it was left in this way with the objective of solving that difference during the subsequent face-to-face (workshop) validation stage. Six criteria not proposed in the initial list were suggested during the survey: ethical and legal aspects; absence of alternative diagnostic technologies; severity of the disease; broader social impact; environmental impact; and innovation. See Table 3 where this process is shown in detail.

      Face-to-face validation of the preliminary VF and exploratory research

      A total of 30 stakeholders, which represented a broad spectrum of the healthcare sector, attended the meeting held a month after the survey. Five were representatives from governmental offices, 5 public or private payors, 4 from HTA agencies, 8 from the academia, 2 representatives of technology producers’ chambers, 4 technology producers, 2 representatives of a scientific society, and 1 representative of a patient advocacy program. All except 1 assistant from the UK were from the Latin American region (Argentina [14], Brazil [6], Colombia [2], Mexico [3], El Salvador [1], Costa Rica [1], Perú [1]). Two plenary discussions and 2 small group work phases were held. During the meeting, the 6 new criteria proposed during the survey were categorized (using the same hierarchical scale) through an instant voting system. The results can be seen in bolded font in Table 3. Then during working groups, a total set of 15 criteria and 21 subcriteria were analyzed for the second validation round. As a result of this step, stakeholders agreed that 11 of 15 criteria remained at the same categorization status during face-to-face validity round, whereas 4 were recategorized. (see Table 3). A list of barriers and facilitators for the use of the VFs in diagnostic technologies were identified and debated. These and the key messages for the diagnostic technology assessment can be found in Tables 4 and 5.
      Table 4Value framework proposed.
      Essential/core
      Criteria
      See definitions of criteria and subcriteria in Appendix 3 (in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.11.008).
      Subcriteria
      See definitions of criteria and subcriteria in Appendix 3 (in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.11.008).
      Clinical Benefit and Test PerformanceClinical consecuences of the use of the test
      Test performance
      Safety and Unwanted consequencesProcedural safety
      Consequences of the wrong diagnosis
      Safety of test preparation
      Safety of test operators
      Quality of scientific evidence
      Economical aspectsEconomic evaluation (clinical effectiveness and/or budget impact analysis)
      Other costs
      Organizational aspects and feasibility within the clinical pathImpact on the health services provision system
      Impact on the path of patient care
      High importance
      Health priority of the health system
      Disease burden
      EquityNeglected diseases test
      Test in communicable diseases and high prevalence
      Low access to health services
      Ethical and legal aspect
      Severity of the disease
      Absence of alternative diagnostic technologies
      Medium importance
      Nonclinical benefitsExperience of who takes the test
      Value of the information
      Load on caregivers or family
      Preparation and/or care
      Number of results associated with the test
      Test processing time
      Self test
      Environmental impact
      Broader social impact
      Low importance
      Innovation
      See definitions of criteria and subcriteria in Appendix 3 (in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.11.008).
      Table 5Main barriers and facilitators for implementing diagnostic value frameworks and differential aspects in the assessment of diagnostic technologies.
      BarriersFacilitators
      Underutilization of value frameworks in general, not only in diagnostic technologiesGood reception of any effort aimed at improving HTA processes
      Difficulty finding the evidence to support the components of the frameworkIndustry, payers, and government aligned on the need for an opportunity for dialogue between the parties
      Lack of technical capacity in HTA processesAbsence of a defined and legitimate framework that makes coverage decisions transparent
      Health systems are fragmented, and it is difficult for everyone to incorporate the same frameworkOpportunity window to standardize the evaluation processes of health technologies
      Differential aspects of diagnostic technologies compared to other technologies (mainly drugs)
      Diagnostic technologies must always be considered in the context of the patient’s treatment path.
      It is necessary to be precise to apply a common and homologated taxonomy in the evaluation of diagnostic technologies.
      It is difficult in these technologies to translate the evidence into the decision.
      The local clinical practice guidelines do not usually specify the type of diagnostic test necessary as if they do it with drugs.
      In the context of value-based medicine, it is difficult to reflect the value of diagnostic technologies.
      HTA indicates health technology assessment.

      Final version of value framework proposed

      After the final integration and after stakeholder consultation described above, 15 criteria and 21 subcriteria for defining the value of a diagnostic technology were included. Five criteria were rated as essential or core, 6 of them as highly relevant (high importance), and 3 as moderately relevant (medium importance); only 1 of them was judged as of low relevance (less importance). Table 4 (and Appendix 3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2020.11.008) show the different criteria and subcriteria of this value framework targeted to the assessment of broad diagnostic technologies. Some criteria were recategorized during the workshop—for example, organizational impact was reconsidered from being a high-importance criterion to being an essential criterion, and equity was also recategorized (from initially being of medium importance to high importance). Innovation, which had been initially categorized as of medium importance, was finally placed in the low importance category.

      Discussion

      Diagnostic technologies present particular challenges in value assessment for decision makers.
      • Wurcel V.
      • Cicchetti A.
      • Garrison L.
      • et al.
      The value of diagnostic information in personalised healthcare: a comprehensive concept to facilitate bringing this technology into healthcare systems.
      ,
      • Riley R.
      • Lesteven D.
      • Plun-Favreau J.
      • Ferrara J.
      • Kapitein P.
      • Collins P.
      The Value of Knowing and Knowing the Value: Improving the Health Technology Assessment of Complementary Diagnostics. White Paper.
      Several diagnostic VF exist, but they are either outdated, not based on a systematic evidence search and collaborative process, developed in only 1 jurisdiction, aiming at only 1 type of diagnostic technology, or potentially biased toward a particular perspective. Additionally, all were produced in high-income contexts. Our project developed new diagnostic technologies VF, based both on synthetizing the literature on existing VF and complementing this with a deliberative and iterative process with a broad range of stakeholders related to the HTA world in Latin America.
      Frameworks identified came mostly from high-income countries where HTA and decision-making processes are more mature for the need and use of a VF to complement the more established assessment of drugs. In Latin America there is a lag in the use of VFs in HTA, where they can be a valuable tool to promote greater transparency in the decision-making process and to facilitate the participation of key stakeholders.
      • Pichon-Riviere A.
      • Garcia-Marti S.
      • Oortwijn W.
      • Augustovski F.
      • Sampietro-Colom L.
      Defining the value of health technologies in Latin America: developments in value frameworks to inform the allocation of healthcare resources.
      Regarding the development of VFs aimed for use in a broad spectrum of tests, they were made mostly by HTA agencies and government offices to complement the established process in place regarding drugs. More specific VF, such as those for companion diagnostic tests and genetic tests, were usually shaped by scientific societies, researchers, and technology producers. Only a few were specifically targeted toward the reimbursement process (Palmetto MolDX or the National Framework for codependent technologies in Australia) and were developed for specific kind of diagnostic tests, such as molecular and companion tests.
      • Merlin T.
      • Farah C.
      • Schubert C.
      • Mitchell A.
      • Hiller J.E.
      • Ryan P.
      Assessing personalized medicines in Australia: a national framework for reviewing codependent technologies.
      ,
      • Palmetto G.B.A.
      Molecular Diagnostic Program (MolDX®) coverage, coding, and pricing standard requirements.
      Some VFs share a common root—for example, with the ones proposed by the Evaluation of Genomic Applications in Practice and Prevention (EGAPP), Advisory Committee on Heritable Disorders in Newborns and Children (ACHDNC), United Kingdom Genetic Testing Network (UKGTN), and EuroGentest are based in the ACCE model process for evaluation of genetic tests framework.
      • Teutsch S.M.
      • Bradley L.A.
      • Palomaki G.E.
      • et al.
      The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group.
      ,
      EuroGentest
      Instrument for quality self-assessment in provision of genetic counselling.
      ,
      • Calonge N.
      • Green N.S.
      • Rinaldo P.
      • et al.
      Committee report: method for evaluating conditions nominated for population-based screening of newborns and children.
      , The ACCE framework takes its name from the 4 main criteria for evaluating a genetic test—analytic validity, clinical validity, clinical utility, and associated ethical, legal, and social implications—and is composed of a set of 44 questions.
      Centers for Disease Control and Prevention
      ACCE model process for evaluating genetic tests.
      Our VF shares several dimensions with other VFs, such as clinical validity, clinical utility, analytical validity, and economic aspects, but also includes some other dimensions not usually considered in other VFs (such as equity, innovation, the impact on caregivers, safety to the personnel that performs the test, the potential environmental impact, and technology sustainability). We employed a mixed methodology to design the framework, where qualitative methods were merged with a quantitative approach. Another characteristic of our VF is that it also assessed the importance of the difference criteria: from the 15 criteria, 5 were considered essential, 6 of high, 3 of moderate, and 1 of low importance. Nevertheless, as relative importance of values can vary in different settings, decision makers can customize this VF and prioritize either the 5 essential criteria (or 11 if including those of high importance) or choose their unique set in the decision-making process and not use the whole VF.
      We developed our framework aiming at the assessment process of a wide range of diagnostic technologies, incorporating the perspectives of multiple stakeholders. We consider that its main use could be to inform the process of designing a benefit package or the incorporation of these technologies in any health subsector.
      The proposed VF builds on previous knowledge and reflects current opinions and perspectives of the main actors of the health system, from the technology producers to users, payers, patients, and caregivers. Starting from the systematic targeted literature review and an initial list of criteria, each of the participants had the opportunity to assess them and propose additional dimensions they considered important, which were then also discussed and rated by the whole group. This open dialogue generated a broad set of 15 criteria to consider when valuing these technologies. We believe that one of the big assets of our VF is the fact that it was led by an independent academic group through an educational and unrestricted grant, as well as that the heterogeneous group of stakeholders had a predominance of HTA agency officials and public and private payers, with the additional inclusion of technology producers, regulators, patients, and academics. Our aim was also centered on capturing different perspectives and being as comprehensive as possible but going further in the quest to fill the gap in assessing the complexity of diagnostic tests. Due to the methodology we followed (Latin American participants, with more participation by HTA professionals from governments, agencies, payers, and academia, and an equal weighting scheme in the process), this VF reflects the inherent value structure of this unique experience and is probably more oriented to governmental and payer decision-making. Further research could explore its extrapolation to other groups, regions, and settings.
      Most countries lack a specific process regarding the assessment of diagnostic technologies for their funding. At this particular moment, the world is facing several dilemmas, among which is the uncertainty in decision making about diagnostic tests strategies for COVID-19 pandemic. Each jurisdiction has its own peculiarities in health system, access to technology, equity, and even geographical differences to consider while deciding which test to incorporate into a comprehensive strategy against the virus.
      Our VF could be useful for decision makers who currently do not have a “fit for purpose” VF when assessing diagnostic technologies, and especially in lower- and middle-income countries, focused on reimbursement and/or benefit package design.
      Most of the participants agreed on the need for a VF that makes decision making transparent and that facilitates the evaluation of diagnostic tests within the entire patient care process. Organizational challenges that these technologies imply, access to diagnostic technologies but also to treatments resulting from the presence of a diagnosis, aspects of equity, and impact of the environment were highlighted as differential components that must be present when evaluating a diagnostic technology.
      A potential limitation of our VF for potential widespread use is that, even considering that the targeted systematic review was global, the stakeholder group was composed mainly of Latin American actors. Future studies could assess its validity in other contexts or countries.
      The International Society for Pharmacoeconomics and Outcomes Research Devices and Diagnostics Special Interest Group in 2016 reviewed 5 selected European and North American diagnostic-specific HTA programs (specifically targeted to molecular diagnostics) and their frameworks, aiming to identify common and best practices.
      • Garfield S.
      • Polisena J.
      • Spinner D.S.
      • et al.
      Health technology assessment for molecular diagnostics: practices, challenges, and recommendations from the medical devices and diagnostics special interest group.
      They concluded that the HTA process and posterior reimbursement decision should have transparent processes and a stronger link between the HTA and funding decisions and a recognition that these technologies require a differential approach during it assessment. It exceeds the scoping of this work to determine whether the effort to propose a specific VF will ultimately result in a better assessment of diagnostic technologies. Nevertheless, any effort that assists in providing transparency and accountability within the process of incorporation or rejection of a diagnostic technology is valuable.
      Our study provides an up-to-date, evidence- and stakeholder-informed VF that includes 15 criteria and can be a useful tool to specifically assess the value of diagnostic technologies. Future steps for its implementation include criteria operationalization for its use in real-life settings, assessing its usability and impact on decision making in different settings, and its transferability to other lower and middle income or high-income settings.
      Author Contributions: Concept and design: Augustovski, Alfie, Alcaraz, García Marti, Drummond, Pichon-Riviere
      Acquisition of data: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Analysis and interpretation of data: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Drafting of the manuscript: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Critical revision of the paper for important intellectual content: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Statistical analysis: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Provision of study materials or patients: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Obtaining funding: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Administrative, technical, or logistic support: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Supervision: Augustovski, Alfie, Alcaraz, Marti, Drummond, Pichon-Riviere
      Conflict of Interest Disclosures: Drs Augustoviski, Alfie, Alcaraz, Marti, and Pichon-Riviere reported receiving grants from Roche Diagnostics during the conduct of the study. Dr Drummond is an editor of Value in Health and had no role in the peer review process of this article.
      Funding/Support: This work was funded by an educational and unrestricted grant from Roche.
      Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
      Acknowledgment: We want to acknowledge members of the advisory committee, Carlos Gouvéa, (ALADDIV-Latin American Alliance for the Development of In Vitro Diagnosis; executive president, Brazil), Analía Lopez (Chief of Cabinet, Ministry of Health; Argentina), Vania Canuto Santos (Conitec, Chief, Brazil), and Adriana Robayo, (IETS-Institute of Technological Evaluation in Health; chair, Colombia) as well as all stakeholders involved: Acuña Merchán, Lizbeth (Colombian high cost account fund; director, Colombia); Andisco, Daniel (Argentine Society of Radiology; representative, Argentina); Arrighi, Emilia (Patient Advocacy, Austral University Hospital Foundation; director, Argentina); Boni Silvia (Administración Nacional de Medicamentos, Alimentos y Tecnología Médica, Argentina); Caballero Ñopo, Patricia (National Health Institute; board member, Perú); Caccavo, Francisco (Secretary of Health of the Argentine Republic; adviser, Argentina); Ferreira, Karine (Roche Diagnostics Brasil Ltda; Disease Strategy Lead LATAM, Brazil); Fonseca Seixas, Leticia (ALADDIV, Latin American Alliance for the Development of In Vitro Diagnosis; representative, Brazil); Galindo Suárez, Rosa María (Center for Research and Development of Health Sciences Autonomous University of Nuevo León; academic, Mexico); Garay, Ulises (Roche Diagnostics, Argentina) ; Gonzalez Malla, Carlos (National Institute of Social Services for Retirees and Pensioners; Board member, Argentina); Guevara Vazquez, Giovanni (Ministry of Health of El Salvador; adviser, El Salvador); Hernández San Román, Esteban (CENETEC, National Center for Health Technologies; board member, Mexico); Marques Martins, Ana Cristina (ANS-Agência Nacional de Saúde Suplementar; board member, Brazil); Perez, Ana (Ministry of Health of Uruguay Health Technology Assessment; chair, Uruguay); Rodriguez Camus, Maria Francisca (San Sebastián University; academic member, Chile); Rossi, Esteban (Becton Dickinson; vice president and general manager Latin America, Brazil); Sancho Jiménez, Ana Eduviges (Ministry of Health of Costa Rica; Health Technology Assessment, chair); Santos, Marisa (National Institute of Cardiology; Member, Brazil); Wurcel, Victoria (Value of Diagnostic Information Initiative; member, Argentina). We also want to acknowledge Gabriela Rodriguez (project manager, IECS), Daniel Comande (librarian, IECS), Maria Belizan (qualitative research unit, IECS), and Mariana Comolli (communications coordinator, IECS).

      References

        • Garrison L.P.
        • Towse A.
        A strategy to support efficient development and use of innovations in personalized medicine and precision medicine.
        J Manag Care Spec Pharm. 2019; 25: 1082-1087
        • Lee D.W.
        • Neumann P.J.
        • Rizzo J.A.
        Understanding the medical and nonmedical value of diagnostic testing.
        Value Health. 2010; 13: 310-314
        • Akhmetov I.
        • Bubnov R.V.
        Assessing value of innovative molecular diagnostic tests in the concept of predictive, preventive, and personalized medicine.
        EPMA J. 2015; 6: 19
        • Garau M.
        • Towse A.
        • Garrison L.
        • Housman L.
        • Ossa D.
        Can and should value-based pricing be applied to molecular diagnostics?.
        Per Med. 2013; 10: 61-72
        • National Institute for Health and Care Excellence
        Diagnostics Assessment Programme Manual.
        • Smith A.F.
        • Shinkins B.
        • Hall P.S.
        • Hulme C.T.
        • Messenger M.P.
        Toward a framework for outcome-based analytical performance specifications: a methodology review of indirect methods for evaluating the impact of measurement uncertainty on clinical outcomes.
        Clin Chem. 2019; 65: 1363-1374
        • Wurcel V.
        • Cicchetti A.
        • Garrison L.
        • et al.
        The value of diagnostic information in personalised healthcare: a comprehensive concept to facilitate bringing this technology into healthcare systems.
        Public Health Genomics. 2019; 22: 8-15
        • Faulkner E.
        • Holtorf A.P.
        • Walton S.
        • et al.
        Being precise about precision medicine: what should value frameworks incorporate to address precision medicine? a report of the personalized precision medicine special interest group.
        Value Health. 2020; : 1-11
        • Garfield S.
        • Polisena J.
        • Spinner D.S.
        • et al.
        Health technology assessment for molecular diagnostics: practices, challenges, and recommendations from the medical devices and diagnostics special interest group.
        Value Health. 2016; 19: 577-587
        • Govaerts L.
        • Simoens S.
        • Van Dyck W.
        • Huys I.
        Health policy analysis shedding light on reimbursement policies of companion diagnostics in European countries.
        Value Health. 2020; 23: 606-615
        • Emanuel E.J.
        • Persad G.
        • Upshur R.
        • et al.
        Fair allocation of scarce medical resources in the time of Covid-19.
        N Engl J Med. 2020; 382: 2049-2055
        • Peeling R.W.
        • Wedderburn C.J.
        • Garcia P.J.
        • et al.
        Serology testing in the COVID-19 pandemic response.
        Lancet Infect Dis. 2020; 20: e245-e249
        • Thanh Le T.
        • Andreadakis Z.
        • Kumar A.
        • et al.
        The COVID-19 vaccine development landscape.
        Nat Rev Drug Discov. 2020; 19: 305-306
        • Pascarella G.
        • Strumia A.
        • Piliego C.
        • et al.
        COVID-19 diagnosis and management: a comprehensive review.
        J Intern Med. 2020; 288: 192-206
        • Mandelblatt J.S.
        • Ramsey S.D.
        • Lieu T.A.
        • Phelps C.E.
        Evaluating frameworks that provide value measures for health care interventions.
        Value Health. 2017; 20: 185-192
        • Neumann P.J.
        • Willke R.J.
        • Garrison L.P.
        A health economics approach to US value assessment frameworks—introduction: an ISPOR Special Task Force Report [1].
        Value Health. 2018; 21: 119-123
        • Avalere Health
        Patient-Perspective Value Framework (PPVF).
        • Rocha-Gonçalves F.
        • Borges M.
        • Redondo P.
        • Laranja-Pontes J.
        Health technology assessment and value: the cancer value label (CAVALA) methodology.
        Ecancermedicalscience. 2016; 10: 1-10
        • Health Information and Quality Authority
        A guide to health technology assessment at HIQA.
        • Culyer A.J.
        • Bombard Y.
        An equity checklist: a framework for health technology assessments.
        Management. 2011; 32: 428-441
        • Schnipper L.E.
        • Davidson N.E.
        • Wollins D.S.
        • et al.
        American Society of Clinical Oncology statement: a conceptual framework to assess the value of cancer treatment options.
        J Clin Oncol. 2015; 33: 2563-2577
      1. ICER Value Assessment Framework.
        • Medical Services Advisory Committee
        Guidelines for the assessment of diagnostic technologies.
        http://www.ag.gov.au/cca
        Date accessed: June 28, 2019
        • Merlin T.
        • Farah C.
        • Schubert C.
        • Mitchell A.
        • Hiller J.E.
        • Ryan P.
        Assessing personalized medicines in Australia: a national framework for reviewing codependent technologies.
        Med Decis Mak. 2013; 33: 333-342
        • Teutsch S.M.
        • Bradley L.A.
        • Palomaki G.E.
        • et al.
        The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group.
        Genet Med. 2009; 11: 3
        • Institut national d’excellence en santé et services sociaux
        Assessment mechanism for medical biology tests at INESSS.
        • Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen
        General methods: version 5.0.
        • AdvaMed
        A framework for comprehensive assessment of the value of diagnostic tests.
        • Frueh F.W.
        • Quinn B.
        Molecular diagnostics clinical utility strategy: a six-part framework.
        Expert Rev Mol Diagn. 2014; 14: 777-786
        • Palmetto G.B.A.
        Molecular Diagnostic Program (MolDX®) coverage, coding, and pricing standard requirements.
        • Bojke L.
        • Grigore B.
        • Jankovic D.
        • Peters J.
        • Soares M.
        • Stein K.
        Informing reimbursement decisions using cost-effectiveness modelling: a guide to the process of generating elicited priors to capture model uncertainties.
        Pharmacoeconomics. 2017; 35: 867-877
        • Mann G.
        • Squire S.B.
        • Bissell K.
        • et al.
        State of the Art. Beyond accuracy: creating a comprehensive evidence base for tuberculosis diagnostic tools.
        Int J Tuberc Lung Dis. 2010; 14: 1518-1524
        • Canestaro W.J.
        • Pritchard D.E.
        • Garrison L.P.
        • Dubois R.
        • Veenstra D.L.
        Improving the efficiency and quality of the value assessment process for companion diagnostic tests: the Companion test Assessment Tool (CAT).
        J Manag Care Spec Pharm. 2016; 21: 700-712
        • Anonychuk A.
        • Beastall G.
        • Shorter S.
        • Kloss-Wolf R.
        • Neumann P.
        A framework for assessing the value of laboratory diagnostics.
        Healthc Manag Forum. 2012; 25: S4
        • EUnetHTA
        HTA Core Model for Diagnostic Technologies v 1.0r Work Package 4.
        www.eunethta.net
        Date accessed: April 8, 2020
        • EuroGentest
        Instrument for quality self-assessment in provision of genetic counselling.
        • Calonge N.
        • Green N.S.
        • Rinaldo P.
        • et al.
        Committee report: method for evaluating conditions nominated for population-based screening of newborns and children.
        Genet Med. 2010; 12: 153-159
        • Rosenkötter N.
        • Vondeling H.
        • Blancquaert I.
        • Mekel O.C.L.
        • Kristensen F.B.
        • Brand A.
        The contribution of health technology assessment, health needs assessment, and health impact assessment to the assessment and translation of technologies in the field of public health genomics.
        Public Health Genomics. 2010; 14: 43-52
        • Rousseau F.
        • Lindsay C.
        • Charland M.
        • et al.
        Development and description of GETT: a genetic testing evidence tracking tool: International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) - IFCC Sientific Division Committee on Molecular Diagnostics.
        Clin Chem Lab Med. 2010; 48: 1397-1407
        • UK Genetic Testing Network
        Evaluation of new genetic tests for NHS services.
        • Riley R.
        • Lesteven D.
        • Plun-Favreau J.
        • Ferrara J.
        • Kapitein P.
        • Collins P.
        The Value of Knowing and Knowing the Value: Improving the Health Technology Assessment of Complementary Diagnostics. White Paper.
        • Pichon-Riviere A.
        • Garcia-Marti S.
        • Oortwijn W.
        • Augustovski F.
        • Sampietro-Colom L.
        Defining the value of health technologies in Latin America: developments in value frameworks to inform the allocation of healthcare resources.
        Int J Technol Assess Health Care. 2019; 35
        • Centers for Disease Control and Prevention
        ACCE model process for evaluating genetic tests.