If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Identifying the Need for Good Practices in Health Technology Assessment: Summary of the ISPOR HTA Council Working Group Report on Good Practices in HTA
Address correspondence to: Don Husereau, MSc, BScPharm, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Room 101, 600 Peter Morand Crescent, Ottawa, ON, Canada, K1G 5Z3.
School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, Ottawa, ON, CanadaDepartment of Public Health, Health Services Research and Health Technology Assessment, UMIT - University of Health Sciences, Medical Informatics and Technology, Hall in Tirol, Austria
School of Population and Global Health, University of Melbourne, Melbourne, AustraliaDepartment of Health Technology & Services Research, University of Twente, Enschede, The Netherlands
Department of Public Health, Health Services Research and Health Technology Assessment, UMIT - University of Health Sciences, Medical Informatics and Technology, Hall in Tirol, AustriaDivision of Health Technology Assessment, ONCOTYROL - Center for Personalized Cancer Medicine, Innsbruck, AustriaInstitute for Technology Assessment, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, and Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA
The systematic use of evidence to inform healthcare decisions, particularly health technology assessment (HTA), has gained increased recognition. HTA has become a standard policy tool for informing decision makers who must manage the entry and use of pharmaceuticals, medical devices, and other technologies (including complex interventions) within health systems, for example, through reimbursement and pricing. Despite increasing attention to HTA activities, there has been no attempt to comprehensively synthesize good practices or emerging good practices to support population-based decision-making in recent years. After the identification of some good practices through the release of the ISPOR Guidelines Index in 2013, the ISPOR HTA Council identified a need to more thoroughly review existing guidance. The purpose of this effort was to create a basis for capacity building, education, and improved consistency in approaches to HTA-informed decision-making. Our findings suggest that although many good practices have been developed in areas of assessment and some other key aspects of defining HTA processes, there are also many areas where good practices are lacking. This includes good practices in defining the organizational aspects of HTA, the use of deliberative processes, and measuring the impact of HTA. The extent to which these good practices are used and applied by HTA bodies is beyond the scope of this report, but may be of interest to future researchers.
Introduction
Health technology assessment (HTA) has become a standard policy tool for informing decision makers who must manage the entry and use of pharmaceuticals, medical devices, and other technologies (including complex interventions) within health systems, for example, through reimbursement and pricing. Despite increasing HTA activity, there has been no attempt to comprehensively synthesize good practices or emerging good practices to support population-based decision-making in recent years.
The purpose of the ISPOR HTA Council Working Group was to provide an up-to-date review of current literature that includes guidance on good practices in the use of evidence to inform population-based healthcare decision-making for pharmaceuticals (drugs and vaccines), medical devices, and other health technologies, that is, HTA. Population-based decisions are those linked to management, administration, and other forms of health system governance and stewardship. The use of evidence to inform individual decisions between patients and clinicians is outside of the scope of this review; nevertheless, the Working Group recognizes that HTA may be used to broadly inform clinical practice decisions through clinical practice guidelines or clinical pathway development and thus have not excluded these from the scope of the article.
The rationale for identifying good HTA practices in using evidence to inform population-based healthcare decision-making is to provide a basis for capacity building, education, and improved consistency in approaches to HTA-informed decision-making. The primary audiences for this report are those who manage, design, or seek to improve HTA processes, although we hope that it is informative to a wider audience of patients, care providers, payers, academics, and industry stakeholders.
Given the large scope of this work and to achieve its objectives, the HTA Council Working Group created an overview report with a summary of key references related to good practices in HTA. The overview report outlines where guidance for good practices has been identified and where good practices are still emerging or could not be identified. This report will focus on prioritizing next steps that may be taken by ISPOR and other interested parties and is a summary of the effort. The full report can be found on the ISPOR website (https://www.ispor.org/member-groups/councils-roundtables/health-technology-assessment-council) and as a Supplementary Appendix to this article available at https://doi.org/10.1016/j.jval.2018.08.010.
Methods
The Working Group’s approach in developing this report was based on literature review and expert opinion. In this respect it followed a similar approach to that of ISPOR Task Forces.
The need for a review of best practices was first identified by the ISPOR HTA Council after a review of the ISPOR Guideline Index for Outcomes Research.
ISPOR Guidelines Index for Outcomes Research and Use in Health Care Decision-making. Guidelines Index for Outcomes Research and Use in Health Care Decision-making. n.d.
The council then identified co-chairs who invited members of the Working Group. An outline for the report was then drafted and reviewed by members of the Working Group.
An early challenge for the Working Group was arriving at consistent conceptual definitions of an HTA process and its associated terminology. In the end, HTA processes were characterized using a combination of concepts derived from healthcare decision-making,
Methods, procedures, and contextual characteristics of health technology assessment and health policy decision-making: comparison of health technology assessment agencies in Germany, United Kingdom, France, and Sweden.
Int J Technol Assess Health Care.2009; 25: 305-314
ISPOR Guidelines Index for Outcomes Research and Use in Health Care Decision-making. Guidelines Index for Outcomes Research and Use in Health Care Decision-making. n.d.
The proposed framework assumes that the goal of HTA is to support healthcare decision-making, and it addresses all aspects, including how HTA processes are governed and defined (“Defining the HTA process”); how research information is identified, analyzed, and interpreted (“Assessment”); how these interpretations are applied and weighed to the context of a decision (“Contextualization”); and how this ultimate interpretation and weighting is intended to support healthcare decisions (“Implementation and Monitoring HTA”).
Fig. 1Components of HTA within the healthcare decision-making process. HTA indicates health technology assessment.
Sections of the report identified through the framework were assigned and drafted by individual Working Group members who were encouraged to use comprehensive approaches toward searching for existing descriptions of current practice, guidance for best practice, and to provide expert opinion (preferably based on published reports), identifying issues related to each section assigned. Systematic reviews were typically not conducted by Working Group members, although all authors were encouraged to conduct them or identify systematic reviews in their assigned areas.
Once drafted, the report was reviewed by all members, revised, and circulated to members of a larger review group (see Acknowledgements); it was then further revised, leading to this final report. In parallel, findings were summarized and presented at open workshops during ISPOR meetings (Boston, MA, USA, and Glasgow, Scotland, UK).
Findings
General Findings
In some areas, we were unable to identify good practices specific to HTA. This included good practices in defining the organizational aspects of HTA, the use of deliberative processes, and measuring the impact of HTA. In some areas, such as guidance for the interpretation of individual studies or bodies of evidence, there was an abundance of available practice guidance that was either discipline or HTA specific.
Priority setting for health technology assessment. Theoretical considerations and practical approaches. Priority setting Subgroup of the EUR-ASSESS Project.
Int J Technol Assess Health Care.1997; 13: 144-185
AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program.
A questionnaire to assess the relevance and credibility of observational studies to inform health care decision-making: an ISPOR-AMCP-NPC Good Practice Task Force report.
Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and Elaboration: A Report of the ISPOR Health Economic Evaluations Publication Guidelines Task Force.
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.
Interpreting indirect treatment comparisons and network meta-analysis for health-care decision-making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 1.
Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision-making: an ISPOR-AMCP-NPC Good Practice Task Force report.
The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations.
Using qualitative evidence in decision-making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force.
Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada.
What are the effects of HTA reports on the health system? Evidence from the research literature.
in: Velasco Garrido M. Kristensen F.B. Palmhøj Nielsen C. Busse R. Health Technology Assessment and Health Policy-Making in Europe: Current status, Challenges and Potential. WHO,
Geneva2008: 109-136
Priority setting for health technology assessment. Theoretical considerations and practical approaches. Priority setting Subgroup of the EUR-ASSESS Project.
Int J Technol Assess Health Care.1997; 13: 144-185
AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program.
Summarized Research in Information Retrieval for HTA (SuRe Info) Cochrane Risk of Bias Tools EuNetHTA Guidance ISPOR-AMCP-NPC Good Practice Task Force Questionnaire MedTecHTA Recommendations HTA Core Model
A questionnaire to assess the relevance and credibility of observational studies to inform health care decision-making: an ISPOR-AMCP-NPC Good Practice Task Force report.
Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and Elaboration: A Report of the ISPOR Health Economic Evaluations Publication Guidelines Task Force.
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.
Interpreting indirect treatment comparisons and network meta-analysis for health-care decision-making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 1.
Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision-making: an ISPOR-AMCP-NPC Good Practice Task Force report.
The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations.
Using qualitative evidence in decision-making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force.
Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada.
What are the effects of HTA reports on the health system? Evidence from the research literature.
in: Velasco Garrido M. Kristensen F.B. Palmhøj Nielsen C. Busse R. Health Technology Assessment and Health Policy-Making in Europe: Current status, Challenges and Potential. WHO,
Geneva2008: 109-136
Twenty years ago, the EUR-ASSESS Project made it clear that HTA is not defined by a set of methods but by its intent, and given the wide scope of HTA, it should not be viewed as a single discipline or field. Rather, HTA is multidisciplinary and rooted in good practices in evaluation, including sound research methods.
Today, HTA still uses a range of approaches intended to inform decision-making and based in research. There is now a more widely shared understanding of the standards that HTA should aim to meet and understanding of the importance of developing, agreeing, and implementing good practices.
Our findings suggest that many good practices have been developed in areas of assessment and in some aspects of defining HTA processes (priority setting, framing, and scoping principles, as well as in areas of implementation). Few good practices were found related to structure, governance, or organizational aspects of HTA and measuring HTA impact.
Using these underlying concepts, the challenge for the Working Group was to arrive at consensus regarding the extent to which good practices can be identified and are available. The wide scope of this overview and the approach taken to search and identify relevant guidance, coupled with many approaches not widely publicized and a rapidly growing literature, means that it is possible that some good practices may have been overlooked. The Working Group also acknowledges that regional practices also vary according to resource constraints and health system structures, although this implies there can never be a “one-size-fits-all” approach to HTA. This is, however, not an excuse for applying substandard approaches that may ultimately undermine the intent of HTA.
HTA, encompassing evidence synthesis, may be viewed as informing evidence-based decision-making—two related but distinct concepts.
The process of rigorous review and synthesis of scientific evidence focuses on assessing the relative benefits, harms, and costs of healthcare technologies using sound analytic judgments. Evidence-based decision-making, in most cases, explicitly or implicitly incorporates other considerations (eg, affordability, ethical issues, feasibility, and acceptability) that may require mechanisms of contextualization of assessment results, such as deliberative processes, to support them.
These latter considerations, the discussion of which is sometimes called “appraisal,” can be supported or coordinated by HTA bodies and have recently received heightened attention; their crucial importance in HTA has been recognized. This has led to a fuzzy distinction between the activities of HTA and decision-making, particularly in processes of contextualization, for example, in appraisal and reimbursement committees and the recommendations that come from them. Such recommendations may involve both analytic judgments (such as willingness to include indirect comparison and surrogate endpoints as source of evidence or how quality adjusted life years [QALYs] were derived) and consideration of social values (such as weighing the value of a QALY in the very young or old).
The ability of decision makers to override recommendations of HTA bodies, based on other considerations and variations in approaches to HTA, makes its role even more difficult to discern, even to experts in the field.
This has led to much criticism of HTA in recent years, resulting from the decision-making processes and the extent to which they are transparent and deliberative. Unfortunately, this criticism may result in some spillover and skepticism regarding the assessment process. The future acceptance of HTA may depend on greater clarity regarding the scope of these two processes, largely identified with “assessment” and “contextualization” in this document, and additional measures to enhance the transparency by decision makers regarding the key elements that actually are driving decisions.
Moving systematic review and synthesis beyond clinical, epidemiological, and economic research into qualitative and quantitative research in patient-, caregiver-, and citizen-generated information (such as perceptions, valuation, and outcomes) is an immediate need in HTA. As part of this effort, there is a need for more research into the structured approaches to deliberative decision making. Such research could potentially support the application of multicriteria decision analysis
or other promising methods of integrating social values. This will represent a continuation of the EUR-ASSESS approach as implemented in the HTA Core Model and would help further “populate” the nonclinical domains of the model such as “patient and social” and organizational aspects with good methodologies and more evidence.
Beyond a clear delineation of the roles of HTA and decision making (as well as scientific judgment and value judgment), HTA bodies may also need to consider what healthcare decisions are best supported by HTA. The move to early dialog and scientific advice on evidence generation to technology developers can be seen as advancement toward more constructive HTA processes, where alignment between patients, payers, regulators, and technology producers is created through shared information requirements and collaborative planning.
It is also a stepping stone to HTA, considering the costs of innovation, when informing healthcare decision makers. Recognition of the overlapping roles of regulatory and HTA processes is another key area of evolution and development for HTA.
Efforts by researchers in the disciplines that contribute to HTA will undoubtedly continue to include review of their own good practices and produce guidelines and textbooks that will have immediate relevance for HTA. Taken together, priorities for good practice guidance in HTA, as reflected in this article and the ISPOR Outcomes Research Guidelines Index,
ISPOR Guidelines Index for Outcomes Research and Use in Health Care Decision-making. Guidelines Index for Outcomes Research and Use in Health Care Decision-making. n.d.
will likely need to focus on developing good practices in using evidence to support decision-making through monitoring of HTA implementation and its input to various types of decision-making, rather than concentrating the focus of guidance production on HTA research practices (eg, evidence review and synthesis, outcomes research, and health economics), while encouraging and increasingly building on high-quality research guidance from these “contributing” fields of research. With the evolving ISPOR Guidelines Index and this review of current guidance, it may be easier to prioritize where efforts should be put in developing good practices in HTA.
Acknowledgements
We are sincerely grateful to the following people, who generously gave their time and shared their expertise by submitting written comments on a draft version of this article: Meindert Boysen, Karen Facey, Ansgar Gerhardus, Clifford Goodman, IQWiG, Zoltán Kaló, Sukyeong Kim, Bryan Luce, Aurelio Mejía, Andrew Mitchell, Wija Oortwijn, Matthias Perleth, Maureen Smith, Brian Solow, Lizzie Thomas, Janet Wale, Richard Willke, Ingrid Zechmeister-Koss, and Kun Zhao. The steady and capable support of ISPOR and its staff, especially Kelly Lenahan, is also genuinely appreciated.
ISPOR Guidelines Index for Outcomes Research and Use in Health Care Decision-making. Guidelines Index for Outcomes Research and Use in Health Care Decision-making. n.d.
Methods, procedures, and contextual characteristics of health technology assessment and health policy decision-making: comparison of health technology assessment agencies in Germany, United Kingdom, France, and Sweden.
Int J Technol Assess Health Care.2009; 25: 305-314
Priority setting for health technology assessment. Theoretical considerations and practical approaches. Priority setting Subgroup of the EUR-ASSESS Project.
Int J Technol Assess Health Care.1997; 13: 144-185
AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program.
A questionnaire to assess the relevance and credibility of observational studies to inform health care decision-making: an ISPOR-AMCP-NPC Good Practice Task Force report.
Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and Elaboration: A Report of the ISPOR Health Economic Evaluations Publication Guidelines Task Force.
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.
Interpreting indirect treatment comparisons and network meta-analysis for health-care decision-making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 1.
Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision-making: an ISPOR-AMCP-NPC Good Practice Task Force report.
The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations.
Using qualitative evidence in decision-making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force.
Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada.
What are the effects of HTA reports on the health system? Evidence from the research literature.
in: Velasco Garrido M. Kristensen F.B. Palmhøj Nielsen C. Busse R. Health Technology Assessment and Health Policy-Making in Europe: Current status, Challenges and Potential. WHO,
Geneva2008: 109-136
Author Contributions: The ISPOR HTA Council provided the original direction for this work. F.B.K. and D.H. led the writing of the article, including the drafting of the outline and article, and are the guarantors of this work. All authors approved the outline of the work, helped to write and revise the article, and read and approved the final version of the article.
Disclosures: None of the authors have received funding to draft the report. All authors have signed the ICJME Form for Disclosure of Potential Conflicts of Interest and read information regarding disclosure of potential conflict of interest and have made declarations based on these. Every attempt was made to conduct this research in compliance with the 2017 ISPOR Code of Ethics.
36144-8&id=gr1.jpg){kind=link}