- Kaufmann D.
- Kraay A.
- Heintz E.
- Gerber-Grote A.
- Ghabri S.
- Hamers F.F.
- Rupel V.P.
- et al.
- Sutton A.J.
- Abrams K.R.
- Jones D.R.
- et al.
Submission template for pharmaceuticals and submission template for medical devices. n.d.
- Gerhardus A.
- Dorendorf E.
- Røttingen J.-A.
- Santamera A.S.
|HTA Practice||Good practices identified||Example(s)||Notes||References|
|Define the HTA process|
|Structure/governance/organizational aspects of HTA||Few||WHO and World Bank frameworks||Not specific to HTA|
Governance Indicators: Where Are We, Where Should We Be Going? The World Bank; 2007.
|Framework/principles for the HTA process||Yes||Various||Some developed for comparison and benchmarking|
|Priority setting process||Yes||EUnetHTA procedure|
|Framing and scoping||Yes||HTA Core Model, Danish guidelines, NICE||Assumed many scoping processes unpublished|
|Assessment (synthesizing evidence)|
|Identifying and interpreting individual studies||Yes||Summarized Research in Information Retrieval for HTA (SuRe Info)|
Cochrane Risk of Bias Tools
ISPOR-AMCP-NPC Good Practice Task Force Questionnaire
HTA Core Model
|Tools for some study types still nascent|
Is there a European view on health economic evaluations? Results from a synopsis of methodological guidelines used in the EUnetHTA partner countries.
Pharmacoeconomics. 2016; 34 (EUnetHTA Joint Action 2, Work Package 7, Subgroup 3): 59-76
|Interpreting bodies of evidence||Yes||Assessing methodological quality of systematic reviews (AMSTAR) tool|
Methods for Meta-Analysis in Medical Research. n.d.
|Contextualizing (using evidence)|
|Deliberative processes||Few||OHTAC Deliberative Framework||Few good practices dedicated to HTA|
|Patient engagement and patient preferences||Yes||HTAi Values and Preferences Tool||Many approaches|
|Weighted stakeholder preferences and multicriteria decision analysis||Yes||EVIDEM|
|Use of thresholds||Yes||UK NICE||Specific to certain health systems|
|Interpreting or adapting HTAs from other jurisdictions||Yes||EUnetHTA adaptation checklist|
ISPOR Good Research Practices Task Force report on transferability of economic evaluations
|Specific guidance for economic evaluation also available|
|Use of budget impact analyses||Few||Institute for Clinical and Economic Review|
|Implementing and monitoring HTA|
|Implementing HTA||Yes||SUPPORT Tools||Different approaches|
Submission template for pharmaceuticals and submission template for medical devices. n.d.
|Measuring HTA Impact||Few||“Six step” model|
What are the effects of HTA reports on the health system? Evidence from the research literature.
in: Velasco Garrido M. Kristensen F.B. Palmhøj Nielsen C. Busse R. Health Technology Assessment and Health Policy-Making in Europe: Current status, Challenges and Potential. WHO, Geneva2008: 109-136
Early dialogue consolidated procedure. n.d.
- Supplementary Appendix
- ISPOR Good Practices for Outcomes Research. n.d.
- ISPOR Guidelines Index for Outcomes Research and Use in Health Care Decision-making. Guidelines Index for Outcomes Research and Use in Health Care Decision-making. n.d.Accessed August 1, 2018)
- Clinical decision-making: from theory to practice. Anatomy of a decision.JAMA. 1990; 263: 441-443
- Methods, procedures, and contextual characteristics of health technology assessment and health policy decision-making: comparison of health technology assessment agencies in Germany, United Kingdom, France, and Sweden.Int J Technol Assess Health Care. 2009; 25: 305-314
- The HTA Core Model®-10 Years of Developing an International Framework to Share Multidimensional Value Assessment.Value Health. 2017; 20: 244-250
- Governance Indicators: Where Are We, Where Should We Be Going? The World Bank; 2007.Accessed January 24, 2018.)
- The world health report 2000 - Health systems: improving performance. n.d.
- European Observatory on Health Systems and Policies.in: Strengthening Health System Governance: Better Policies, Stronger Performance. Open University Press, Maidenhead, England2016
- A Framework for Good Governance in the Pharmaceutical Sector. The Hashemite Kingdom of Jordan. n.d.
- Governance for Sustainable Human Development: A UNDP Policy Document.United Nations Development Programme, New York1997
- Key principles for the improved conduct of health technology assessments for resource allocation decisions.Int J Technol Assess Health Care. 2008; 24 (discussion 362-368): 244-258
- Can we reliably benchmark health technology assessment organizations?.Int J Technol Assess Health Care. 2012; 28: 159-165
- The role of health technology assessment on pharmaceutical reimbursement in selected middle-income countries.Health Policy. 2010; 95: 174-184
- Mapping of health technology assessment in selected countries.Int J Technol Assess Health Care. 2013; 29: 424-434
- Towards integrated health technology assessment for improving decision-making in selected countries.Value Health. 2017; 20: 1121-1130
- A comparison of reimbursement recommendations by European HTA agencies: is there opportunity for further alignment?.Front Pharmacol. 2017; 8: 384
- Priority setting for health technology assessment. Theoretical considerations and practical approaches. Priority setting Subgroup of the EUR-ASSESS Project.Int J Technol Assess Health Care. 1997; 13: 144-185
- The health systems’ priority setting criteria for selecting health technologies: a systematic review of the current evidence.Med J Islam Repub Iran. 2016; 30: 329
- Priority setting for health technology assessments: a systematic review of current practical approaches.Int J Technol Assess Health Care. 2007; 23: 310-315
- How to choose health technologies to be assessed by HTA? A review of criteria for priority setting.Epidemiol Prev. 2015; 39: 39-44
- AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program.J Clin Epidemiol. 2010; 63: 491-501
- Health Technology Assessment Handbook.Danish Centre for Health Technology Assessment, Copenhagen2007
- HTA Core Model® Online Handbook. n.d.
- Technology appraisal guidance. NICE guidance. n.d.https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidanceDate accessed: October 5, 2017
- Keeping up to date with information retrieval research: Summarized Research in Information Retrieval (SuRe Info).J Eur Assoc Health Inform Libr. 2013; 9: 17-19
- The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.BMJ. 2011; 343: d5928
- RoB 2.0 tool—Risk of bias tools. n.d.
- ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions.BMJ. 2016; 355: i4919
- Observational Evidence and Strength of Evidence Domains: Case Examples.Agency for Healthcare Research and Quality (US), Rockville, MD2014
- Developing a Protocol for Observational Comparative Effectiveness Research: A User’s Guide. n.d.
- Observational data TSD. n.d.http://scharr.dept.shef.ac.uk/nicedsu/technical-support-documents/observational-data-tsd/Date accessed: October 5, 2017
- STRengthening Analytical Thinking for Observational Studies. n.d.
- A questionnaire to assess the relevance and credibility of observational studies to inform health care decision-making: an ISPOR-AMCP-NPC Good Practice Task Force report.Value Health. 2014; 17: 143-156
- Best Practices for Conducting Economic Evaluations in Health Care: A Systematic Review of Quality Assessment Tools.Agency for Healthcare Research and Quality (US), Rockville, MD2012
- Methods for the Economic Evaluation of Health Care Programmes.Oxford University Press, Oxford2015
- Criteria list for assessment of methodological quality of economic evaluations: consensus on health economic criteria.Int J Technol Assess Health Care. 2005; 21: 240-245
- Examining the value and quality of health economic analyses: implications of utilizing the QHES.J Manag Care Pharm. 2003; 9: 53-61
- Is there a European view on health economic evaluations? Results from a synopsis of methodological guidelines used in the EUnetHTA partner countries.Pharmacoeconomics. 2016; 34 (EUnetHTA Joint Action 2, Work Package 7, Subgroup 3): 59-76
- Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and Elaboration: A Report of the ISPOR Health Economic Evaluations Publication Guidelines Task Force.Value Health. 2013; 16: 231-250
- iqwig.de. Methods Paper. n.d.
- Pharmacoeconomic Guidelines Around the World. n.d.
- Methods of international health technology assessment agencies for economic evaluations—a comparative analysis.BMC Health Serv Res. 2013; 13: 371
- Pharmacoeconomic guidelines around the world.ISPOR Connect. 2004; 10: 5-15
- Budget impact analysis-principles of good practice: report of the ISPOR 2012 Budget Impact Analysis Good Practice II Task Force.Value Health. 2014; 17: 5-14
- Qualitative and mixed methods provide unique contributions to outcomes research.Circulation. 2009; 119: 1442-1452
- Qualitative methods in patient-centered outcomes research.Qual Health Res. 2017; 27: 434-442
- Complexity of the paradigms present in quality criteria of qualitative research grids.SAGE Open. 2015; 5: 1-13
- Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups.Int J Qual Health Care. 2007; 19: 349-357
- Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ.BMC Med Res Methodol. 2012; 12: 181
- Guidance for developers of health research reporting guidelines.PLoS Med. 2010; 7: e1000217
- Using expert opinion in health technology assessment: a guideline review.Int J Technol Assess Health Care. 2016; 32: 131-139
- Mapping the integration of social and ethical issues in health technology assessment.Int J Technol Assess Health Care. 2007; 23: 9-16
- Guidance for considering ethical, legal, and social issues in health technology assessment: application to genetic screening.Int J Technol Assess Health Care. 2008; 24: 412-422
- EUnetHTA HTA Adaptation Toolkit. n.d.
- Integrating ethics in health technology assessment: many ways to Rome.Int J Technol Assess Health Care. 2015; 31: 131-137
- Revealing and acknowledging value judgments in health technology assessment.Int J Technol Assess Health Care. 2014; 30: 579-586
- Healthcare technology assessment.in: Have H. ten Encyclopedia of Global Bioethics. Springer, New York2015: 1-13
- Complex health care interventions: characteristics relevant for ethical analysis in health technology assessment.GMS Health Technol Assess. 2016; 12: Doc01
- Ethical analysis to improve decision-making on health technologies.Bull World Health Organ. 2008; 86: 617-623
- Methodological guidance documents for evaluation of ethical considerations in health technology assessment: a systematic review.Expert Rev Pharmacoecon Outcomes Res. 2014; 14: 203-220
- INTEGRATE-HTA: adopting and implementing an integrated perspective on complex interventions.J Public Health (Oxf). 2017; 39: 209-212
- An integrated perspective on the assessment of technologies: INTEGRATE-HTA.Int J Technol Assess Health Care. 2017; 33: 1-8
- Framework for systematic identification of ethical aspects of healthcare technologies: the SBU approach.Int J Technol Assess Health Care. 2015; 31: 124-130
- Current state of ethics literature synthesis: a systematic review of reviews.BMC Med. 2016; 14: 152
- Ethics and health technology assessment: handmaiden and/or critic?.Int J Technol Assess Health Care. 2006; 22: 307-312
- Different methods for ethical analysis in health technology assessment: an empirical study.Int J Technol Assess Health Care. 2011; 27: 305-312
- Q-SEA—a tool for quality assessment of ethics analyses conducted as part of health technology assessments.GMS Health Technol Assess. 2017; 13: Doc02
- Ethics expertise for health technology assessment: a Canadian national survey.Int J Technol Assess Health Care. 2014; 30: 131-136
- Key recommendations from the MedtecHTA project.Health Econ. 2017; 26: 145-152
- EUnetHTA Guidelines. n.d.
- Integration of Existing Systematic Reviews.Agency for Healthcare Research and Quality (US), Rockville, MD2014
- Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.BMC Med Res Methodol. 2007; 7: 10
- The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.BMJ. 2009; 339: b2700
- International comparison of the definition and the practical application of health technology assessment.Int J Technol Assess Health Care. 2005; 21: 89-95
- Methods for Meta-Analysis in Medical Research. n.d.
- Interpreting indirect treatment comparisons and network meta-analysis for health-care decision-making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 1.Value Health. 2011; 14: 417-428
- Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision-making: an ISPOR-AMCP-NPC Good Practice Task Force report.Value Health. 2014; 17: 157-173
- Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis.J Clin Epidemiol. 2018; 93: 36-44
- A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis.BMJ. 2014; 349: g5630
- The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations.Ann Intern Med. 2015; 162: 777-784
- RAMESES publication standards: meta-narrative reviews.BMC Med. 2013; 11: 20
- RAMESES publication standards: realist syntheses.BMC Med. 2013; 11: 21
- Protocol-developing meta-ethnography reporting guidelines (eMERGe).BMC Med Res Methodol. 2015; 15: 103
- Assessing the Quality of Mixed Methods Research: Toward a Comprehensive Framework.Sage Inc., Thousand Oaks, CA2010: 531-556
- Using qualitative evidence in decision-making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).PLoS Med. 2015; 12: e1001895
- Deliberation as discussion.in: Elster J. Deliberative Democracy. Cambridge University Press, Cambridge1998: 44-68
- Value assessment frameworks for HTA agencies: the organization of evidence-informed deliberative processes.Value Health. 2017; 20: 256-260
- Deliberative Processes in Decisions About Health Care Technologies: Combining Different Types of Evidence, Values, Algorithms and People.Office of Health Economics, London2009
- Final Report: Conceptualizing and Combining Evidence for Health System Guidance.Canadian Health Services Research Foundation, Ottawa, ON2005
- Patient-based health technology assessment: a vision of the future.Int J Technol Assess Health Care. 2007; 23: 30-35
- Bringing “the public” into health technology assessment and coverage policy decisions: from principles to practice.Health Policy. 2007; 82: 37-50
- Patient and public involvement policy. n.d.https://www.nice.org.uk/about/nice-communities/public-involvement/patient-and-public-involvement-policyDate accessed: March 27, 2017
- Value to whom? The patient voice in the value discussion.Value Health. 2017; 20: 286-291
- Exploring qualitative research synthesis: the role of patients’ perspectives in health policy design and decision-making.Patient. 2011; 4: 143-152
- Patient-centric HTA: different strokes for different folks.Expert Rev Pharmacoecon Outcomes Res. 2015; 15: 591-597
- Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute.Qual Life Res. 2015; 24: 1033-1041
- Conjoint analysis applications in health—a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force.Value Health. 2011; 14: 403-413
- Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force.Value Health. 2013; 16: 3-13
- Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR Conjoint Analysis Good Research Practices Task Force.Value Health. 2016; 19: 300-315
- Patients’ perspectives in health technology assessment: a route to robust evidence and fair deliberation.Int J Technol Assess Health Care. 2010; 26: 334-340
- Guidance for patient involvement in HTA. 2016.https://www.eupati.eu/health-technology-assessment/guidance-for-patient-involvement-in-hta/Date accessed: December 8, 2017
- Multiple criteria decision analysis for health care decision-making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force.Value Health. 2016; 19: 1-13
- Multi-Criteria Decision Analysis to Support Healthcare Decisions.Springer, New York2017
- Priority setting for health technology assessment at CADTH.Int J Technol Assess Health Care. 2010; 26: 341-347
- Integrating patients’ views into health technology assessment: Analytic hierarchy process (AHP) as a method to elicit patient preferences.Int J Technol Assess Health Care. 2011; 27: 369-375
- Advancing MCDA and HTA into Coverage Decision-Making. Multi-Criteria Decision Analysis to Support Healthcare Decisions.Springer, Cham, Switzerland2017: 119-146
- Implementation of EUnetHTA core Model® in Lombardia: the VTS framework.Int J Technol Assess Health Care. 2014; 30: 105-112
- Which health technologies should be funded? A prioritization framework based explicitly on value for money.Isr J Health Policy Res. 2012; 1: 44
- Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada.BMC Health Serv Res. 2011; 11: 329
- Multicriteria decision analysis for including health interventions in the universal health coverage benefit package in Thailand.Value Health. 2012; 15: 961-970
- Extended cost-effectiveness analysis for health policy assessment: a tutorial.Pharmacoeconomics. 2016; 34: 913-923
- The NICE cost-effectiveness threshold: what it is and what that means.Pharmacoeconomics. 2008; 26: 733-744
- Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold.Health Technol Assess. 2015; 19: 1-503, v-vi
- Systematic overview of cost-effectiveness thresholds in ten countries across four continents.J Comp Eff Res. 2015; 4: 485-504
- Transferability of economic evaluations across jurisdictions: ISPOR Good Research Practices Task Force report.Value Health. 2009; 12: 409-418
- Challenges faced in transferring economic evaluations to middle income countries.Int J Technol Assess Health Care. 2015; 31: 442-448
- Variability of cost-effectiveness estimates for pharmaceuticals in Western Europe: lessons for inferring generalizability.Value Health. 2005; 8: 10-23
- What do international pharmacoeconomic guidelines say about economic data transferability?.Value Health. 2010; 13: 1028-1037
- EUnetHTA information management system: development and lessons learned.Int J Technol Assess Health Care. 2014; 30: 514-520
- Danske Regioner—Medicinrådet—In English. n.d.
- Assessing Resource Impact Methods Guide 2011.NIHCE, London2011
- Final Value Assessment Framework for 2017-2019. n.d.
- Knowledge Transfer Study Group. How can research organizations more effectively transfer research knowledge to decision makers?.Milbank Q. 2003; 81 (171-172): 221-248
- SUPPORT Tools for evidence-informed health policymaking (STP).Health Res Policy Syst. 2009; 7: I1
- Health Systems Evidence. n.d.
- The AMCP format for formulary submissions version 3.0.J Manag Care Pharm. 2010; 16: 1-30
- Submission template for pharmaceuticals and submission template for medical devices. n.d.https://www.eunethta.eu/services/submission-guidelines/submission-template-pharmaceuticals-submission-template-medical-devDate accessed: January 24, 2018
- HTA implementation roadmap in Central and Eastern European countries.Health Econ. 2016; 25: 179-192
- The Pharmaceutical Price Regulation Scheme. 02/07. Office of Fair Trading, London2007
- Influence of health technology assessment and its measurement.Int J Technol Assess Health Care. 2016; 32: 376-384
- What are the effects of HTA reports on the health system? Evidence from the research literature.in: Velasco Garrido M. Kristensen F.B. Palmhøj Nielsen C. Busse R. Health Technology Assessment and Health Policy-Making in Europe: Current status, Challenges and Potential. WHO, Geneva2008: 109-136
- The impact of HTA reports on health policy: a systematic review.GMS Health Technol Assess. 2005; 1: Doc02
- Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme.Health Technol Assess. 2016; 20: 1-254
- Returns on research funded under the NIHR Health Technology Assessment (HTA) Programme: economic analysis and case studies.Rand Health Q. 2016; 5: 5
- Assessing the impact of health technology assessment.Int J Technol Assess Health Care. 1997; 13: 68-80
- Assessing the impact of health technology assessment on the Austrian healthcare system.Int J Technol Assess Health Care. 2013; 29: 84-91
- The impact of health technology assessment reports on decision-making in Austria.Int J Technol Assess Health Care. 2012; 28: 77-84
- The impact of HTA and procurement practices on the selection and prices of medical devices.Soc Sci Med. 2017; 174: 89-95
- What is technology assessment?.Int J Technol Assess Health Care. 2009; 25: 7-9
- Evidence synthesis and evidence-based decision-making: related but distinct processes.Med Decis Making. 2005; 25: 487-489
- Variation in health technology assessment and reimbursement processes in Europe.Value Health. 2017; 20: 67-76
- A health economics approach to US value assessment frameworks—summary and recommendations of the ISPOR Special Task Force Report .Value Health. 2018; 21: 161-165
- Changing health technology assessment paradigms?.Int J Technol Assess Health Care. 2016; 32: 191-199
- Early dialogue consolidated procedure. n.d.http://eunethta.eu/outputs/eunethta-early-dialogue-consolidated-procedureDate accessed: January 19, 2018
- Interactions between health technology assessment, coverage, and regulatory processes: emerging issues, goals, and opportunities.Int J Technol Assess Health Care. 2011; 27: 253-260
- EMA and EUnetHTA finalise joint work plan for 2017-2020. n.d.
Author Contributions: The ISPOR HTA Council provided the original direction for this work. F.B.K. and D.H. led the writing of the article, including the drafting of the outline and article, and are the guarantors of this work. All authors approved the outline of the work, helped to write and revise the article, and read and approved the final version of the article.
Disclosures: None of the authors have received funding to draft the report. All authors have signed the ICJME Form for Disclosure of Potential Conflicts of Interest and read information regarding disclosure of potential conflict of interest and have made declarations based on these. Every attempt was made to conduct this research in compliance with the 2017 ISPOR Code of Ethics.
User licenseElsevier user license |
For non-commercial purposes:
- Read, print & download
- Text & data mine
- Translate the article
- Reuse portions or extracts from the article in other works
- Redistribute or republish the final article
- Sell or re-use for commercial purposes
Elsevier's open access license policy