Advertisement

ISPOR Code of Ethics 2017 (4th Edition)

  • Jessica Santos
    Affiliations
    Global Compliance and Quality Director, Kantar Health, Cambridge, Wales, UK
    Search for articles by this author
  • Francis Palumbo
    Correspondence
    Address correspondence to: Frank Palumbo.
    Affiliations
    Professor and Executive Director, Center on Drugs & Public Policy, University of Maryland School of Pharmacy, Bethesda, MD, USA
    Search for articles by this author
  • Elizabeth Molsen-David
    Affiliations
    Director, Scientific & Health Policy Initiatives, ISPOR, Lawrenceville, NJ, USA
    Search for articles by this author
  • Richard J. Willke
    Affiliations
    Chief Science Officer, ISPOR, Lawrenceville, NJ, USA
    Search for articles by this author
  • Author Footnotes
    Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Louise Binder
    Footnotes
    † Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Affiliations
    Health Policy Consultant, Save Your Skin Foundation, Toronto, ON, Canada
    Search for articles by this author
  • Author Footnotes
    Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Michael Drummond
    Footnotes
    † Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Affiliations
    Co-Editor-in-Chief, Value in Health and Professor, Health Economics, Centre for Health Economics, University of York, Heslington, York, England, UK
    Search for articles by this author
  • Author Footnotes
    Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Anita Ho
    Footnotes
    † Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Affiliations
    Associate Professor, Director of Undergraduate Medical Ethics Curriculum, Centre for Biomedical Ethics, National University of Singapore, Singapore
    Search for articles by this author
  • Author Footnotes
    Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    William D. Marder
    Footnotes
    † Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Affiliations
    Chief Data Strategist, IBM Watson Health, Cambridge, MA, USA
    Search for articles by this author
  • Author Footnotes
    Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Louise Parmenter
    Footnotes
    † Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Affiliations
    Vice President, Global Head Scientific Affairs, QuintilesIMS, Reading, England, UK
    Search for articles by this author
  • Author Footnotes
    Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Gurmit Sandhu
    Footnotes
    † Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Affiliations
    Patient Engagement Specialist, Gurmit Sandhu Consulting GmbH, Basel, Switzerland
    Search for articles by this author
  • Author Footnotes
    Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Asrul A. Shafie
    Footnotes
    † Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Affiliations
    Associate Professor, University Sains Malaysia, Member of National Good Governance of Medicine Policy and member of Ethics Board in Malaysia, Minden, Malaysia
    Search for articles by this author
  • Author Footnotes
    Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    David Thompson
    Footnotes
    † Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
    Affiliations
    Editor-in-Chief, Value & Outcomes Spotlight and Senior Vice President, Real World Evidence Advisory, INC Research / inVentiv Health, Boston, MA, USA
    Search for articles by this author
Open ArchivePublished:December 01, 2017DOI:https://doi.org/10.1016/j.jval.2017.10.018

      Abstract

      As the leading health economics and outcomes research (HEOR) professional society, ISPOR has a responsibility to establish a uniform, harmonized international code for ethical conduct. ISPOR has updated its 2008 Code of Ethics to reflect the current research environment. This code addresses what is acceptable and unacceptable in research, from inception to the dissemination of its results.
      There are nine chapters: 1 – Introduction; 2 – Ethical Principles respect, beneficence and justice with reference to a non-exhaustive compilation of international, regional, and country-specific guidelines and standards; 3 – Scope HEOR definitions and how HEOR and the Code relate to other research fields; 4 – Research Design Considerations primary and secondary data related issues, e.g., participant recruitment, population and research setting, sample size/site selection, incentive/honorarium, administration databases, registration of retrospective observational studies and modeling studies; 5 – Data Considerations privacy and data protection, combining, verification and transparency of research data, scientific misconduct, etc.; 6 – Sponsorship and Relationships with Others (roles of researchers, sponsors, key opinion leaders and advisory board members, research participants and institutional review boards (IRBs) / independent ethics committees (IECs) approval and responsibilities); 7 – Patient Centricity and Patient Engagement new addition, with explanation and guidance; 8 – Publication and Dissemination; and 9 – Conclusion and Limitations.

      Background to the Task Force

      The last ISPOR Code of Ethics (Code) was published in 2008. The Code needed to be updated to reflect the Society’s immense growth in both membership and geographic coverage. The HEOR profession and research landscape has changed dramatically with the increased collection and use of real-world data, health information technology (HIT), genomic information and social media. Other issues such as data privacy and patient centricity have arisen.
      In November 2015, the ISPOR Health Science Policy Council recommended the task force proposal to update the Code, with the ISPOR Board of Directors approving the Code of Ethics Task Force in December 2015.
      The task force was composed of a geographically diverse group of stakeholders from research organizations, academia, patient organizations, and ISPOR leadership members.
      To develop a broad, consensus and representative Code, the task force presented its findings to date during two Code of Ethics Forum presentations in 2016 at the ISPOR International Meeting in Washington, DC, and the European Congress in Vienna, Austria. In addition, ISPOR’s membership and the regional chapters and consortia were invited to submit comments during the review rounds. ISPOR members submitting written comments are listed by name in the report’s acknowledgments section.
      The ISPOR Code of Ethics Task Force distilled the Code's main points into a 72-point Code summary that is woven through the Code’s chapters. The summary, in its entirety, and an acronym glossary follow the report. Additional material can be found in 10 detailed appendices that include other relevant codes of ethics, HEOR data sources, data protection considerations, recruitment, safety and reporting, incentive and disclosure requirements, IRB/IEC roles and research participant involvement. ISPOR’s Code of Ethics 2017 (4th Edition), the 72-point summary and appendices will be available on the ISPOR website: https://www.ispor.org.
      Co-Authors are listed alphabetically after the fourth author, Richard J. Willke.
      ISPOR VISION: ISPOR is the leading global scientific and educational organization for health economics and outcomes research and their use in decision making to improve health.
      ISPOR’s MISSION: To promote health economics and outcomes research excellence to improve decision making for health globally.

      Preamble to the ISPOR Code of Ethics

      ISPOR expects its members to adhere to the highest ethical standards. ISPOR’s activities and those of its members affect a number of constituencies. These include, but are not limited to:
      • Patients, Caregivers, and Patients’ Associations—who are ultimately going to experience the greatest impact of the ISPOR guidelines, research, and initiatives.
      • Health Care Professionals (HCPs)—who will be treating or not treating patients with therapies, medications, and procedures made available or not made available due to health care research.
      • Health Care Organizations (HCOs)—hospitals, clinics, other health care settings; the care provided greatly affects health outcomes, quality of care, and patient satisfaction.
      • Decision Makers and Payers—who must decide what is covered so as to optimize (1) the health of patients and (2) resource utilization. This includes:
        • Government Groups—who require the results of health care research to set policy and prices, such as the U.S. Food and Drug Administration (FDA) or the European Medicines Agency (EMA).
        • Insurers—who base health care coverage and/or payment decisions on health care research.
        • Employers—where health care research affects their decisions on providing health benefits.
        • Administrators and Other Stakeholders, such as managed care personnel—who need results that are both practical and useful.
      • Professional Outcomes Researchers—who perform HEOR as a profession independently or on behalf of another party.
      • Industries/Manufacturers—whose products are often the subject or focus of health care research, including, but not limited to, pharmaceutical, medical device, and sanitary technologies.
      • Academic Institutions and Universities—where research is conducted and students are trained.
      • Colleagues—where relationships in conducting research and related activities are particularly critical.
      • Research Employees—who are concerned about how they are regarded, compensated, and treated by the researchers for whom they work.
      • Students/Trainees—where respect and appropriate behavior of researchers and employers are important. They are the future of the profession.
      • Clients—for whom health care research is conducted and researcher relationships are developed and maintained.
      Through behaviors and practices intended to ensure that health care research is designed, conducted, and reported in the most proper and ethical way possible, the ISPOR Code of Ethics (Code) is a means for the HEOR field to avoid or address credibility challenges based on methodology or bias concerns. By accomplishing this, the various affected constituencies will be able to trust and benefit from research findings as much as possible.
      The Code also includes some general ethical considerations for the Society. As part of membership, members agree to comply with the Code when they join or renew membership annually. However, we recognize that members’ own organizations may also have ethical codes that should be followed.
      We recognize that there may also be relevant legal considerations. A member’s unethical behavior or practice reflects poorly on the Society and the HEOR community’s reputation. ISPOR may deny or revoke membership and participation in groups or meetings if a member is convicted of a felony, other act, moral turpitude or upon suspension of a license in a medical or health profession.
      In addition, ISPOR expects all of its representatives and members to act in a manner consistent with United Nations Convention Against Corruption [

      United Nations Convention Against Corruption. Available from: https://www.unodc.org/unodc/en/treaties/CAC/. [Accessed September 25, 2017].

      ], a multilateral convention of UN members, including the United States; the U.S. Foreign Corrupt Practices Act (FCPA) [

      The Foreign Corrupt Practices Act of 1977, 15 U.S.C. §§ 78dd-1, et seq. (FCPA). Available from: https://www.justice.gov/criminal-fraud/foreign-corrupt-practices-act. [Accessed September 25, 2017].

      ]; and national or international bribery and anticorruption laws. Finally, it is ISPOR’s policy to comply fully and strictly with both U.S. federal and state antitrust laws, and other applicable international antitrust laws and regulations.

      Chapter 1: Introduction to the ISPOR Code of Ethics

      As the leading HEOR
      Pharmacoeconomics is a subdiscipline of health economics. The ISPOR Code of Ethics uses the broader term, health economics, combined with outcomes research to form health economics and outcomes research or HEOR, which has become predominant since ISPOR was founded more than 20 years ago.
      professional society, ISPOR has a responsibility to establish a uniform, harmonized international set of standards or guidelines for members to follow. Since 1998, an ISPOR Code of Ethics has been publicized to HEOR practitioners. This latest 2017 4th edition reflects the changing environment in which ISPOR and its membership conduct research.
      Those practicing in the HEOR area have a long history of civil discourse and of developing “good practices” associated with different research methods. Such discussions and the templates developed are ways to reduce the unwarranted variation in professional outputs. Nonetheless, a code of ethics differs from a recommended good or best practice recommendation, and is concerned with principles, such as informed consent, data privacy, and equity in health care.
      The core principles embodied in a code of ethics represent values that, on the one hand, must not be compromised but, on the other hand, may need to be weighed against one another. They are the guiding standards that are essential for the professionalism of researchers and the confidence that users and members of other professions can have in HEOR.
      The composition of ISPOR as an organization is an important preface to what is to follow. The global nature of ISPOR sets it apart from many other organizations, with differences in cultures and, sometimes, points of view on important issues, such as data privacy.
      ISPOR members represent multiple disciplines that approach intellectual problems in HEOR with a variety of tools and research designs. They differ in the relationships that they have with different health care systems around the globe. They come from diverse employment settings with complex and dynamic structures.
      As a multidisciplinary, global organization, ISPOR strives for representativeness, transparency, and balance in its activities, thereby avoiding the appearance of bias or conflict of interest. This includes, but is not limited to, sponsorship of its conferences and other activities, as well as the selection of presenters at its conferences. To the extent that it is feasible, ISPOR program planning and selection committees should have a membership representative of its major constituencies (see Preamble). ISPOR should also have a Board of Directors that is representative of the various constituencies the Society serves.
      Furthermore, because significant research funding will come from funders with interests in specific findings (at times commercial, private nonprofit, as well as governmental institutions), ISPOR should continue to maintain its own statement of objectivity and autonomy. ISPOR strives to ensure that its journals, Value in Health, Value in Health Regional Issues, and any other ISPOR journal, only publish papers that have gone through a rigorous peer-review process and whose authors are listed pursuant to strict criteria.
      Even though economics, price and coverage discussions and the like are a major part of ISPOR's identity, they should not be construed as encompassing ISPOR’s total identity. Rather, ISPOR is conscious of broader ethical issues affecting global and regional medical resource allocation, public health policies, and the global health care environment or are relevant to research topics, such as patient autonomy, patient outcomes, and research conduct. These issues include, but are not limited to, prejudice, equity in health care delivery, and patient access.
      The HEOR profession and research landscape have changed dramatically since the publication of the 2008 Code (Appendix 1) [

      International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Code of Ethics (2008). Available from: https://www.ispor.org/workpaper/CodeOfEthics.asp.

      ], with the increased collection and use of real-world data, health information technology (HIT), genomic information, and social media, plus the current focus on patient centricity and data privacy issues, among others. Furthermore, most professional codes that ISPOR referenced in the past have been updated since last publication. Finally, due to the Society’s immense growth in both membership and geographic coverage, it is important to recognize that there may be conflicting standards of professional conduct in regions of the world that now need to be considered in ISPOR’s 2017 Code (4th edition).
      Rather than merely reducing unwarranted variance, a code of ethics is intended to promulgate the standards that define what is acceptable and unacceptable in the conduct of all aspects of research, from its inception to the dissemination of its results. This revised Code represents a collective effort to articulate those standards.
      Therefore:
      • 1.
        The ISPOR Code of Ethics was developed as guidance for the health economics and outcomes research community as a whole.
      • 2.
        ISPOR strives for representativeness, transparency, and objectivity in its activities.
      • 3.
        ISPOR has a Board of Directors that is representative of the various constituencies that the Society serves.
      • 4.
        The ISPOR program planning and selection committees should have membership representative of its major constituencies.
      • 5.
        ISPOR strives for a balance in sponsorship of its activities by providing decision criteria for acceptance and disbursement of funds to ensure full transparency and minimize the possibility of bias or conflict of interest.
      • 6.
        Like other professional societies, ISPOR is conscious of broader ethical issues that impact global and regional medical resource allocation, public health policies, and the global health care environment or are relevant to research topics such as patient autonomy, patient outcomes, and research conduct. These issues include, but are not limited to, prejudice, equity in health care delivery, and access.

      Chapter 2: Application of Ethical Principles to the ISPOR Code of Ethics

      Both the past and the current ISPOR Codes of Ethics draw from international standards and guidelines. A non-exhaustive compilation of international, regional, and country-specific guidelines and standards in the research field, including patient engagement resources and publication ethics codes, were reviewed and summarized (Appendix 2). This range of standards includes, but is not limited to, the World Medical Association’s Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects [

      World Medical Association. Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. [Accessed September 26, 2017].

      ], the International Council on Harmonization Good Clinical Practice (ICH GCP) [

      International Conference on Harmonization Good Clinical Practice (ICH GCP) Available from: http://ichgcp.net. [Accessed September 26, 2017].

      ], the U.S. Agency for Healthcare Research and Quality (AHRQ) National Guideline Clearinghouse for summaries of evidence-based clinical practice guidelines [

      Agency for Healthcare Research and Quality (AHRQ). AHRQ’s National Guideline Clearinghouse. Available from: https://www.guideline.gov/.

      ], the European Federation of Pharmaceutical Industries and Associations (EFPIA) [
      European Federation of Pharmaceutical Industries and Associations (EFPIA)
      ], and the European Patients Academy (EUPATI) for guidance on patient involvement in research and development [

      European Patients Academy (EUPATI). Guidance for Patient Involvement in R&D. Available from: https://www.eupati.eu/guidance-patient-involvement/.

      ] and health technology assessment (HTA) [

      European Patients Academy (EUPATI) Guidance for patient involvement in HTA. Available from: https://www.eupati.eu/health-technology-assessment/guidance-for-patient-involvement-in-hta/. [Accessed September 22, 2017].

      ].
      Therefore:
      • 7.
        Members should maintain current knowledge of research practices, of general principles, and of local and regional relevant practices.
      The ISPOR Code closely follows the Belmont Report’s [

      U.S. Department of Health, Education, and Welfare (USHHS). The Belmont Report. (Office of the Secretary Ethical Principles and Guidelines for the Protection of Human Subjects of Research), The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research) April 18, 1979. Available from: https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/index.html.

      ] three fundamental ethical principles that form the basis for the National Commission’s topic-specific reports and the regulations that incorporate its recommendations. Application of these principles requires careful consideration of informed and voluntary consent, risks and benefits, and the selection of participants for research.
      Respect for persons—protecting the autonomy of all people; treating them with courtesy and respect; and allowing for informed and voluntary consent. Researchers must be truthful and conduct no deception;
      Beneficence—the philosophy of “Do no harm” while maximizing benefits for the research and minimizing risks to the research participants; and
      Justice—ensuring reasonable, non-exploitative, and well-considered procedures are administered fairly—the fair distribution of costs and benefits to potential research participants—and equally.
      ISPOR’s Code places additional emphasis on privacy, transparency, and civility. This reflects the responsibilities associated with increased data access, the global nature of research, and a broad range of research participants and health care system stakeholders.
      Therefore:
      • 8.
        Members must conduct activities honestly, with integrity and good judgment, and in the best interests of the patients, health care professionals, decision makers, outcomes researchers, pharmaceutical manufacturers, and other public health communities we serve.
      • 9.
        Privacy and confidentiality: It is essential that protected health information (PHI) and other personal data of patients are handled with the utmost care so that patient confidentiality is maintained at all times and that no breaches to patient privacy occur.
      • 10.
        Transparency and Integrity: Members must disclose research methods in sufficient detail to permit replication. Funding sources should be clearly acknowledged, and any conflicts of interest declared [
        • Wang S.V.
        • Schneeweiss S.
        • Berger M.L.
        • et al.
        Reporting to improve reproducibility and facilitate validity assessment for healthcare database studies.
        ,
        • Berger M.L.
        • Sox H.
        • Willke R.J.
        • et al.
        Good practices for real-world data studies of treatment and/or comparative effectiveness: recommendations from the Joint ISPOR-ISPE Special Task Force on Real-World Evidence in Health Care Decision Making.
        ].
      • 11.
        Designing, conducting, and especially reporting of the study should be an objective (unbiased) reflection of the full range of findings generated.
      • 12.
        Civility: Members’ research and discussion should respect the dignity of all participants. Respecting the dignity of patients and providers of care is clearly a top responsibility. It is also a responsibility to treat fellow researchers with respect.
      All HEOR studies should respect and protect the human subjects enrolled in these studies, using the principles of the Declaration of Helsinki (1964–2013) [

      World Medical Association. Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. [Accessed September 26, 2017].

      ]. Medical research is subject to ethical standards that promote and ensure respect for all human subjects and protect their health and rights. While the primary purpose of medical research is to generate new knowledge, this goal can never take precedence over the rights and interests of individual research participants.

      Chapter 3: Scope of the ISPOR Code of Ethics

      The ISPOR Code of Ethics is specifically oriented to HEOR. While there is overlap with other fields, our goal is a discipline-oriented code. It is important to note that the scope of ISPOR’s Code of Ethics does not include ethical considerations related to the use or impact of specific HEOR measures (e.g. potential age-related biases implicit in quality-adjusted life-years).
      The Code does not cover societal decision making based on HEOR evidence, such as the formation of HTA policies. As long as reporting of research is complete and transparent, users of ISPOR members’ research can judge use or impact issues independently. For more on these issues, please refer to the Recommendations for Conduct, Methodological Practices, and Reporting of Cost-effectiveness Analyses Second Panel on Cost-Effectiveness in Health and Medicine [

      Sanders GD, Neumann PJ, Basu A, et al., eds. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses second panel on cost-effectiveness in health and medicine. JAMA 2016;316(10):1093–103.

      ].
      HEOR is a multidisciplinary field that combines aspects of health economics with the methods of outcomes research in the evaluation of the impact of health care interventions on patient well-being, population health, and health system efficiency. It employs economic and patient-centered outcomes to complement traditional clinical development information (e.g., efficacy, safety, quality) to provide broader evidence about the long-term clinical effectiveness, benefits, risks, and economic effects (e.g., costs and cost-effectiveness) of treatments with the aim to improve healthcare decision making.
      Please note that this is a working definition for HEOR. A formal definition will appear in the next edition of Health Care Cost, Quality and Outcomes: ISPOR Book of Terms. Expected publication: 2019
      Health economics is a discipline that analyzes the aspects of all activities designed to improve or maintain health and health care, typically focusing on the costs (inputs) and the consequences (outcomes) of health care interventions. It is concerned with issues related to efficiency, effectiveness, utility, value, quality, ethics and behavior in the production and consumption of health and health care, evaluation and analysis of population health access, and resource allocation to benefit society given scarcity. In broad terms, health economists study the functionality of health care systems and health-affecting behaviors [
      ].
      Outcomes research is the scientific discipline that evaluates the effects of health care interventions on patient well-being, including clinical, economic, and patient-centered outcomes.

      Difference and Relationship to Other Research Fields

      HEOR is closely related to other common research types, such as clinical trial/studies, noninterventional observations, epidemiologic investigations, real-world research, and market research studies. (See Appendix 3 for more information.)
      There is no single legal instrument or practical guidance for HEOR. At times, this results in differences in definitions and terms across groups and countries. HEOR can utilize any techniques from the research types mentioned above. The objective is to evaluate the effect of health care interventions on patient well-being, including clinical, economic, and patient-centered and other relevant outcomes, as well as the functioning of health care systems and health-affecting behaviors.
      Therefore:
      • 13.
        Members should adhere to the standards of practice for their respective fields of research; hence they should identify and acknowledge any applicable official guidelines and standards used.
      This 2017 Code of Ethics covers the following five topics in depth: research design, data considerations, sponsorship, patient engagement, and publication and dissemination, with appendices providing ancillary detail to these sections.

      Chapter 4: Research Design Considerations

      HEOR comprises a range of research designs from modeling and retrospective analyses using secondary data to prospective observational and clinical trial designs. (See Appendix 4 for more on HEOR data sources.) No matter the chosen research design, HEOR is conducted following the core scientific principles of objectivity, transparency, reporting, and quality assurance. It is defined by the objective(s) and the approach, not by the title of the work or the role of those commissioning it.

      Primary Data-Related Research Considerations

      Participant Recruitment

      ISPOR recognizes that study participants can be recruited via a number of methods. Appendix 5
      • 14.
        From the point of “first contact,” researchers should provide potential subjects information about study intentions and how the research is funded, as well as all information mandated in their proposals to institutional review boards (IRBs) / independent ethics committees (IECs)
        Also known as ethical review boards (ERBs), research ethics boards (REBs), or ethics committees (ECs).
        according to the primary objective with patient consent.
        Also known as ethical review boards (ERBs), research ethics boards (REBs), or ethics committees (ECs).

      Population and Research Setting

      Researchers should be specific with regard to population and setting.
      Therefore:
      • 15.
        Members should describe the analytic study population in terms of its relevant demographic and medical characteristics, geography, time period, and selection criteria.
      • 16.
        Members should justify the chosen target population and choose a suitable research setting for this population. In addition, existing data or literature should be included to provide information about the specific population to which the study results will be applied.

      Sample Size, Site Selection

      Study sample size and sites should not be larger than statistically necessary. However, inadequate sample size (too small) may provide insufficient data to answer the intended research questions or will provide low precision [

      International Conference on Harmonization Good Clinical Practice (ICH GCP) Available from: http://ichgcp.net. [Accessed September 26, 2017].

      ,

      European Medicines Agency (EMA). Guideline on good pharmacovigilance practices (GVP), Module VIII – Post-authorization safety studies (Rev 2). Doc Ref. EMA/813938/2011. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129137.pdfc. [Accessed August 4, 2017].

      ].
      Therefore:
      • 17.
        The number of patients and sites selected for a study should be appropriate to meet the research objectives without being greater than statistically necessary.

      Safety/(Serious) Adverse Events

      Safety and adverse event reporting (AER) is an important aspect of all primary research involving patients and medical interventions. The Guideline on Good Pharmacovigilance Practices (GVP) [

      European Medicines Agency (EMA). Guideline on good pharmacovigilance practices (GVP), Module VIII – Post-authorization safety studies (Rev 2). Doc Ref. EMA/813938/2011. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129137.pdfc. [Accessed August 4, 2017].

      ] described in the European Union’s Directive 2010/84/EU [

      European Union. Directive 2010/84/EU of the European Parliament and of the Council (15 Dec 2010). Official Journal of the European Union. Available from: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:348:0074:0099:EN:PDF. [Accessed September 6, 2016].

      ] applies to investigational medicinal products and non-investigational medicinal products. Similar regulations exist in most other jurisdictions. AER is applicable to some HEOR activities, including clinical trials, primary research, noninterventional studies, market research, and real-world research. (For more information, see Appendix 6.)
      Researchers are expected to collect and report adverse events, not only to comply with regulatory and legal requirements, but also with an understanding of the responsibility to patients and society to comprehensively inform the safety of treatment options.
      A strong international collaborative approach to post-approval surveillance and mandatory adverse reporting is critical. Data collected through social and digital media can be useful, but often clear pharmacovigilance reporting guidelines are not followed because there is neither a single marketing authorization holder nor a single regulatory/oversight authority.
      Therefore:
      • 18.
        The balance of risk or harm to benefit for patients must be considered in HEOR studies and must be communicated to patients via informed consent.
      • 19.
        Safety and adverse event reporting (AER) is an important aspect of all primary research involving patients and medical interventions. It is applicable to many HEOR activities and must follow the most up-to-date international and local guidelines.

      Incentive/Honorarium

      An incentive or honorarium is any benefit given to a participant to encourage participation in a research study. It is commonly used in prospective research and surveys to provide participants with compensation for expenses that may be incurred as part of participating in research. Remuneration is compensation to investigators or consultants for their work or contribution to the study. (For specific details on incentives and honoraria, see Appendix 7.)
      Therefore:
      • 20.
        Any incentive, honorarium or payment is subject to receivers’ and providers’ internal compliance guidelines. IRB/IEC approval and must be detailed in the proposal submitted for review.
      • 21.
        Researchers should be diligent in ensuring that the incentive would not induce research participants to accept risks they would not be willing to accept if they were offered a smaller or no incentive.

      Secondary Data-Related Research Considerations

      Administrative Databases and Other Large Datasets

      Health care systems generate operational and administrative data that have been used extensively in HEOR studies. HEOR uses a wide range of secondary research sources, including proprietary databases, claims databases, patient registries, routine data sources, electronic health records, systematic reviews, evidence synthesis, social media, internet, and other related sources. Data can range from a longitudinal administrative database to a constant flow from the internet and wearable devices or from controlled clinical trials to unstructured social media feeds.
      Examples include governmental databases, such as the individual data sources maintained and made available by the U.S. Center for Medicare and Medicaid Services (CMS) Chronic Condition Data Warehouse (CCW) [

      Center for Medicare and Medicaid Services (CMS) Chronic Conditions Data Warehouse (CCDW). Available from: https://www.ccwdata.org.

      ] and SEER Medicare [

      American Academy of Otolaryngology – Head and Neck Surgery, Surveillance, Epidemiology and End Results (SEER). Available from: http://www.entnet.org/content/surveillance-epidemiology-and-end-results-seer-medicare.

      ], the United Kingdom’s HES [

      Hospital Episodes Statistics (HES). Available from: http://digital.nhs.uk/hesdata.

      ], France’s SNIIRAM [

      Système National d'Informations Inter Régimes de l'Assurance Maladie (SNIIRAM) (French: National Information System Inter Plans Health Insurance). Available from: http://www.ameli.fr/.

      ], and so on, as well as a number of private databases. Some research involves combining various datasets (e.g., Medicare Current Beneficiary Survey and Medicare Claims Parts A, B, C, or D). This diversity in types of data sets presents multiple analytic challenges.
      Because the data were initially collected for another purpose, the first important step for those creating and then using secondary data is to be sure that all intellectual property rights have been respected and that the appropriate permissions have been secured. This is typically done by the database supplier. These permissions include protection of the privacy of the individuals whose characteristics are captured in the database, as well as their informed consent for secondary use of their data, where applicable. Privacy is discussed below.
      The cost of creating databases for secondary use is sometimes borne by governments, and the users are charged nominal fees. When private entities build databases for secondary use, they will often do so in anticipation of user fees that help make database creation and distribution a worthwhile commercial endeavor. In either case, the researcher needs to be assured that the database was legally and ethically constructed.
      The vast majority of HEOR studies currently conducted involve the analysis of secondary data. Retrospective observational studies are often conducted using administrative databases or clinical registries. Modeling studies involve the synthesis and analysis of data from several sources, including previously conducted clinical trials, clinical registries, routinely available cost data, and the published literature.
      The use of secondary data has ethical and legal challenges related to the collection and storage of personal data that are different from those in primary research studies (discussed above) because the data are already anonymized. If there is doubt or moral concern regarding how the secondary data were generated, researchers can consider a due diligence process on the data source before using it or can use an alternative data set for the study.
      There are instances where a secondary database may not be considered de-identified. One example is the CMS CCW, where age and postal zip code information are included. However, given the large degree of analyst discretion, secondary research studies do raise a number of ethical challenges related to the avoidance of methodologic bias due to the selective use of the available data and the inappropriate use of assumptions with regard to factors such as missing data, the nature of selection bias, outliers, and so on.
      Therefore, the most important general ethical principles in the analysis of secondary data are those of transparency and reasonableness—that is, in the absence of consensus on principles, “a fair process allows us to agree on what is legitimate and fair” [
      • Daniels N.
      Accountability for reasonableness: an update.
      ].
      Therefore:
      • 22.
        When using secondary data sources initially collected for another purpose, HEOR researchers should ensure that intellectual property rights are respected and referenced and that all the appropriate permissions have been secured.
      • 23.
        Given the potential for bias in the analysis of secondary data, the most important general ethical principles are those of reasonableness and transparency.
      • 24.
        Any known or potential source of bias in the data that can affect the results must be disclosed whenever secondary data are used.
      • 25.
        In those instances in which study methods include analysis of a database, members should describe approaches, methods, technologies used to ensure data completeness and validity, as well as the software package(s) used for data analysis. Members should have the education, training, and experience to perform the assigned tasks or provide evidence of collaboration with individuals who are qualified.

      Registration of Retrospective Observational Studies

      For purposes of this Code, observational studies are defined as analysis of existing datasets [
      • Parmenter L.
      US and European perspectives on interventional and observational research designs in post-marketing safety.
      ,

      Food & Drug Administration (FDA). Observational Research Exhibit 18. Available from: http://www.fda.gov/ohrms/dockets/dockets/05p0224/05p-0224-cp00001-Exhibit-18-Observational-Research-vol1.pdf. [Accessed May 2017].

      ]. While the registration of research is more common for clinical trials than for observational studies. Williams et al. argue that “much of the rationale for the prospective registration of clinical trials applies to the registration of observational studies” [
      • Williams R.J.
      • Tse T.
      • Harlan W.R.
      • Zarin D.A.
      Registration of observational studies: is it time?.
      ].
      These obligations include oversight by IRBs, informed consent, and public release of the study findings to advance biomedical knowledge. As with clinical trials, incomplete reporting of observational studies has been documented. Some researchers suggest that observational studies are also at increased risk for publication bias or other types of bias, including misrepresentation of prespecified analyses or disease classification coding. Such biases are a concern because they undermine the validity of observational studies, which are an important component of the medical evidence base in areas of public health, such as detection of rare adverse events.
      Therefore:
      • 26.
        Members are encouraged to register clinical and observational studies prospectively on ClinicalTrials.gov [

        National Institutes of Health. ClinicalTrials.gov. Available from: https://clinicaltrials.gov/. [Accessed November 20, 2017].

        ], patient registries (e.g., patientregistry.ahrq.gov) [

        American Healthcare Research & Quality. Registry of Patient Registries (RoPR). Available at: https://patientregistry.ahrq.gov/. [Accessed November 20, 2017].

        ], EU electronic Register of Post-Authorisation Studies (EUPAS Register) [

        European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP), The European Union electronic Register of Post-Authorisation Studies (EU PAS Register). Available from: http://encepp.eu/encepp/studiesDatabase.jsp. [Accessed May 17, 2017].

        ] or equivalent data-base in their own country. Where an HEOR study is being conducted alongside a clinical study gathering data prospectively (e.g., a clinical trial or observational study), joint registration of a clinical study and its accompanying economic analysis is recommended as an important element toward ensuring research transparency.
      Registering studies recognizes ethical obligations to patients and avoids the potential for publication bias in one’s own country as per local regulation and law.
      ISPOR has published more than 50 Good Practices for Outcomes Research Task Force Reports [

      International Society for Pharmacoeconomics and Outcomes Research (ISPOR). ISPOR Good Practices for Outcomes Research Index. Available from: http://www.ispor.org/workpaper/practices_index.asp.

      ] on conducting outcomes research (clinical, economic or patient-reported) or using outcomes research in health care decisions. While these reports do not address ethical principles directly, the specification of good research methods is an important component of recognizing and eliminating analytic bias.

      Modeling Studies

      In these HEOR studies, secondary data from multiple sources are synthesized using a decision-analytic model. Although this is the main application of modeling, models are sometimes used to extrapolate costs and benefits beyond the end of a clinical trial in a primary research study. The ethical principles discussed here apply equally to both situations.
      The general ethical principles of reasonableness and transparency suggest a number of approaches for the conduct of modeling studies. ISPOR, with the Society for Medical Decision Making (SMDM), published seven Modeling Good Research Practices Task Force Reports [
      • Caro J.J.
      • Briggs A.H.
      • Siebert U.
      • et al.
      Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1.
      ]. The seventh, on model transparency and validation [
      • Eddy D.M.
      • Hollingworth W.
      • Caro J.J.
      • et al.
      Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7.
      ], is the most relevant task force report to the ISPOR Code of Ethics.
      In conducting modeling studies, members should ensure that the input parameters are estimated based on a comprehensive review of the available literature. For the key parameters of the model (e.g., the estimate of relative treatment effect or source data), it may be necessary to conduct a full systematic review and meta-analysis.
      However, decision-analytic models typically rely on numerous parameter estimates, and it is often not possible to undertake a full systematic review for each.
      Therefore:
      • 27.
        Members should be transparent about the estimates they use for key parameters, provide the logic they used in selecting particular estimates, and explore the impact of their choices through sensitivity analysis.
      Sensitivity analysis is widely used in economic evaluation and explores the sensitivity of the study results to the variation in the input parameters.
      Another important issue of modeling studies is the need to make assumptions about the parameter estimates in situations where data are absent, or where there is uncertainty about the parameter estimates or model structure (structural uncertainty) [
      • Briggs A.H.
      • Weinstein M.C.
      • Fenwick E.
      • et al.
      Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6.
      ]. The ethical principles of reasonableness and transparency would dictate that any assumptions are clearly explained and justified. In addition, sensitivity analyses should be conducted to explore the importance (in terms of the overall estimate of cost-effectiveness) of the assumptions made.
      Reporting is discussed in Chapter 8: Publication and Dissemination.

      Chapter 5: Data Considerations

      This section provides guidance on data considerations in privacy, data protection, combining research data, data reliability and validity [
      ], transparency, and scientific integrity. Members should ensure selection of suitable data sources and adequate sample size to power the question(s) being studied.

      Privacy and Data Protection

      Protecting participants’ privacy is paramount to all forms of clinical research, including HEOR. Regulations such as the EU General Data Protection Regulation (GDPR) [

      General Data Protection Regulation (GDPR), European Commission. 2016. Available from: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679&qid=1462359521758&from=EN. [Accessed May 16, 2017]

      ], U.S. Health Insurance Portability and Accountability Act (HIPAA) [

      US Health & Human Services Health Insurance Portability and Accountability Act (HIPAA),US Department of Health & Human Services (1999, 2002). Available from: http://www.hhs.gov/hipaa/. [Accessed May 16, 2017].

      ], and Japan’s 2003 Act on the Protection of Personal Information (APPI) [

      The Act on the Protection of Personal Information, Act No. 57 of (2003). Available from: http://www.cas.go.jp/jp/seisaku/hourei/data/APPI.pdf. [Accessed May 16, 2017].

      ] cover the collection of data relating to an identifiable person.
      For data protection purposes, original holders of personal data can, if contractually bound, transfer personal data to other parties without seeking additional explicit permission of the data subject, as long as the data are being used for a purpose for which the original holder has a lawful basis to process the personal data, including the consent of the data subject.
      This would need to be an integral part of the informed consent process and would require IRB/IEC approval. Details of data processing, security, storing, transfer, and participants’ rights to their personal data are detailed in Appendix 8.

      Combining Research Data

      It is sometimes possible to enrich an existing database by linking additional information that is relevant to the individual patient or the provider. Examples include linking socioeconomic information about the neighborhood surrounding the patient’s home or the training history of the specific provider delivering a service. The most effective linkages take full advantage of the personally identifiable information (PII) of the patient or the provider. Adding data to an existing database can lead to the subtle erosion of privacy protections. As a result, some database providers insist on limiting potential links. It is critical to protect the commitment to privacy during and after the linkage of additional data. Combining of research data must also have been approved by the IRB/IEC.

      Data Verification

      On occasion, access to these data may be requested by journal reviewers or other researchers wishing to verify the analyses used in the research. It is important that researchers, sponsors, and owners of data recognize that the credibility of the research is lessened if other parties cannot adequately verify it.
      This is particularly important if one of the objectives of the research is to inform health care decision makers, who, in turn, may have to justify the basis on which they made a particular decision. This suggests that the right to access, within the law, should be granted by researchers to anonymized, group-level data and that the contracts for undertaking the research should reflect this consideration.
      Therefore:
      • 28.
        When a database (from primary data collection and/or secondary data use) is analyzed, members should provide a description of approaches, tools, and technologies used to store the data and maintain patient privacy/confidentiality and de-identification.
      • 29.
        Personal data should be maintained securely, and adequate backup should be maintained. Data access should be limited to authorized individuals. Control systems should be put in place to ensure authenticity, integrity, and confidentiality of data records when transmitted electronically.
      • 30.
        Researchers should offer the right to access the anonymized, group-level data used in their research. If data access is restricted by proprietary or contractual considerations, those considerations should be disclosed. If journal reviewers deem it important that statistical review of proprietary data be conducted, authors should work with both the data owners and the reviewers to find appropriate confidential arrangements for such review, whenever feasible.

      Transparency of Research and Data

      Transparency of data and replicability of results are important issues that pose challenges for authors, reviewers and journals [

      World Health Organization. WHO statement on public disclosure of clinical trial results (April 9, 2015). Available from: http://www.who.int/ictrp/results/reporting/en/. [Accessed September 26, 2017].

      ]. Some journals have explicit data policies; ISPOR’s journals Value in Health and Value in Health Regional Issues have their own [

      International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Value in Health Policies. Available from: http://www.ispor.org/publications/value/policies.asp. [Accessed May 17, 2017].

      ], and ISPOR members—as well as all contributors—are expected to comply with this policy.
      Nevertheless, it is recognized that for many, if not most, reviewers, detailed review of data, programs, and results is not feasible in the context of performing a timely manuscript review. For those who are able to do so, such review is encouraged. Those who are not able to do so but have reason to believe that data review is required should inquire with the editor about the possibility of employing an independent statistical reviewer.
      Therefore:
      • 31.
        Members’ hypotheses and research designs should be defined a priori, reported transparently, defended relative to alternatives, and planned to recognize and minimize all types of bias.
      • 32.
        Members should fully disclose the identity of sponsors of their research.
      • 33.
        Members should strive to avoid bias and the appearance of bias in conducting research, such as in the choice of methods and data inputs, or in the selective reporting of results.
      • 34.
        Members should be aware of conflicts of interest and the appearance of conflicts of interest. As a point of reference, members should look to the rules on disclosure of potential conflicts of interest described by major peer-reviewed journals and their own institutions.
      • 35.
        Members should maintain their professional autonomy and objectivity in conducting research and in writing or verbally reporting research findings.
      • 36.
        Methods sections of papers should identify and justify all departures from the a priori analysis plan.
      For authors, posting of data and programs is good practice and strongly encouraged, whenever feasible. At a minimum, best efforts should be made to make them available to reviewers upon request, with confidential arrangements. Similarly, transparency of data and replicability of research results should be serious considerations for those organizing conferences, discussing papers, serving on awards or selection committees, writing promotion or tenure letters, hiring researchers, and so on. This is particularly true for those individuals who are are influential to the manuscript’s approach or conclusions.

      Scientific Integrity

      Scientific misconduct is the violation of standard codes of scholarly conduct and ethical behavior in professional scientific research. According to the International Committee of Medical Journal Editors (ICMJE), it includes, but is not necessarily limited to, data fabrication, data falsification, including deceptive manipulation of images, and plagiarism [

      International Committee of Medical Journal Editors (ICMJE), Scientific Misconduct, Expressions of Concern, and Retraction. Available from: http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/scientific-misconduct-expressions-of-concern-and-retraction.html. [Accessed May 17, 2017].

      ]. (See also Chapter 8: Publication and Dissemination.)
      The Committee on Publications Ethics (COPE) has developed procedures for editors to follow if there are concerns about the integrity or conduct of work in submitted or published papers or if scientific misconduct is suspected. The procedure emphasizes transparency and accountability throughout the investigation, as well as communication of the whole process [

      Committee on Publication Ethics (COPE) Council. Ethical guidelines for peer reviewers. September 2017. Available from: www.publicationethics.org.

      ].
      While some may consider failure to publish or submit clinical trial results or other human studies a form of scientific misconduct, each situation of alleged misconduct requires individual assessment by relevant stakeholders.
      Therefore:
      • 37.
        Members should maintain and protect the integrity of data used in their studies, and any other aspect of their research (e.g., respect for patient autonomy, such as informed consent and data privacy).
      • 38.
        Members should not draw conclusions beyond what their data would support and should discuss any limitations in a transparent manner.
      For more information on ethics and data considerations, please see Chapter 8: Publication and Dissemination.

      Chapter 6: Relationship of Sponsor with Researcher and Others

      HEOR sponsors range from life sciences industry and health care insurers to provider and patient associations and governmental bodies. It is understood that much of the funding available to those who pursue HEOR is provided by bodies with vested interests. A central principle of ISPOR’s work is the maintenance of its own objectivity and autonomy from sponsors and commercial interests. While conducting joint research, participants should be respectful of each other’s scientific views and methods.

      Researchers

      Those who conduct HEOR should strive to make the nature, scope, and potential of their work clear to sponsors. This not only includes being transparent about the kind of knowledge scientific research can generate, but also pertains to the ethical dimensions of conducting research. Therefore, researchers should make it clear to sponsors that all outputs from a research project will include the acknowledgement of all sources of funding as part of a conflict of interest declaration.
      Furthermore, researchers should not only avoid being placed in a position where they experience a conflict of interest, but they should also avoid the appearance of a conflict of interest. Even the appearance of a conflict of interest can raise the possibility that their research will be perceived as biased.
      When engaging with sponsors, researchers should be clear about the need to maintain their professional autonomy over all stages of the research, including its design, conduct, and publication. The autonomy of the researcher contributes to the objectivity and value of the research and the validity of the results.
      Sponsors should be informed about the opportunities to enter studies or provide input/measures into research registries. They should operate as partners with no commercial interests and understand their rights (or lack thereof) with regard to access and ownership of data generated or collected as part of the research.
      When researchers accept sponsorship for a particular project, they should be in a position to conduct the research in a manner that is both timely and reflective of the required level of scientific quality and methodologic rigor.
      HEOR is conducted through close collaboration within teams and between teams, nationally and internationally. The principle of civility is important to ensure that the contribution from all parties is respected and understood. Employees and employers have a responsibility to ensure that the reputations, rights, and interests of all parties are respected and that appropriate standards protecting proprietary information are adhered to. Furthermore, particular care should be taken to ensure that the relationship between students and more senior faculty is appropriately conducted.
      Therefore:
      • 39.
        Members should respect the reputations and rights of colleagues when engaged in collaborative projects.
      • 40.
        Members should collaborate with team members, provide competent work, and protect proprietary information.
      • 41.
        Members should treat their research employees and all non-research subordinates with respect and should compensate them fairly for their work.
      • 42.
        Members should treat students with respect and refrain from taking advantage of them under any circumstances.

      Responsibility to Sponsors

      HEOR must not be used to obtain confidential information about competing products and companies from participants who are bound by confidentiality agreements with those companies.
      If the contract between sponsor and researcher allows for subcontracting, a researcher may transfer any or all of his or her duties and functions to one or more subcontractors (e.g., a clinical research organization (CRO) employee). All parties, including subcontractors, should be contractually bound by the same legal and ethical requirements.
      Therefore:
      • 43.
        At all times, members acting as sponsors should allow researchers to maintain their scientific integrity and adhere to relevant standards in conducting and reporting research.
      • 44.
        Members should respect contractual rights when they agree to perform work for hire and should refrain from disseminating information that might be proprietary.

      Key Opinion Leaders (KOLs) and Advisory Board Members

      The role of KOLs brings some ISPOR members into close contact with sponsors. As such, we reiterate the centrality of independence, professional autonomy, and objectivity to the scientific process, including dissemination of scientific findings.
      Therefore:
      • 45.
        When acting as KOLs or advisory board members, ISPOR members should be transparent about payments and any other benefits that they receive for acting in this capacity.
      • 46.
        When acting as KOLs or advisory board members, ISPOR members should ensure that the information they are presenting is an accurate representation of the facts available. They should respond to questions and queries honestly and to the best of their abilities.
      • 47.
        When relying on KOLs or advisory board members, ISPOR members should critically triangulate the claims being made. Where appropriate, they should seek independent corroboration of any factual claims and consider the full range of alternatives for themselves.
      • 48.
        When acting as KOLs or advisory board members, ISPOR members should maintain their independence and professional autonomy and act transparently (e.g., declare conflicts of interest).

      IRB/IEC Approval

      Sponsors should ensure that IRB/IEC approval is obtained, as appropriate, for the planned research. It is the responsibility of an IRB/IEC to ensure that the rights, safety, and well-being of those involved in research are protected. Furthermore, it should provide public assurance of that protection by, among other things, reviewing, approving, and providing a favorable opinion on the research proposal and the suitability of the investigator, facilities, and the methods and material to be used in obtaining and documenting informed consent of research subjects.
      Requirements of the IRB/IEC approval depend on the research type, study objectives, interaction with patients, and competent authority requirement from different countries. Some recommendations of IRB/IEC for different studies appear in Appendix 9. The legal status, composition, function, operations, and regulatory requirements pertaining to independent ethics committees may differ among countries.
      Appendix 10 provides considerations for different participants groups, including healthy volunteers, patients, protected classes, children, and vulnerable populations.

      Responsibility to Research Participants

      Researchers should be open and transparent about the aim and objectives of their research, its design, its conduct, and its potential consequences or outcomes. They should be clear with participants about what is being asked of them, the right to refuse to participate, and the possibilities of withdrawing at a later date.
      While it might not always be possible, realistic, or particularly desirable, researchers should, where appropriate, communicate research results to participants. Responsibilities to communicate aggregated results to participants should be clearly stated in consent materials or processes.
      Informed consent is the tool to ensure that trial participants understand the context and specifics of clinical trials and/or health care–related research. The informed consent document should be relevant, easily understandable, and practical. It should not serve as a theoretical exercise for the researcher. A copy of the signed informed consent must be provided to each participant.
      Ethical review of research proposals should, where appropriate, seek input from individuals or organizations that are able to represent the perspective of patients.
      Therefore:
      • 49.
        Members should respect the autonomy of research participants when designing and conducting studies, specifically including, but not limited to, informed consent and data privacy.
      • 50.
        Members should, where appropriate, seek input from individuals or organizations that are able to represent the perspective of patients.
      • 51.
        Members should, where appropriate, communicate their research findings to participants.

      Chapter 7: Patient Centricity and Patient Engagement in Research

      The ISPOR Code has been updated to reflect an increased focus on patient centricity and patient engagement in research by regulatory and other stakeholders seeking to understand and incorporate patients’ perspectives and experiences. Organizations embracing patient centricity in their research improve the quality of outcomes, utility, and efficiency of clinical trials. Moreover, it addresses ethical concerns and societal and moral obligations. Furthermore, the involvement of patients and/or their representatives at all stages of research increases transparency, mutual respect, and trust among patients, researchers, and other stakeholders, including payers and providers, as well as the likelihood of improved outcomes [
      Patient Centered Outcomes Research Institute (PCORI)
      ].
      Reflecting this evolution in the research environment, as an organization, ISPOR has moved to become more patient centered. This aligns with ISPOR members’ interests, as well as ISPOR’s overall mission to promote HEOR excellence to improve decision making for health globally.
      In 2017, the ISPOR Board of Directors (Board) voted to create an ISPOR Patient Council, an advisory body to the Board, that will determine how best to engage patient representatives in research and decision-making processes. Because this is a new area for most ISPOR members, this section is covered more broadly for educational purposes.

      Understanding Patient Centricity and Patient Engagement

      As of 2017, there is no consistent definition of patient centricity or patient engagement. Significant variation exists in how different stakeholders and sectors (e.g., regulators, HTA agencies, the pharmaceutical and medical device industry, academia, hospitals, and patient organizations) define these terms.
      An ISPOR Special Interest Group has undertaken a manuscript project researching these terms and definitions.
      Patient-centric research should focus on the outcomes that are meaningful to patients. Those outcomes should be important to patients’ survival, function, or feelings as identified or affirmed by the patients themselves, or judged to be in patients’ best interests by providers and caregivers when patients cannot report for themselves [
      • Smith S.K.
      • Selig W.
      • Harker M.
      • et al.
      Patient engagement practices in clinical research among patient groups, industry, and academia in the United States: a survey.
      ,
      Patient Centered Outcomes Research Institute (PCORI)
      ,
      • Walton M.K.
      • Powers J.A.
      • Hobart J.
      • et al.
      Clinical outcome assessments: a conceptual foundation—Report of the ISPOR Clinical Outcomes Assessment Emerging Good Practices Task Force.
      ]. Patient-centered outcomes may or may not be measured by patient self-report, i.e., patient-reported [
      • Benjamin K.
      • Vernon M.K.
      • Patrick D.L.
      • et al.
      Patient reported outcome and observer reported outcome assessment in rare disease clinical trials—emerging good practices: an ISPOR COA Emerging Good Practices report.
      ].
      To understand what is important to patients, they must be meaningfully engaged in the research from start to finish. Patient input is valuable throughout the medical product lifecycle from early development to dissemination and postmarketing surveillance. The involvement of knowledgeable patient representatives to include the patient perspective is especially critical in early phases to determine unmet needs, set research questions, and choice of the correct study endpoint(s) for medical label claims [
      • Perfetto E.M.
      • Burke L.B.
      Criteria for successful patient engagement: an essential part of patient-focused drug development.
      ].
      For some illnesses, there is a significant impact on family life and family caregivers. It is important to include family and caregiver engagement under these circumstances. Their perspective and that of patient representatives and advocacy organizations is especially useful to strengthen trial design and utility [
      • Benjamin K.
      • Vernon M.K.
      • Patrick D.L.
      • et al.
      Patient reported outcome and observer reported outcome assessment in rare disease clinical trials—emerging good practices: an ISPOR COA Emerging Good Practices report.
      ,
      • Matza L.S.
      • Patrick D.
      • Riley A.W.
      • et al.
      Pediatric patient-reported outcome instruments for research to support medical product labeling: Report of the ISPOR PRO good research practices for the assessment of children and adolescents task force.
      ].

      Levels and Timing of Patient Engagement

      There are a number of useful frameworks for patient engagement [
      • Carman K.L.
      • Dardess P.
      • Maurer M.
      • et al.
      Patient and family engagement: a framework for understanding the elements and developing interventions and policies.
      ,
      • Domecq J.P.
      • Prutsky G.
      • Elraiyah T.
      • et al.
      Patient engagement in research: a systematic review.
      ,
      • Guise J.M.
      • O’Haire C.
      • McPheeters M.
      • et al.
      A practice-based tool for engaging stakeholders in future research: a synthesis of current practices.
      ,
      • Perfetto E.M.
      • Oehrlein E.M.
      M-CERSI Conference on Patient-Focused Drug Development Planning Committee
      Assessing meaningful patient engagement in drug development: a definition, framework, and rubric.
      ]. They describe (1) patient involvement through interchange between the patient and provider; (2) the stages of research in which patients can be involved; and (3) prioritizing stakeholder engagement in research. They serve as a conceptual basis for patient engagement in medical product development.
      Early and meaningful engagement of members of patient organizations in research is highly recommended. Collaboration with patient organizations as part of the research team is also encouraged. Use of a patient advisory board can provide patient input at the study design stage, can improve site selection and recruitment (e.g., within indigenous or other historically disadvantaged populations), improve data collection, and reduce patient burden.
      Patients (or patient organizations) should actively contribute to draft protocols and documents that are for patients to ensure that the content and format are understood. To accurately capture patients’ values and preferences, patients should be involved in benefit/risk evaluation [

      Medical Device Innovation Consortium (MDIC) Patient-Centered Benefit-Risk Assessment (PCBR). Available from: http://mdic.org/wp-content/uploads/2014/06/Patient-Centered-Benefit-Risk-PCBR-Project-update.pdf.

      ] and related activities throughout the development lifecycle [
      Patient Centered Outcomes Research Institute (PCORI)
      ,
      • Perfetto E.M.
      • Burke L.B.
      Criteria for successful patient engagement: an essential part of patient-focused drug development.
      ,
      • Perfetto E.M.
      • Oehrlein E.M.
      M-CERSI Conference on Patient-Focused Drug Development Planning Committee
      Assessing meaningful patient engagement in drug development: a definition, framework, and rubric.
      ]. A planned sequential approach is recommended where feedback from patients is collected and considered [
      • Patrick D.L.
      What does “patient-centered outcomes” mean?.
      ].
      Patients and patient organizations can also help in the translation of research results by helping to develop and share layperson-level summaries of clinical trial results. Finally, patient input is also needed in assessing real-world effectiveness, cost-effectiveness, and value. These assessments should be enriched with patient input and guided by patient experiences [
      • Smith S.K.
      • Selig W.
      • Harker M.
      • et al.
      Patient engagement practices in clinical research among patient groups, industry, and academia in the United States: a survey.
      ,
      National Health Council (NHC)
      The patient voice in value: the National Health Council patient-centered value model rubric.
      ].

      Partnering with Patient Organizations

      Collaboration with patient advocacy organizations can be a sound platform for successful patient engagement. Researchers will need to familiarize themselves with the many types of organizations that vary in size and scope (e.g., rare versus high-prevalence diseases; local, regional, and international). They have a range of experiences, organizational cultures, governance structures, priorities, and ability and capacity to engage.

      Ethical Considerations

      Ethical issues often arise in the patient engagement process. To prevent or address ethical issues arising in the patient engagement process, it is important to follow published guidelines. Rare Diseases Europe (EURORDIS) has published a Charter for Collaboration between Sponsors and Patient Organizations for Clinical Trials in Rare Diseases [

      EURORDIS Rare Disease Europe. The EURORDIS Charter in practice. Available from: http://www.eurordis.org/IMG/pdf/Charter_Clinical_Trials-Final.pdf. [Accessed May 17, 2017].

      ]. The European Patients’ Academy (EUPATI) has published guidance for patient involvement in research and development [

      European Patients Academy (EUPATI). Guidance for Patient Involvement in R&D. Available from: https://www.eupati.eu/guidance-patient-involvement/.

      ] and HTA [

      European Patients Academy (EUPATI) Guidance for patient involvement in HTA. Available from: https://www.eupati.eu/health-technology-assessment/guidance-for-patient-involvement-in-hta/. [Accessed September 22, 2017].

      ]. The Patient Centered Outcomes Research Institute (PCORI) and the United States-based National Health Council (NHC) [
      National Health Council (NHC)
      The patient voice in value: the National Health Council patient-centered value model rubric.
      ], among others [
      ], have also published guidance for stakeholders. Consultation with experienced stakeholders is very useful, too.
      When working with a patient organization, a contract between patients and research partners is recommended. The contract should be respectful and clearly outline roles and deliverables. The contract should recognize patients as experts in their health condition and compensate them appropriately. Further information on written agreements and compensation are available from PCORI [

      Patient Centered Outcomes Research Institute (PCORI). Financial compensation of patients, caregivers, and patient/caregiver organizations engaged in PCORI-funded research as engaged research partners. Available from: http://www.pcori.org/sites/default/files/PCORI-Compensation-Framework-for-Engaged-Research-Partners.pdf. [Accessed May 17, 2017].

      ], EUPATI [
      European Patients Academy (EUPATI)
      ], and the EFPIA [
      European Federation of Pharmaceutical Industries and Associations (EFPIA)
      ].
      Researchers should recognize that a large majority of patients are not trained as researchers. Researchers should incorporate patients’ input in framing research questions and selecting correct methods for study conduct while recognizing that the research methodologic or analytic approach will be driven by other considerations as well. However, patient input should be included throughout the research lifecycle.
      Therefore:
      • 52.
        Patient input should be included throughout the medical product lifecycle from early development to dissemination and postmarketing surveillance.
      • 53.
        Researchers should endeavor to involve patients and their representatives as partners before, during, and after conducting research.
      • 54.
        To prevent or address ethical issues arising in the patient engagement process, follow published guidelines. Consultation with experienced stakeholders is useful as well.

      Chapter 8: Publication and Dissemination

      The main purpose of publishing or otherwise disseminating HEOR is to provide reliable and relevant information related to health care treatments and programs. Therefore, it is important that members submitting manuscripts ensure that they do not contain inaccuracies or misrepresent data.
      Publications can discuss methodologic principles, the results of empirical studies, or policy choices. The main users of HEOR include decision makers concerned with population-based choices, health professionals deciding on treatment options, and patients wishing to understand more about the treatments available.

      Scientific Misconduct: Plagiarism

      Plagiarism—the act of passing off another author’s work as one’s own, verbatim or paraphrased—is perhaps the most fundamental ethics violation by an author in any field of endeavor. Copyright laws protect writers’ words as their legal property. Furthermore, it is extremely important to give comprehensive citations to avoid unintentional plagiarism. (See Chapter 5 for more on scientific misconduct.)
      In the health and medical sciences, including HEOR, there is a gray area as to what constitutes plagiarism in the context of an individual author publishing new work that is similar in many respects to prior work on which he or she was one of several authors. On occasion, an author is invited to submit a special article or book chapter due to prior participation in an area of important research with the expectation that his or her contribution will derive from the prior work. In these instances, it is important that the author double-check to make sure that no written material (or table or figure) is replicated from the earlier work without permission from the copyright holder.
      ISPOR initiated a scientific and health policy group publication rule: “No member of an ISPOR Task Force or Special Interest Group should publish any material from an upcoming report, public presentation, or project deliverable without first consulting the larger group for permission prior to submission and publication.
      Therefore:
      • 55.
        Members should not engage in any act of plagiarism, including self-plagiarism. If publishing work similar to anything jointly authored with others, members should ensure that no replication of the prior work was unintentionally done.
      • 56.
        It is extremely important to give comprehensive citations to avoid unintentional plagiarism.
      • 57.
        Members should not publish any material relating to the activities of an ISPOR Task Force, Special Interest Group, or other ISPOR group at any stage, without first consulting fellow group members/co-authors for permission.

      Bias

      A key concern in publication and dissemination is the possibility of bias, either publication bias, whereby studies with negative or inconclusive results tend not to be published, or analytic bias, whereby analysts make inappropriate methodologic choices that favor one treatment option over another. Bias is a particularly pertinent concern in the field of HEOR, where a high proportion of studies are industry sponsored and where the analyst often has considerable discretion in the choice of methods and assumptions.
      ISPOR has published more than 50 Good Practices for Outcomes Research Task Force Reports on conducting outcomes research (clinical, economic, or patient-reported) or using outcomes research in health care decisions. While these reports do not address ethical principles directly, the specification of good research methods is an important component of recognizing and eliminating analytic bias.
      The main method of disseminating HEOR is through peer-reviewed journals. Therefore, a major source of ethical principles and good publishing practice is the ICMJE recommendations that have the endorsement and support of all the major clinical and health services research journals [
      • Smith S.K.
      • Selig W.
      • Harker M.
      • et al.
      Patient engagement practices in clinical research among patient groups, industry, and academia in the United States: a survey.
      ]. The recommendations for ISPOR members in this chapter are consistent with those of the ICMJE, but offer more detail relevant to this particular field of research.

      Freedom to Publish the Findings without Restrictions

      Both peer-reviewed journals and the users of HEOR take an interest in the nature of the relationship between the researcher and the sponsor. This is one indicator of the likelihood of any bias in the research. This relationship is usually expressed through a contract between the researcher and sponsor. In negotiating the contract, researchers should pay particular attention to the need for transparency throughout the research process and the freedom to publish the findings without restrictions [
      • Smith S.K.
      • Selig W.
      • Harker M.
      • et al.
      Patient engagement practices in clinical research among patient groups, industry, and academia in the United States: a survey.
      ].
      Members should seek to establish, in advance, a clear agreement with the sponsor on whether the results of a given piece of work can be published or presented. This could include statements on whether the sponsor has a right to review or approve any manuscript prior to publication and which party has the intellectual property rights for the research outputs.
      It is important to specify publication rights, one way or another, in the contract. University contracts usually do specify publication rights. Generally, a university will not sign off on a contract where a sponsor can stop publication of research findings. However, sponsor prior review and comment are generally accepted by universities.
      Individual researchers or vendors may be willing to “work for hire.” This does not guarantee publication rights. In such a case, if anything is published, it should be disclosed that publication rights were not guaranteed in advance. Preventing publication would not be acceptable in any case that includes safety issue revelations—failure to disclose could result in a public health hazard.
      Therefore:
      • 58.
        In the case of sponsored research, members should agree to a contract that clearly sets out their rights, scope of work, and rights of the sponsor (e.g., intellectual property rights and rights to publish) in the conduct and reporting of the study. A summary of this agreement should be provided in the published paper.

      Transparency in Reporting

      Transparency in reporting is also essential to reduce the possibility of bias in research. Several reporting guidelines exist, including those developed by Consolidated Standards of Reporting Trials (CONSORT) for clinical research (including quality of life measurement) [
      • Schulz K.F.
      • Altman D.G.
      • Moher D.
      for the CONSORT Group
      CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomized trials.
      ], Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) [
      • Moher D.
      • Liberati A.
      • Tetzlaff J.
      • Altman D.G.
      Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
      ], ISPOR’s Consolidated Health Economic Evaluation Reporting Standards (CHEERS), [
      • Husereau D.
      • Drummond M.
      • Petrou S.
      • et al.
      Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—explanation and elaboration: A report of the ISPOR Health Economic Evaluations Publication Guidelines Good Reporting Practices Task Force.
      ,
      • Husereau D.
      • Drummond M.
      • Petrou S.
      • et al.
      on behalf of the CHEERS Task Force
      Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.
      ] developed and published with other leading journals, and STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) for observational studies [

      Strengthening the Reporting of OBservational studies in Epidemiology (STROBE) Statement. Available from: http://www.strobe-statement.org/index.php?id=strobe-home. [Accessed May 17, 2017].

      ]. High-quality reporting also aids the peer-review process, although journal editors and reviewers may also ask for access to the original data, the statistical analyses performed, or the models used in the research.
      Authors of publications should endeavor to respond as fully as possible to requests for additional information on their data or methods. Offering full access to data, analyses, and models represents a level of transparency that can enhance the credibility of the research. However, access to some data may be restricted by contractual obligations, proprietary reasons, IRB/IEC restrictions, or the general need to protect the privacy of participants in the research. In addition, allowing access to executable electronic copies of models has raised specific concerns on the part of researchers who fear that their intellectual capital could be undermined if the model were copied [
      • Eddy D.M.
      • Hollingworth W.
      • Caro J.J.
      • et al.
      Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7.
      ].
      Nevertheless, peer reviewers and journal editors may feel that access is required to adequately verify the quality of the research. Researchers should remember that peer reviewers are bound by confidentiality agreements. Furthermore, some journals have strengthened these to reassure authors that the intellectual capital in their work will be protected.
      Therefore:
      • 59.
        Transparency in reporting is also essential to reduce the possibility of bias in research. Follow established reporting guidelines and endeavor to respond as fully as possible to requests for additional information on data or methods.
      Joint registration of a clinical study and its accompanying economic analysis is recommended as an important element toward ensuring research transparency.
      Therefore:
      • 60.
        Where an HEOR study is being conducted alongside a clinical study gathering data prospectively (e.g., a clinical trial or observational study), members should report whether the clinical study has been registered and, if so, where.
      Where research is disseminated in non–peer-reviewed journals or through electronic media, such as websites or social media, the scrutiny of peer review does not generally exist (although comments sections on web posts might be considered an informal peer review). The way in which researchers should approach this depends on whether they are purporting to report fact or opinion—unless it is clear that mere opinions are being expressed, authors should be willing to offer the same level of access to underlying data and/or analyses as they would to journal peer reviewers.
      Therefore:
      • 61.
        Members should endeavor to publicly disseminate their work and to publish it in peer-reviewed journals, when possible.
      • 62.
        Members should work, where appropriate, to encourage the establishment and/or maintenance of an appropriate peer review process that examines the quality of the methodologic rigor independently of the organization for which the individual works.
      • 63.
        Members serving as peer reviewers for journals should respect the confidentiality of the material under review and understand that their access to it is solely for the purposes of performing the review.
      • 64.
        The description of study methods (design, study setting, data sources and input values, sampling, and analyses) should be complete and transparent enough for a suitably trained researcher to replicate the study.
      • 65.
        Methods sections of papers should give thorough, transparent attention to all measures taken to minimize bias.
      • 66.
        Where allowable by law and IRB/IEC approval, members should respond favorably to requests from journal editors and reviewers for access to original data and electronic copies of models where this access is required to ensure a rigorous peer review process and where commercial-in-confidence arrangements can be maintained.
      • 67.
        In those instances, in which study methods include analysis of a database (retrospective or prospective), members should describe approaches, methods, technologies used to ensure data completeness, and validity, as well as the software package(s) used for data analysis. Members should have the education, training, and experience to perform the assigned tasks.
      • 68.
        In those instances, in which sharing of model(s) and/or data source(s) is not feasible, members should be encouraged to provide supporting material demonstrating model and/or data validity, such as range and logic checks and assessment of data completeness.
      • 69.
        If submitting to a journal or publication that does not have peer review, or disseminating a report via electronic media, members should avoid the inclusion of material that cannot be supported by basic article references or make it clear that the article represents the author’s own opinion. If research is being reported, then access to the underlying data and/or analyses should be offered in the same manner as would be done under a peer-review process.

      Study Authorship

      The named authors formally take responsibility for the report of the research. Therefore, some study users view the identity of the authors as one indicator of the likely quality and reliability of the research, although when acting as editors or reviewers of papers for journals, ISPOR members should make judgments based solely on the quality of the research, not the identity or affiliations of the authors (if these are not already anonymized by the journal concerned).
      Authorship also provides recognition of the researchers’ contribution. Therefore, it is wrong to include an author who did not make a substantive contribution due to their name recognition and perceived status. Similarly, it is wrong to exclude an individual who had made a substantial contribution because of their affiliation. Criteria include:
      • 1.
        Substantial contributions to the conception or design of the work; or the acquisition, analysis or interpretation of data for the work; AND
      • 2.
        Drafting of the work or revising it critically for important intellectual content; AND
      • 3.
        Final approval of the version to be published; AND
      • 4.
        Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
      The ICMJE and many peer-reviewed journals require the corresponding author to confirm that these conditions have been met. Specific journals, such as JAMA, have guidelines, and these are very useful generally.
      Journals now generally require individual authors of a manuscript to certify by signature that they have contributed sufficiently to be listed as an author. However, journals vary in their requirements for certification, so ISPOR, as an organization, encourages its members to adhere to fair and equitable requirements for authorship and to respect their colleagues in the process.
      Other individuals participating in the research, but not qual- ifying as authors, should be acknowledged.

      Financial Disclosure, Conflicts of Interest, and Past Work Relationships

      Another important condition of authorship is that individuals disclose any financial and/or other relationships that may be perceived to be conflicts of interest with respect to the work being reported. In the field of HEOR, it is particularly important to disclose any present or past relationships with the manufacturers of any products referred to in the research or any competitor products.
      In reporting past relationships, many researchers will have a large number of such relationships stretching back over a number of years. A common time frame is to report any relationships within the past 3 years, but different journals have different guidelines. The ICMJE specifies no limit.
      Therefore:
      • 70.
        It is important to disclose financial and/or other relationships that could be perceived as a conflict of interest, especially present or past relationships with manufacturers referred to in the research or any competitor products.

      Chapter 9: Conclusion and Limitations

      This Code cuts across virtually all areas of research and dissemination. It is meant to be a comprehensive guide for HEOR researchers. The Code is a means for the science of HEOR to avoid or address credibility challenges based on methodology or bias concerns and to ensure that health care research is designed, conducted, and reported to the highest ethical standard. By following the highest possible ethical practices, stakeholders, especially affected constituencies, will be more likely to participate in, as well as trust and benefit from, research.
      As part of membership, members agree to comply with ISPOR’s Code of Ethics when they join or renew membership annually. Nevertheless, ISPOR recognizes that its activities and those of its members affect a number of constituencies, and that there may be conflicting standards of professional conduct. Patients as stakeholders and patient engagement are two relatively new concepts impacting health care research, especially in Europe and North America.
      Furthermore, ISPOR recognizes that within the fast-changing climates of different health care systems, it is difficult to address all ethical issues HEOR practitioners face. New data sources might emerge. Genomic sequencing and the internet might make privacy almost impossible to protect, and open data [

      Open Data Institute, Available from: https://theodi.org/what-is-open-data. [Accessed September 22, 2017].

      ] antitrust/competition legislations might pose new challenges to intellectual property rights. Although the impact of this much-needed social movement is slowly starting to become clearer, its relevance and impact on ISPOR members, especially researchers, requires further elucidation and guidance. ISPOR will strive to keep its Code as current as possible.
      Therefore:
      • 71.
        ISPOR expects its members to adhere to the highest ethical standards. By following these practices, stakeholders will be more likely to participate in health care research, as well as trust and benefit from it.
      • 72.
        HEOR is improved by following the highest possible ethical practices in terms of improved research, decision making, and, overall health and health care.

      ISPOR Code of Ethics 2017 (4th Edition)SUMMARY POINTS

      Chapter 1: Introduction to the ISPOR Code of Ethics

      • 1.
        The ISPOR Code of Ethics was developed as guidance for the health economics and outcomes research community as a whole.
      • 2.
        ISPOR strives for representativeness, transparency, and objectivity in its activities.
      • 3.
        ISPOR has a Board of Directors that is representative of the various constituencies that the Society serves.
      • 4.
        The ISPOR program planning and selection committees should have membership representative of its major constituencies.
      • 5.
        ISPOR strives for a balance in sponsorship of its activities by providing decision criteria for acceptance and disbursement of funds to ensure full transparency and minimize the possibility of bias or conflict of interest.
      • 6.
        Like other professional societies, ISPOR is conscious of broader ethical issues that impact global and regional medical resource allocation, public health policies, and the global health care environment or are relevant to research topics such as patient autonomy, patient outcomes, and research conduct. These issues include, but are not limited to, prejudice, equity in health care delivery, and access.

      Chapter 2: Application of Ethical Principles to the ISPOR Code of Ethics

      • 7.
        Members should maintain current knowledge of research practices, of general principles, and of local and regional relevant practices.
      • 8.
        Members must conduct activities honestly, with integrity and good judgment, and in the best interests of the patients, health care professionals, decision makers, outcomes researchers, pharmaceutical manufacturers, and other public health communities we serve.
      • 9.
        Privacy and confidentiality: It is essential that protected health information (PHI) and other personal data of patients are handled with the utmost care so that patient confidentiality is maintained at all times and that no breaches to patient privacy occur.
      • 10.
        Transparency and Integrity: Members must disclose research methods in sufficient detail to permit replication. Funding sources should be clearly acknowledged, and any conflicts of interest declared [11,12].
      • 11.
        Designing, conducting, and especially reporting of the study should be an objective (unbiased) reflection of the full range of findings generated.
      • 12.
        Civility: Members’ research and discussion should respect the dignity of all participants. Respecting the dignity of patients and providers of care is clearly a top responsibility. It is also a responsibility to treat fellow researchers with respect.

      Chapter 3: Scope of the ISPOR Code of Ethics

      • 13.
        Members should adhere to the standards of practice for their respective fields of research; hence they should identify and acknowledge any applicable official guidelines and standards used.

      Chapter 4: Research Design Considerations

      • 14.
        From the point of “first contact,” researchers should provide potential subjects information about study intentions and how the research is funded, as well as all information mandated in their proposals to institutional review boards (IRBs) / independent ethics committees (IECs)‡ according to the primary objective with patient consent.
      • 15.
        Members should describe the analytic study population in terms of its relevant demographic and medical characteristics, geography, time period, and selection criteria.
      • 16.
        Members should justify the chosen target population and choose a suitable research setting for this population. In addition, existing data or literature should be included to provide information about the specific population to which the study results will be applied.
      • 17.
        The number of patients and sites selected for a study should be appropriate to meet the research objectives without being greater than statistically necessary.
      • 18.
        The balance of risk or harm to benefit for patients must be considered in HEOR studies and must be communicated to patients via informed consent.
      • 19.
        Safety and adverse event reporting (AER) is an important aspect of all primary research involving patients and medical interventions. It is applicable to many HEOR activities and must follow the most up-to-date international and local guidelines.
      • 20.
        Any incentive, honorarium or payment is subject to receivers’ and providers’ internal compliance guidelines. IRB/IEC approval and must be detailed in the proposal submitted for review.
      • 21.
        Researchers should be diligent in ensuring that the incentive would not induce research participants to accept risks they would not be willing to accept if they were offered a smaller or no incentive.
      • 22.
        When using secondary data sources initially collected for another purpose, HEOR researchers should ensure that intellectual property rights are respected and referenced and that all the appropriate permissions have been secured.
      • 23.
        Given the potential for bias in the analysis of secondary data, the most important general ethical principles are those of reasonable- ness and transparency.
      • 24.
        Any known or potential source of bias in the data that can affect the results must be disclosed whenever secondary data are used.
      • 25.
        In those instances in which study methods include analysis of a database, members should describe approaches, methods, technologies used to ensure data completeness and validity, as well as the software package(s) used for data analysis. Members should have the education, training, and experience to perform the assigned tasks or provide evidence of collaboration with individuals who are qualified.
      • 26.
        Members are encouraged to register clinical and observational studies prospectively on ClinicalTrials.gov [25], patient registries (e.g., patientregistry.ahrq.gov) [26], EU electronic Register of Post-Authorisation Studies (EUPAS Register) [27] or equivalent data-base in their own country. Where an HEOR study is being conducted alongside a clinical study gathering data prospectively (e.g., a clinical trial or observational study), joint registration of a clinical study and its accompanying economic analysis is recommended as an important element toward ensuring research transparency.
      • 27.
        Members should be transparent about the estimates they use for key parameters, provide the logic they used in selecting particular estimates, and explore the impact of their choices through sensitivity analysis.

      Chapter 5: Data Considerations

      • 28.
        When a database (from primary data collection and/or secondary data use) is analyzed, members should provide a description of approaches, tools, and technologies used to store the data and maintain patient privacy/confidentiality and de-identification.
      • 29.
        Personal data should be maintained securely, and adequate backup should be maintained. Data access should be limited to authorized individuals. Control systems should be put in place to ensure authenticity, integrity, and confidentiality of data records when transmitted electronically.
      • 30.
        Researchers should offer the right to access the anonymized, group-level data used in their research. If data access is restricted by proprietary or contractual considerations, those considerations should be disclosed. If journal reviewers deem it important that statistical review of proprietary data be conducted, authors should work with both the data owners and the reviewers to find appropriate confidential arrangements for such review, whenever feasible.
      • 31.
        Members’ hypotheses and research designs should be defined a priori, reported transparently, defended relative to alternatives, and planned to recognize and minimize all types of bias.
      • 32.
        Members should fully disclose the identity of sponsors of their research.
      • 33.
        Members should strive to avoid bias and the appearance of bias in conducting research, such as in the choice of methods and data inputs, or in the selective reporting of results.
      • 34.
        Members should be aware of conflicts of interest and the appearance of conflicts of interest. As a point of reference, members should look to the rules on disclosure of potential conflicts of interest described by major peer-reviewed journals and their own institutions.
      • 35.
        Members should maintain their professional autonomy and objectivity in conducting research and in writing or verbally reporting research findings.
      • 36.
        Methods sections of papers should identify and justify all departures from the a priori analysis plan.
      • 37.
        Members should maintain and protect the integrity of data used in their studies, and any other aspect of their research (e.g., respect for patient autonomy, such as informed consent and data privacy).
      • 38.
        Members should not draw conclusions beyond what their data would support and should discuss any limitations in a transparent manner.

      Chapter 6: Relationship of Sponsor with Researcher and Others

      • 39.
        Members should respect the reputations and rights of colleagues when engaged in collaborative projects.
      • 40.
        Members should collaborate with team members, provide competent work, and protect proprietary information.
      • 41.
        Members should treat their research employees and all non-research subordinates with respect and should compensate them fairly for their work.
      • 42.
        Members should treat students with respect and refrain from taking advantage of them under any circumstances.
      • 43.
        At all times, members acting as sponsors should allow researchers to maintain their scientific integrity and adhere to relevant standards in conducting and reporting research.
      • 44.
        Members should respect contractual rights when they agree to perform work for hire and should refrain from disseminating information that might be proprietary.
      • 45.
        When acting as KOLs or advisory board members, ISPOR members should be transparent about payments and any other benefits that they receive for acting in this capacity.
      • 46.
        When acting as KOLs or advisory board members, ISPOR members should ensure that the information they are presenting is an accurate representation of the facts available. They should respond to questions and queries honestly and to the best of their abilities.
      • 47.
        When relying on KOLs or advisory board members, ISPOR members should critically triangulate the claims being made. Where appropriate, they should seek independent corroboration of any factual claims and consider the full range of alternatives for themselves.
      • 48.
        When acting as KOLs or advisory board members, ISPOR members should maintain their independence and professional autonomy and act transparently (e.g., declare conflicts of interest).
      • 49.
        Members should respect the autonomy of research participants when designing and conducting studies, specifically including, but not limited to, informed consent and data privacy.
      • 50.
        Members should, where appropriate, seek input from individuals or organizations that are able to represent the perspective of patients.
      • 51.
        Members should, where appropriate, communicate their research findings to participants.

      Chapter 7: Patient Centricity and Patient Engagement in Research

      • 52.
        Patient input should be included throughout the medical product lifecycle from early development to dissemination and postmarketing surveillance.
      • 53.
        Researchers should endeavor to involve patients and their representatives as partners before, during, and after conducting research.
      • 54.
        To prevent or address ethical issues arising in the patient engagement process, follow published guidelines. Consultation with experienced stakeholders is useful as well.

      Chapter 8: Publication and Dissemination

      • 55.
        Members should not engage in any act of plagiarism, including self- plagiarism. If publishing work similar to anything jointly authored with others, members should ensure that no replication of the prior work was unintentionally done.
      • 56.
        It is extremely important to give comprehensive citations to avoid unintentional plagiarism.
      • 57.
        Members should not publish any material relating to the activities of an ISPOR Task Force, Special Interest Group, or other ISPOR group at any stage, without first consulting fellow group members/co-authors for permission.
      • 58.
        In the case of sponsored research, members should agree to a contract that clearly sets out their rights, scope of work, and rights of the sponsor (e.g., intellectual property rights and rights to publish) in the conduct and reporting of the study. A summary of this agreement should be provided in the published paper.
      • 59.
        Transparency in reporting is also essential to reduce the possibility of bias in research. Follow established reporting guidelines and endeavor to respond as fully as possible to requests for additional information on data or methods.
      • 60.
        Where an HEOR study is being conducted alongside a clinical study gathering data prospectively (e.g., a clinical trial or observational study), members should report whether the clinical study has been registered and, if so, where.
      • 61.
        Members should endeavor to publicly disseminate their work and to publish it in peer-reviewed journals, when possible.
      • 62.
        Members should work, where appropriate, to encourage the establishment and/or maintenance of an appropriate peer review process that examines the quality of the methodologic rigor independently of the organization for which the individual works.
      • 63.
        Members serving as peer reviewers for journals should respect the confidentiality of the material under review and understand that their access to it is solely for the purposes of performing the review.
      • 64.
        The description of study methods (design, study setting, data sources and input values, sampling, and analyses) should be complete and transparent enough for a suitably trained researcher to replicate the study.
      • 65.
        Methods sections of papers should give thorough, transparent attention to all measures taken to minimize bias.
      • 66.
        Where allowable by law and IRB/IEC approval, members should respond favorably to requests from journal editors and reviewers for access to original data and electronic copies of models where this access is required to ensure a rigorous peer review process and where commercial-in-confidence arrangements can be maintained.
      • 67.
        In those instances, in which study methods include analysis of a database (retrospective or prospective), members should describe approaches, methods, technologies used to ensure data completeness, and validity, as well as the software package(s) used for data analysis. Members should have the education, training, and experience to perform the assigned tasks.
      • 68.
        In those instances, in which sharing of model(s) and/or data source(s) is not feasible, members should be encouraged to provide supporting material demonstrating model and/or data validity, such as range and logic checks and assessment of data completeness.
      • 69.
        If submitting to a journal or publication that does not have peer review, or disseminating a report via electronic media, members should avoid the inclusion of material that cannot be supported by basic article references or make it clear that the article represents the author’s own opinion. If research is being reported, then access to the underlying data and/or analyses should be offered in the same manner as would be done under a peer-review process.
      • 70.
        It is important to disclose financial and/or other relationships that could be perceived as a conflict of interest, especially present or past relationships with manufacturers referred to in the research or any competitor products.

      Chapter 9: Conclusion and Limitations

      • 71.
        ISPOR expects its members to adhere to the highest ethical standards. By following these practices, stakeholders will be more likely to participate in health care research, as well as trust and benefit from it.
      • 72.
        HEOR is improved by following the highest possible ethical practices in terms of improved research, decision making, and, overall health and health care.

      Acknowledgments

      We gratefully acknowledge the following who submitted valuable written comments on the first and/or final draft of this task force report. All comments were considered. Most were substantive and constructive in improving the report.
      Special thanks to: Rob Camp (EUPATI), Jesús Enríquez Castillo, Kelly Lenahan, Brigitta Monz, Alesia Sadosky, and Jordana Schmier for their especially thorough and valuable reviews.
      Many thanks to the ISPOR members and ISPOR Regional Chapter & Consortia members who submitted written comments: Bhagwan Aggarwal, Tanzeel Ahmed Randhawa, Shohei Akita, Ola Ghaleb Al Ahdab, Ibrahim Alabbadi, Amad Al-Azzawi, Maryam Alowayesh, Nermeen Ashoush, Omid Beiki, Tania Maria Beume, Oksana Blavatska, Rob Boer, Lance Brannman, Giacomo Matteo Bruno, Michael Cangelosi, Josip Čulig, Saibal Das, Gordana Dragovic, Louise Drudy, Manal El-Hamamsy, Sonya Eremenco, Christoph Goergen, Amédé Gogovor, Vladimir Guzvic, Jan Hirsch, John Hornberger, Syed Umer Jan, Slobodan Jankovic, Denny John, Christopher Jones, Paschal Kaganda Nyendo, Deniz Karagülle, Kirandeep Kaur, Chanda Kulkarni, Alexey Kurilev, Yu Heng Kwan, Philip Lim, Enwu Long, Valentina Lorenzoni, Hsing Luh, Modesto Martinez Pillado, Bill McGhan, Christia Middleton, Sunita Nair, Winifred Ojieabu, Alfred Osoro, Oresta Piniazhko, Nathaniel Posner, Julian Prescod, KV Ramanath, Patricia Sacco, David Schwicker, Pathiyil Ravi Shankar, Anand Shewale, Sarah Shingler, Ralph Sobiech, Sushma Srivastava, Alexey Sychev, Ava Tai, Karunrat Tewthanom, Manoj G Tyagi, Jill Van Den Bos, Albert Wertheimer, Russell Wheeler, and Vladimir Zah and the Executive Committee of the ISPOR Ukraine Chapter.

      Supplementary material

      References

      1. United Nations Convention Against Corruption. Available from: https://www.unodc.org/unodc/en/treaties/CAC/. [Accessed September 25, 2017].

      2. The Foreign Corrupt Practices Act of 1977, 15 U.S.C. §§ 78dd-1, et seq. (FCPA). Available from: https://www.justice.gov/criminal-fraud/foreign-corrupt-practices-act. [Accessed September 25, 2017].

      3. International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Code of Ethics (2008). Available from: https://www.ispor.org/workpaper/CodeOfEthics.asp.

      4. World Medical Association. Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. [Accessed September 26, 2017].

      5. International Conference on Harmonization Good Clinical Practice (ICH GCP) Available from: http://ichgcp.net. [Accessed September 26, 2017].

      6. Agency for Healthcare Research and Quality (AHRQ). AHRQ’s National Guideline Clearinghouse. Available from: https://www.guideline.gov/.

        • European Federation of Pharmaceutical Industries and Associations (EFPIA)
        EFPIA Code of Practice on Relationships Between the Pharmaceutical Industry and Patient Organizations. 2011; (Available from: http://transparency.efpia.eu/uploads/Modules/Documents/code_po2011.pdf. [Accessed May 17, 2017])
      7. European Patients Academy (EUPATI). Guidance for Patient Involvement in R&D. Available from: https://www.eupati.eu/guidance-patient-involvement/.

      8. European Patients Academy (EUPATI) Guidance for patient involvement in HTA. Available from: https://www.eupati.eu/health-technology-assessment/guidance-for-patient-involvement-in-hta/. [Accessed September 22, 2017].

      9. U.S. Department of Health, Education, and Welfare (USHHS). The Belmont Report. (Office of the Secretary Ethical Principles and Guidelines for the Protection of Human Subjects of Research), The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research) April 18, 1979. Available from: https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/index.html.

        • Wang S.V.
        • Schneeweiss S.
        • Berger M.L.
        • et al.
        Reporting to improve reproducibility and facilitate validity assessment for healthcare database studies.
        Value Health. 2017; 20: 1023-1024
        • Berger M.L.
        • Sox H.
        • Willke R.J.
        • et al.
        Good practices for real-world data studies of treatment and/or comparative effectiveness: recommendations from the Joint ISPOR-ISPE Special Task Force on Real-World Evidence in Health Care Decision Making.
        Value Health. 2017; 20: 1003-1008
      10. Sanders GD, Neumann PJ, Basu A, et al., eds. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses second panel on cost-effectiveness in health and medicine. JAMA 2016;316(10):1093–103.

      11. Berger M.L. Bingefors K. Hedblom E.C. Health Care Cost, Quality, and Outcomes: ISPOR Book of Terms. ISPOR, Lawrenceville, NJ2003
      12. European Medicines Agency (EMA). Guideline on good pharmacovigilance practices (GVP), Module VIII – Post-authorization safety studies (Rev 2). Doc Ref. EMA/813938/2011. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129137.pdfc. [Accessed August 4, 2017].

      13. European Union. Directive 2010/84/EU of the European Parliament and of the Council (15 Dec 2010). Official Journal of the European Union. Available from: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:348:0074:0099:EN:PDF. [Accessed September 6, 2016].

      14. Center for Medicare and Medicaid Services (CMS) Chronic Conditions Data Warehouse (CCDW). Available from: https://www.ccwdata.org.

      15. American Academy of Otolaryngology – Head and Neck Surgery, Surveillance, Epidemiology and End Results (SEER). Available from: http://www.entnet.org/content/surveillance-epidemiology-and-end-results-seer-medicare.

      16. Hospital Episodes Statistics (HES). Available from: http://digital.nhs.uk/hesdata.

      17. Système National d'Informations Inter Régimes de l'Assurance Maladie (SNIIRAM) (French: National Information System Inter Plans Health Insurance). Available from: http://www.ameli.fr/.

        • Daniels N.
        Accountability for reasonableness: an update.
        BMJ. 2008; 337: a1850
        • Parmenter L.
        US and European perspectives on interventional and observational research designs in post-marketing safety.
        Applied Clinical Trials. 2012; (Available from)
      18. Food & Drug Administration (FDA). Observational Research Exhibit 18. Available from: http://www.fda.gov/ohrms/dockets/dockets/05p0224/05p-0224-cp00001-Exhibit-18-Observational-Research-vol1.pdf. [Accessed May 2017].

        • Williams R.J.
        • Tse T.
        • Harlan W.R.
        • Zarin D.A.
        Registration of observational studies: is it time?.
        CMAJ. 2010; 182 (Available from): 1638-1642
      19. National Institutes of Health. ClinicalTrials.gov. Available from: https://clinicaltrials.gov/. [Accessed November 20, 2017].

      20. American Healthcare Research & Quality. Registry of Patient Registries (RoPR). Available at: https://patientregistry.ahrq.gov/. [Accessed November 20, 2017].

      21. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP), The European Union electronic Register of Post-Authorisation Studies (EU PAS Register). Available from: http://encepp.eu/encepp/studiesDatabase.jsp. [Accessed May 17, 2017].

      22. International Society for Pharmacoeconomics and Outcomes Research (ISPOR). ISPOR Good Practices for Outcomes Research Index. Available from: http://www.ispor.org/workpaper/practices_index.asp.

        • Caro J.J.
        • Briggs A.H.
        • Siebert U.
        • et al.
        Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1.
        Value Health. 2012; 15: 796-803
        • Eddy D.M.
        • Hollingworth W.
        • Caro J.J.
        • et al.
        Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7.
        Value Health. 2012; 15: 843-850
        • Briggs A.H.
        • Weinstein M.C.
        • Fenwick E.
        • et al.
        Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6.
        Value Health. 2012; 15: 835-842
      23. Esposito D. Reliability and Validity of Data Sources for Outcomes Research & Disease and Health Management Programs. ISPOR, Lawrenceville, NJ2013
      24. General Data Protection Regulation (GDPR), European Commission. 2016. Available from: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679&qid=1462359521758&from=EN. [Accessed May 16, 2017]

      25. US Health & Human Services Health Insurance Portability and Accountability Act (HIPAA),US Department of Health & Human Services (1999, 2002). Available from: http://www.hhs.gov/hipaa/. [Accessed May 16, 2017].

      26. The Act on the Protection of Personal Information, Act No. 57 of (2003). Available from: http://www.cas.go.jp/jp/seisaku/hourei/data/APPI.pdf. [Accessed May 16, 2017].

      27. World Health Organization. WHO statement on public disclosure of clinical trial results (April 9, 2015). Available from: http://www.who.int/ictrp/results/reporting/en/. [Accessed September 26, 2017].

      28. International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Value in Health Policies. Available from: http://www.ispor.org/publications/value/policies.asp. [Accessed May 17, 2017].

      29. International Committee of Medical Journal Editors (ICMJE), Scientific Misconduct, Expressions of Concern, and Retraction. Available from: http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/scientific-misconduct-expressions-of-concern-and-retraction.html. [Accessed May 17, 2017].

      30. Committee on Publication Ethics (COPE) Council. Ethical guidelines for peer reviewers. September 2017. Available from: www.publicationethics.org.

        • Patient Centered Outcomes Research Institute (PCORI)
        Strategic Plan. 2013; (Available from: https://www.pcori.org/sites/default/files/PCORI-Strategic-Plan.pdf. [Accessed September 22, 2017])
        • Smith S.K.
        • Selig W.
        • Harker M.
        • et al.
        Patient engagement practices in clinical research among patient groups, industry, and academia in the United States: a survey.
        PloS One. 2015; (October 14) (. Available from)
        • Patient Centered Outcomes Research Institute (PCORI)
        Engagement rubric for applicants. 2016; (June 6,) (Available from:)
        • Walton M.K.
        • Powers J.A.
        • Hobart J.
        • et al.
        Clinical outcome assessments: a conceptual foundation—Report of the ISPOR Clinical Outcomes Assessment Emerging Good Practices Task Force.
        Value Health. 2015; 18: 741-752
        • Benjamin K.
        • Vernon M.K.
        • Patrick D.L.
        • et al.
        Patient reported outcome and observer reported outcome assessment in rare disease clinical trials—emerging good practices: an ISPOR COA Emerging Good Practices report.
        Value Health. 2017; 20: 838-855
        • Perfetto E.M.
        • Burke L.B.
        Criteria for successful patient engagement: an essential part of patient-focused drug development.
        (May 15) Presented at the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) 3rd Patient Representatives Roundtable, 2015
        • Matza L.S.
        • Patrick D.
        • Riley A.W.
        • et al.
        Pediatric patient-reported outcome instruments for research to support medical product labeling: Report of the ISPOR PRO good research practices for the assessment of children and adolescents task force.
        Value Health. 2013; 16: 461-479
        • Carman K.L.
        • Dardess P.
        • Maurer M.
        • et al.
        Patient and family engagement: a framework for understanding the elements and developing interventions and policies.
        Health Aff (Millwood). 2013; 32: 223-231
        • Domecq J.P.
        • Prutsky G.
        • Elraiyah T.
        • et al.
        Patient engagement in research: a systematic review.
        BMC Health Serv Res. 2014; 14 (89-6963-14-89)
        • Guise J.M.
        • O’Haire C.
        • McPheeters M.
        • et al.
        A practice-based tool for engaging stakeholders in future research: a synthesis of current practices.
        J Clin Epidemiol. 2013; 66: 666-674
        • Perfetto E.M.
        • Oehrlein E.M.
        • M-CERSI Conference on Patient-Focused Drug Development Planning Committee
        Assessing meaningful patient engagement in drug development: a definition, framework, and rubric.
        (March) University of Maryland, Baltimore, Maryland, School of Pharmacy Conference, 2015 (Available from: http://www.pharmacy.umaryland.edu/media/SOP/wwwpharmacyumarylandedu/centers/cersievents/pfdd/mcersi-pfdd-framework-rubric.pdf. [Accessed September 22, 2017])
      31. Medical Device Innovation Consortium (MDIC) Patient-Centered Benefit-Risk Assessment (PCBR). Available from: http://mdic.org/wp-content/uploads/2014/06/Patient-Centered-Benefit-Risk-PCBR-Project-update.pdf.

        • Patrick D.L.
        What does “patient-centered outcomes” mean?.
        (May 18) Presented at the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) 18th Annual Meeting, New Orleans, Louisiana2013
        • National Health Council (NHC)
        The patient voice in value: the National Health Council patient-centered value model rubric.
        (March)2016 (. Available from: http://www.nationalhealthcouncil.org/sites/default/files/Value-Rubric.pdf. [Accessed September 22, 2017])
      32. EURORDIS Rare Disease Europe. The EURORDIS Charter in practice. Available from: http://www.eurordis.org/IMG/pdf/Charter_Clinical_Trials-Final.pdf. [Accessed May 17, 2017].

      33. Patient Partner Project (European Forum for Good Clinical Practice (EFGCP), the European Genetic Alliances Network (EGAN), Genetic Alliance UK and the Dutch Genetic Alliance (VSOP)) Patient involvement in clinical research: a guide for sponsors and investigators. 2011; (Available from)
      34. Patient Centered Outcomes Research Institute (PCORI). Financial compensation of patients, caregivers, and patient/caregiver organizations engaged in PCORI-funded research as engaged research partners. Available from: http://www.pcori.org/sites/default/files/PCORI-Compensation-Framework-for-Engaged-Research-Partners.pdf. [Accessed May 17, 2017].

        • European Patients Academy (EUPATI)
        Compensation in clinical trials. 2015; ((October 28,) (Available from: https://www.eupati.eu/clinical-development-and-trials/compensation-clinical-trials/. [Accessed September 26, 2017])
        • Schulz K.F.
        • Altman D.G.
        • Moher D.
        • for the CONSORT Group
        CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomized trials.
        PLoS Med. 2010; 7: e60-e68
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
        Open Med. 2009; 3: e123-e130
        • Husereau D.
        • Drummond M.
        • Petrou S.
        • et al.
        Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—explanation and elaboration: A report of the ISPOR Health Economic Evaluations Publication Guidelines Good Reporting Practices Task Force.
        Value Health. 2013; 16: 231-250
        • Husereau D.
        • Drummond M.
        • Petrou S.
        • et al.
        • on behalf of the CHEERS Task Force
        Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.
        Value Health. 2013; 16: e1-e5
      35. Strengthening the Reporting of OBservational studies in Epidemiology (STROBE) Statement. Available from: http://www.strobe-statement.org/index.php?id=strobe-home. [Accessed May 17, 2017].

      36. Open Data Institute, Available from: https://theodi.org/what-is-open-data. [Accessed September 22, 2017].