Advertisement
Themed Section: Assesing the Value of Next-Generation Sequencing| Volume 21, ISSUE 9, P1043-1047, September 2018

Download started.

Ok

Valuation of Health and Nonhealth Outcomes from Next-Generation Sequencing: Approaches, Challenges, and Solutions

Open ArchivePublished:August 18, 2018DOI:https://doi.org/10.1016/j.jval.2018.06.010

      Abstract

      Background

      Next-generation sequencing (NGS) technologies have seen variable adoption in the clinic. This is partly due to a lack of clinical and economic studies, with the latter increasingly challenged to examine patient preferences for health and nonhealth outcomes (e.g., false-positive rate).

      Objectives

      To conduct a structured review of studies valuing patients’ preference-based utility for NGS outcomes, to highlight identified methodological challenges, and to consider how studies addressed identified challenges.

      Methods

      We searched MEDLINE (PubMed), Embase (Ovid), and Web of Science for published studies examining outcomes from health care decisions informed by NGS. We focused our search on direct elicitations of preference-based utility. We reviewed included studies and qualitatively grouped and summarized stated challenges and solutions by theme.

      Results

      Eleven studies were included. Most of them (n = 6) used discrete choice experiments to value utility. We categorized challenges into four themes: 1) valuing the full range of NGS outcomes, 2) accounting for accuracy and uncertainty surrounding effectiveness, 3) allowing for simultaneous multiple and cascading risks, and 4) incorporating downstream consequences. Studies found strong evidence of utility for NGS information, regardless of health improvement. Investigators addressed challenges by simplifying complex choices, by including health outcomes alongside nonhealth outcomes, and by using multiple elicitation techniques.

      Conclusions

      The breadth and complexity of NGS-derived information makes the technology a unique and challenging application for utility valuation. Failing to account for the utility or disutility of NGS-related nonhealth outcomes may lead to overinvestment or underinvestment in NGS, and so there is a need for research addressing unresolved challenges.

      Keywords

      Introduction

      Next-generation sequencing (NGS) is an umbrella term for massively parallel DNA sequencing technologies. The result of the application of NGS is information obtained from simultaneously interrogating multiple genes or the whole genome and their biological inter-relationships. Although NGS shows promise for more accurate patient stratification, the translation of NGS into the clinic has been variable [
      • Tripathy D.
      • Harnden K.
      • Blackwell K.
      • Robson M.
      Next generation sequencing and tumor mutation profiling: Are we ready for routine use in the oncology clinic?.
      ,
      • Auffray C.
      • Caulfield T.
      • Khoury M.J.
      • et al.
      Genome Medicine: past, present and future.
      ]. The variability in uptake has been attributed to a lack of evidence base demonstrating clinical effectiveness, clinical utility, and cost effectiveness.
      Health technology assessment (HTA) guidelines typically stipulate that off-the-shelf instruments should inform quality-adjusted life-years (QALYs) when answering questions of cost effectiveness. These instruments might not capture all the benefit-risk trade-offs of NGS health, nonhealth, and process outcomes. Buchanan et al. [
      • Buchanan J.
      • Wordsworth S.
      • Schuh A.
      Patients’ preferences for genomic diagnostic testing in chronic lymphocytic leukaemia: a discrete choice experiment.
      ] highlighted that measures informing QALYs do not incorporate preferences for nonhealth outcomes (e.g., false-positive rate) or process outcomes (e.g., time waiting for results). This observation is important in context of the assertion by Marshall et al. [
      • Marshall D.A.
      • Gonzalez J.M.
      • MacDonald K.V.
      • Johnson F.R.
      Estimating preferences for complex health technologies: lessons learned and implications for personalized medicine.
      ] that the value of NGS depends on the information that patients receive and the benefits that patients and providers ascribe to NGS information.
      Recently, the Second Panel on Cost Effectiveness in Health and Medicine made allowance for an economic evaluation reference case that takes account of nonhealth outcomes. The panel noted that decision makers need a “quantification and valuation of all health and non-health effects of interventions” [
      • Sanders G.D.
      • Neumann P.J.
      • Basu A.
      • et al.
      Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second Panel on Cost-Effectiveness in Health and Medicine.
      ]. In principle, this recommendation supports including preference-based utility in economic evaluation beyond what off-the-shelf instruments usually encapsulate.
      The types of outcomes that NGS produces are challenging to value, however. This is because NGS has the potential to uncover a multitude of complex clinically and nonclinically actionable results with far-ranging personal and familial implications. Given the import and complexity of NGS information, our objectives were to 1) conduct a structured review of studies valuing the preference-based utility of NGS health, nonhealth, or process outcomes from consumers’ perspectives; 2) highlight the conceptual and methodological challenges these studies encountered when estimating utility; and 3) consider how the included studies addressed the conceptual and stated challenges.

      Methods

      We conducted a literature search of full-text peer-reviewed articles in MEDLINE (PubMed), Embase (Ovid), and Web of Science. We restricted our search to articles in English published between January 1, 2005, and December 31, 2017. We chose the year 2005 because this was the year that NGS was being implemented in research settings. Our search strategy is outlined in the Appendix in Supplemental Materials found at doi:10.1016/j.jval.2018.06.010. After initial identification, we imported all articles into EndNote X6. Two of the researchers independently evaluated the title and abstract of all publications to identify articles for inclusion. We limited the search to direct elicitation of preference-based utility. We excluded studies that did not estimate stated preferences, did not focus on patient and/or general public perspectives, or did not focus on NGS. We identified stated challenges through authors’ statements on the motivation for estimating utility and in the discussion of study limitations. Using directed content analysis and the study by Marshall et al. [
      • Marshall D.A.
      • Gonzalez J.M.
      • MacDonald K.V.
      • Johnson F.R.
      Estimating preferences for complex health technologies: lessons learned and implications for personalized medicine.
      ], we grouped challenges according to categories. Solutions were based on study design and analytic approach, on next steps discussed for research, and on feedback from the working group.

      Results

      Study Acquisition Flow

      Figure 1 presents the flow of the included studies. The PubMed search identified 105 records. Four additional records were identified from searches in Ovid (MEDLINE) and Web of Science, as well as from citations in key articles. After screening titles and abstracts, 82 records were excluded and 27 full-text articles were assessed for eligibility. Of these, 11 studies directly elicited preference-based utility to examine the value of NGS health and nonhealth outcomes. Reasons for exclusion were studies not specifically examining NGS (n = 11) or not focusing on preferences from public or patient perspectives (n = 5).
      Fig. 1
      Fig. 1Flowchart describing articles identified and evaluated on the basis of inclusion criteria. NGS, next-generation sequencing.

      Study Characteristics

      Clinical context, end points, and perspective

      Detailed characteristics of each study are available in the Appendix in Supplemental Materials. The clinical contexts included NGS for prenatal testing, genomic testing to inform cancer interventions, and return of genomic information irrespective of disease. Of the included studies, 36% examined preferences from the general population’s perspective, 46% focused on the perspectives of patients or their families, and 18% examined both perspectives. The studies specified a number of end points, including preference-based utility, predicted uptake, and willingness to pay. These end points were chosen for various reasons. Four studies anticipated that their results would be used as inputs in economic evaluation. Two studies aimed to inform shared decision making, one study aimed to guide policy, two studies sought to inform early-stage technology development and investment, and two studies did not explicitly discuss the reason for preference elicitation.

      Methods and approaches to elicit preferences

      Figure 2 provides an overview of the applied methods, end points, and their potential uses within economic evaluation. The methods used were discrete choice experiments (DCEs; n = 6), contingent valuation (CV; n = 1), time trade-off (n = 1), as well as a combination of DCE, CV, probability trade-off, and/or ranking exercises (n = 3). Health, nonhealth, and process outcomes were identified through a combination of literature review, focus groups, in-person interviews, pilot testing, and expert opinion (see the Appendix in Supplemental Materials). Two studies did not state how they determined relevant outcomes. Most studies incorporated attributes for health, nonhealth, or process outcomes (n = 9). Attributes pertaining to health-related quality of life were included in four studies and involved likely benefit from treatment, likelihood of treatment side effects, complication rate, or pregnancy-specific outcomes. Health-related attributes described the risk of developing the disease after identifying a variant (n = 8), actionability of the genomic variant (n = 4), severity of the identified disease (n = 4), and/or carrier implications (n = 2). Nonhealth attributes included cost (n = 5), turnaround time (n = 3), type of procedure (n = 3), test reliability (n = 2), level of physician support or shared decision making (n = 2), and genetic test score (n = 1).
      Fig. 2
      Fig. 2Tree diagram depicting different preference-based approaches to valuation of NGS. CBA, cost-benefit analysis; CUA, cost-utility analysis; CV, contingent valuation; DCE, discrete choice experiment; NGS, next-generation sequencing; NMB, net monetary benefit; QALY, quality-adjusted life-year; TTO, time trade-off; WTP, willingness to pay.

      Conceptual and Methodological Challenges

      The studies highlighted a number of conceptual and methodological challenges for estimating the outcomes from NGS. The challenges relate to a need for valuing the full range of NGS-related health and nonhealth outcomes, capturing how preferences change with accuracy of genomic information, multiple scientific uncertainties, and capturing downstream consequences of NGS-generated information.

      Challenge 1: Valuing the range of NGS-related health, nonhealth, and process outcomes

      NGS produces genomewide information with the potential for multiple outcomes. It is the breadth and complexity of information that makes NGS a unique case for preference-based valuation. The studies noted that NGS provides information on one or all disease diagnoses, prognoses, treatment responses, and hereditary risks. The types of information returned become complex to communicate when NGS uncovers variants of uncertain significance (VUS) or secondary findings (SFs). The potential outcomes of SFs include uncovering risk for diseases that may or may not have effective treatment options, availability of early screening strategies for family members, or pharmacogenetic information on response to drugs. The continuing debate is whether patients should be given information on diseases that are not treatable [
      • Jarvik G.P.
      • Amendola L.M.
      • Berg J.S.
      • et al.
      Return of genomic results to research participants: the floor, the ceiling, and the choices in between.
      ,
      • Ploug T.
      • Holm S.
      Clinical genome sequencing and population preferences for information about “incidental” findings—from medically actionable genes (MAGs) to patient actionable genes (PAGs).
      ,
      • Regier D.A.
      • Peacock S.J.
      • Pataky R.
      • et al.
      Societal preferences for the return of incidental findings from clinical genomic sequencing: a discrete-choice experiment.
      ]. This is because patients may value the information absent of effective treatment and improvement in health status.

      How studies conceptually addressed the broad range of outcomes

      Researchers in the genetics and health economics literature have labeled the value of nonhealth outcomes as “personal utility,” which relates to the benefit that patients ascribe to the spectrum of information derived from genomic technologies, regardless of its potential to improve health [
      • Regier D.A.
      • Friedman J.M.
      • Makela N.
      • et al.
      Valuing the benefit of diagnostic testing for genetic causes of idiopathic developmental disability: willingness to pay from families of affected children.
      ,
      • Foster M.W.
      • Mulvihill J.J.
      • Sharp R.R.
      Evaluating the utility of personal genomic information.
      ]. Articulating the concept in this way was important because it broadened the scope of what the genetics literature considered as beneficial or of value to patients, families, and the health care system. The included articles articulated this concept of personal utility in two ways: 1) as the benefits or harms manifested outside medical contexts, as the value of genomic information, or as the value of knowledge that is separate from clinical effectiveness or 2) as the worth individuals ascribe to the full range of NGS testing outcomes, including health, nonhealth, and process outcomes.
      The included studies provided evidence that supports the importance of health and nonhealth outcomes to the valuation of benefit. In all studies, respondents valued informational and process attributes. Regier et al. [
      • Regier D.A.
      • Peacock S.J.
      • Pataky R.
      • et al.
      Societal preferences for the return of incidental findings from clinical genomic sequencing: a discrete-choice experiment.
      ] found that 27% of the general population would want SF information for disorders with severe quality-of-life consequences, irrespective of whether effective medical treatment was available. Marshall et al. [
      • Marshall D.A.
      • Gonzalez J.M.
      • Johnson F.R.
      • et al.
      What are people willing to pay for whole-genome sequencing information, and who decides what they receive?.
      ] found that 45% of adult respondents were willing to pay for information on a variant for which there was no effective treatment available. Lewis et al. [
      • Lewis M.A.
      • Stine A.
      • Paquin R.S.
      • et al.
      Parental preferences toward genomic sequencing for non-medically actionable conditions in children: a discrete-choice experiment.
      ] determined that parents were interested in the return of highly penetrant nonmedically actionable conditions in children, particularly if manifestations were more severe (e.g., earlier age of onset and greater level of disability). Buchanan et al. [
      • Buchanan J.
      • Wordsworth S.
      • Schuh A.
      Patients’ preferences for genomic diagnostic testing in chronic lymphocytic leukaemia: a discrete choice experiment.
      ] and Cuffe et al. [
      • Cuffe S.
      • Hon H.
      • Qiu X.
      • et al.
      Cancer patients’ acceptance, understanding, and willingness-to-pay for pharmacogenomic testing.
      ] provide evidence of disutility for waiting longer for the results of NGS, with the former also finding that respondents had a preference for who delivered the results. Marshall et al. [
      • Marshall D.A.
      • Deal K.
      • Bombard Y.
      • et al.
      How do women trade-off benefits and risks in chemotherapy treatment decisions based on gene expression profiling for early-stage breast cancer? A discrete choice experiment.
      ] found that women preferred to receive chemotherapy for early-stage breast cancer if they trusted their oncologist’s opinion, irrespective of NGS estimates of recurrence risk and likely benefit from chemotherapy.

      Challenge 2: NGS testing varies in accuracy and has uncertain effectiveness outcomes

      Preference-based research has established the importance of accuracy and clinical utility of diagnostic information [
      • Groothuis-Oudshoorn C.G.
      • Fermont J.M.
      • van Til J.A.
      • IJzerman M.J.
      Public stated preferences and predicted uptake for genome-based colorectal cancer screening.
      ,
      • Eden M.
      • Payne K.
      • Combs R.M.
      • et al.
      Valuing the benefits of genetic testing for retinitis pigmentosa: a pilot application of the contingent valuation method.
      ]. In the context of NGS, it has been noted that the evidence needed to support estimates of accuracy and clinical utility requires significant amounts of data. This is apparent in large-scale sequencing initiatives such as the UK 100,000 Genomes Project and the US Precision Medicine Initiative [
      • Caulfield M.
      • Davies J.
      • Dennys M.
      • et al.
      The 100,000 Genomes Project Protocol.
      ,
      National Institutes of Health
      All of Us Research Program—Protocol V1.
      ]. On a smaller scale, targetable biomarkers are increasingly sought retrospectively as part of randomized controlled trials not powered to find an effect. Given individual genomic heterogeneity and the requirement of large amounts of data, significant valuation challenges arise because NGS outcomes are subject to imprecision around health outcomes (including uncertainty surrounding clinical utility) and prediction error (accuracy), which can lead to disease misclassification [
      • Cuffe S.
      • Hon H.
      • Qiu X.
      • et al.
      Cancer patients’ acceptance, understanding, and willingness-to-pay for pharmacogenomic testing.
      ,
      • Najafzadeh M.
      • et al.
      Genomic testing to determine drug response: measuring preferences of the public and patients using Discrete choice experiment (Dce).
      ].

      How studies addressed accuracy and uncertain effectiveness outcomes

      The impact of accuracy has been included through nonhealth attributes describing sensitivity, specificity, or reliability. In precision oncology, Najafzadeh et al. [
      • Najafzadeh M.
      • et al.
      Genomic testing to determine drug response: measuring preferences of the public and patients using Discrete choice experiment (Dce).
      ] addressed sensitivity through describing the proportion of patients who could be cured by a new medication but did not receive the medication because of inaccurate results. Specificity was the proportion of patients who would not benefit from the new medication but would receive it because of incorrect test results. Buchanan et al. [
      • Buchanan J.
      • Wordsworth S.
      • Schuh A.
      Patients’ preferences for genomic diagnostic testing in chronic lymphocytic leukaemia: a discrete choice experiment.
      ] incorporated test reliability into their DCE, defining the attribute in terms of the number of tests that provide an incorrect result. These studies found that respondents could distinguish between attribute levels describing various levels of accuracy. None of the included studies addressed the potential for imprecision around clinical effectiveness, and this remains an important application of future research.

      Challenge 3: NGS has simultaneous multiple and cascading uncertainties

      NGS information includes a process of patients simultaneously receiving layers of information. The initial decision to undergo genomic testing carries immediate risk-benefit trade-offs, including, for example, the risk of adverse effects associated with a biopsy, test accuracy, and the probability of identifying a pathogenic variant.
      The challenge is to represent these multiple risks and their interdependencies, termed “multiple cascading uncertainties” by Marshall et al. [
      • Marshall D.A.
      • Gonzalez J.M.
      • MacDonald K.V.
      • Johnson F.R.
      Estimating preferences for complex health technologies: lessons learned and implications for personalized medicine.
      ]. From a choice-theoretic perspective, these streams of sequential information and probabilities can be thought of as endogenous to individuals’ choices. This type of endogeneity has been termed “multiple discreteness” [
      • Louviere J.
      • Train K.
      • Ben-Akiva M.
      • et al.
      Recent progress on endogeneity in choice modeling.
      ]. The nature of these probabilities and their influence on choice are not easily incorporated into standard stated preference methods. In terms of econometric estimation, modeling utility without reference to multiple discreteness may result in inconsistent estimators. If these probabilities are included, appropriately presented, and understood by respondents, the econometric model must take into account the interdependence of the attribute levels and their sequential nature with the possibility of diminishing marginal utility as information is returned at various time points [
      • Bhat C.R.
      The multiple discrete-continuous extreme value (MDCEV) model: role of utility function parameters, identification considerations, and model extensions.
      ].

      How studies have examined simultaneous multiple and cascading risks

      Included studies addressed this challenge by simplifying the complexity of the choice process. Regier et al. [
      • Regier D.A.
      • Peacock S.J.
      • Pataky R.
      • et al.
      Societal preferences for the return of incidental findings from clinical genomic sequencing: a discrete-choice experiment.
      ] focused solely on the return of SFs by describing penetrance (number of people with a genetic variant who actually get the disease), availability of effective interventions, quality of life, return of carrier status, and cost. They did not take into account the upstream choice of undergoing NGS testing for the primary condition. Instead, they asked respondents to imagine that they had been diagnosed with an unspecified disease and that SFs may be found. Marshall et al. [
      • Marshall D.A.
      • Gonzalez J.M.
      • MacDonald K.V.
      • Johnson F.R.
      Estimating preferences for complex health technologies: lessons learned and implications for personalized medicine.
      ] simplified their decision scenario by dividing the process into two parts. First, respondents were asked to complete a CV task aimed at understanding the value of NGS information for a broad set of uncertain outcomes. This was followed by DCE tasks that aimed to understand whether respondents would be willing to act on the information they received. The aim of this approach was to reduce the dimensionality of the problem, with the authors concluding that the DCE method alone is limited when inferring more realistic decision-making situations. None of the studies identified approaches to deal with the endogeneity associated with multiple discreteness.

      Challenge 4: Downstream health and nonhealth consequences

      Downstream outcomes from NGS are contingent on whether information is returned and acted upon [
      • Marshall D.A.
      • Gonzalez J.M.
      • MacDonald K.V.
      • Johnson F.R.
      Estimating preferences for complex health technologies: lessons learned and implications for personalized medicine.
      ]. For a hereditary condition, there are downstream outcomes beyond the patient alone because the health care system may fund NGS testing for family members. Additional testing and the concomitant risk-benefit outcomes discussed in the previous section will depend on the patient choosing to notify family members. The choice of the patient can also generate a negative or positive externality. This is because patients’ decisions can affect future health and nonhealth outcomes of family members. From a health care system perspective, a negative externality can occur in the context of overdiagnosis if the affected individual consumes unnecessary health care, potentially incurring the opportunity cost of displacing health from other patients.
      In terms of SF, the downstream benefits of identified conditions with effective treatments can be captured by existing preference-based measures. Findings of VUS are potentially associated with treatable conditions that patients are at risk for in the future, but have the added complexity of uncertainty surrounding whether the variant causes disease. Importantly, a VUS may never be determined as clinically relevant because the numbers of other patients with the same rare variant who both have NGS and who express the phenotype may never be sufficient to establish causality [
      • MacArthur D.G.
      • Manolio T.A.
      • Dimmock D.P.
      • et al.
      Guidelines for investigating causality of sequence variants in human disease.
      ]. Conceptually, this is related to the economic idea of Knightian uncertainty. Proposed by economist Frank Knight [
      • Knight F.H.
      Risk, Uncertainty and Profit (Hart, Schaffner and Marx Prize Essays).
      ], this type of uncertainty arises in scenarios in which individuals cannot know all the information they need to accurately understand the odds of a particular outcome occurring. The implication of Knightian uncertainty is that the odds of an outcome are incalculable and as such both the immediate and the long-term preference-based utilities for the outcome are impossible for an individual to calculate. Taking a Bayesian perspective, however, individuals may still attach probabilities to these unknown unknowns.

      How studies have incorporated downstream consequences

      The approach of Marshall et al. [
      • Marshall D.A.
      • Gonzalez J.M.
      • MacDonald K.V.
      • Johnson F.R.
      Estimating preferences for complex health technologies: lessons learned and implications for personalized medicine.
      ] was to first specify a profile of health problems related to a hypothetical gene variant and then ask individuals to state a preference between possible downstream medical interventions. These interventions would effectively reduce the risk of health problems arising from the variant but would also carry possible adverse side effects and require out-of-pocket costs. The individual could also choose to undergo a watchful waiting scenario. They found that respondents valued NGS information most if there was a noninvasive medical intervention available to reduce their risk of developing a health problem. Preferences for undergoing a preventative medical intervention were greatly affected by the probability of adverse side effects. Kupperman et al. [
      • Kuppermann M.
      • Norton M.E.
      • Thao K.
      • et al.
      Preferences regarding contemporary prenatal genetic tests among women desiring testing: implications for optimal testing strategies.
      ] applied time trade-off techniques to elicit preferences for potential prenatal testing outcomes, including the identification of a VUS. Scenarios involving pregnancy termination in the context of a VUS or having a baby with a VUS received low utility scores.

      Discussion

      We identified 11 studies estimating the preference-based value of NGS outcomes. We found that authors directly elicited preference-based utility for NGS because off-the-shelf instruments do not adequately account for the breadth or types of NGS outcomes that patients or families may value. This hypothesis was empirically supported in the results of the included studies.
      The evidence suggests that ignoring the preference-based value for health and nonhealth NGS outcomes in economic evaluation may lead to overinvestment or underinvestment if a system wants to maximize patients’ utility from health care services. That is, health care systems may overinvest if individuals have disutility for aspects of NGS-generated knowledge and underinvest if they place importance on the NGS knowledge [
      • Ioannidis J.P.
      • Khoury M.J.
      Are randomized trials obsolete or more important than ever in the genomic era?.
      ]. Although such analyses will not replace the use of QALYs as the HTA reference case, we believe demand-side approaches to economic evaluation can be used as additional evidence for decision makers’ consideration [
      • Lakdawalla D.N.
      • Doshi J.A.
      • Garrison Jr, L.P.
      • et al.
      Defining elements of value in health care—a health economics approach: an ISPOR Special Task Force Report [3].
      ].
      Most of the included studies used DCEs to directly elicit preferences. Nevertheless, given the few preference studies of NGS and a lack of HTA recommendations regarding using DCE to inform resource allocation, a crucial area of research is to outline recommendations on which stated preference methods are appropriate for supporting economic evaluation of NGS. We expect that these recommendations will also need to consider important equity issues surrounding the use of metrics other than QALYs. This is a critical issue that is beyond the scope of this study.
      The included studies outlined several challenges when eliciting the value of the large amounts of complex information obtained by NGS. We found that the solutions to challenges were limited and not comprehensive in approach, and they were broadly related to discussions of concepts on what constitutes benefit, descriptions of risk and accuracy, simplifying complex choice tasks, or sequentially using multiple valuation techniques. Although important first steps, these solutions are not unique and there remains scope for a comprehensive solution to valuing NGS outcomes.

      Conclusions

      Researchers are facing challenges when estimating the preference-based value of NGS. We conclude that it is the breadth and complexity of information that makes NGS-guided health care a unique valuation case study. Indeed, we identified challenges that are a direct result of the types of information that NGS provides. There remains considerable scope regarding providing comprehensive solutions to these challenges. We highlight the problems of incorporating uncertainty surrounding outcomes, endogeneity of choice, and downstream consequences of NGS testing as areas for needed research.

      Acknowledgments

      Dr. Regier was supported by the Canadian Centre for Applied Research in Cancer Control, which receives funding from the Canadian Cancer Society Research Institute grant #2015-019789. The authors would like to thank Dr. Kathryn Phillips for her contributions that led to the improvement of this article. The results and view expressed in this manuscript remain the authors’ own.
      Source of financial support: Dr. Marshall reports personal fees from Board Membership: Pfizer; personal fees from Consultancy: Optum Insight; personal fees from Consultancy: Research Triangle Institute; personal fees from Consultancy: Roche; personal fees and non-financial support from Consultancy: Novartis, personal fees and non-financial support from Consultancy: Abbvie; personal fees and non-financial support from Consultancy: Janssen, grants from Grants/grants pending, outside the submitted work; and Received travel support from Illumina to attend a past working group meeting in Boston, MA, USA. Illumina Inc. provided travel fund to discuss the theme issue on precision medicine.

      Supplemental Materials

      References

        • Tripathy D.
        • Harnden K.
        • Blackwell K.
        • Robson M.
        Next generation sequencing and tumor mutation profiling: Are we ready for routine use in the oncology clinic?.
        BMC Med. 2014; 12: 140
        • Auffray C.
        • Caulfield T.
        • Khoury M.J.
        • et al.
        Genome Medicine: past, present and future.
        Genome Med. 2011; 3: 6
        • Buchanan J.
        • Wordsworth S.
        • Schuh A.
        Patients’ preferences for genomic diagnostic testing in chronic lymphocytic leukaemia: a discrete choice experiment.
        Patient. 2016; 9: 525-536
        • Marshall D.A.
        • Gonzalez J.M.
        • MacDonald K.V.
        • Johnson F.R.
        Estimating preferences for complex health technologies: lessons learned and implications for personalized medicine.
        Value Health. 2017; 20: 32-39
        • Sanders G.D.
        • Neumann P.J.
        • Basu A.
        • et al.
        Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second Panel on Cost-Effectiveness in Health and Medicine.
        JAMA. 2016; 316: 1093-1103
        • Jarvik G.P.
        • Amendola L.M.
        • Berg J.S.
        • et al.
        Return of genomic results to research participants: the floor, the ceiling, and the choices in between.
        Am J Hum Genet. 2014; 94: 818-826
        • Ploug T.
        • Holm S.
        Clinical genome sequencing and population preferences for information about “incidental” findings—from medically actionable genes (MAGs) to patient actionable genes (PAGs).
        PLoS One. 2017; 12: e0179935
        • Regier D.A.
        • Peacock S.J.
        • Pataky R.
        • et al.
        Societal preferences for the return of incidental findings from clinical genomic sequencing: a discrete-choice experiment.
        CMAJ. 2015; 187: E190-E197
        • Regier D.A.
        • Friedman J.M.
        • Makela N.
        • et al.
        Valuing the benefit of diagnostic testing for genetic causes of idiopathic developmental disability: willingness to pay from families of affected children.
        Clin Genet. 2009; 75: 514-521
        • Foster M.W.
        • Mulvihill J.J.
        • Sharp R.R.
        Evaluating the utility of personal genomic information.
        Genet Med. 2009; 11: 570-574
        • Marshall D.A.
        • Gonzalez J.M.
        • Johnson F.R.
        • et al.
        What are people willing to pay for whole-genome sequencing information, and who decides what they receive?.
        Genet Med. 2016; 18: 1295-1302
        • Lewis M.A.
        • Stine A.
        • Paquin R.S.
        • et al.
        Parental preferences toward genomic sequencing for non-medically actionable conditions in children: a discrete-choice experiment.
        Genet Med. 2018; 20: 181-189
        • Cuffe S.
        • Hon H.
        • Qiu X.
        • et al.
        Cancer patients’ acceptance, understanding, and willingness-to-pay for pharmacogenomic testing.
        Pharmacogenet Genomics. 2014; 24: 348-355
        • Marshall D.A.
        • Deal K.
        • Bombard Y.
        • et al.
        How do women trade-off benefits and risks in chemotherapy treatment decisions based on gene expression profiling for early-stage breast cancer? A discrete choice experiment.
        BMJ Open. 2016; 6: e010981
        • Groothuis-Oudshoorn C.G.
        • Fermont J.M.
        • van Til J.A.
        • IJzerman M.J.
        Public stated preferences and predicted uptake for genome-based colorectal cancer screening.
        BMC Med Inform Decis Mak. 2014; 14: 18
        • Eden M.
        • Payne K.
        • Combs R.M.
        • et al.
        Valuing the benefits of genetic testing for retinitis pigmentosa: a pilot application of the contingent valuation method.
        Br J Ophthalmol. 2014; 97: 1051-1066
        • Caulfield M.
        • Davies J.
        • Dennys M.
        • et al.
        The 100,000 Genomes Project Protocol.
        Genomics England, London2015
        • National Institutes of Health
        All of Us Research Program—Protocol V1.
        National Institutes of Health, Bethesda, MD2017
        • Najafzadeh M.
        • et al.
        Genomic testing to determine drug response: measuring preferences of the public and patients using Discrete choice experiment (Dce).
        BMC health services research. 2013; 13: 454
        • Louviere J.
        • Train K.
        • Ben-Akiva M.
        • et al.
        Recent progress on endogeneity in choice modeling.
        Mark Lett. 2005; 16: 255-265
        • Bhat C.R.
        The multiple discrete-continuous extreme value (MDCEV) model: role of utility function parameters, identification considerations, and model extensions.
        Transp Res B Method. 2008; 42: 274-303
        • MacArthur D.G.
        • Manolio T.A.
        • Dimmock D.P.
        • et al.
        Guidelines for investigating causality of sequence variants in human disease.
        Nature. 2014; 508: 469-476
        • Knight F.H.
        Risk, Uncertainty and Profit (Hart, Schaffner and Marx Prize Essays).
        Houghton Mifflin, Boston, MA1921: 31
        • Kuppermann M.
        • Norton M.E.
        • Thao K.
        • et al.
        Preferences regarding contemporary prenatal genetic tests among women desiring testing: implications for optimal testing strategies.
        Prenat Diagn. 2016; 36: 469-475
        • Ioannidis J.P.
        • Khoury M.J.
        Are randomized trials obsolete or more important than ever in the genomic era?.
        Genome Med. 2013; 5: 32
        • Lakdawalla D.N.
        • Doshi J.A.
        • Garrison Jr, L.P.
        • et al.
        Defining elements of value in health care—a health economics approach: an ISPOR Special Task Force Report [3].
        Value Health. 2018; 21: 131-139

      Linked Article

      • Correction
        Value in HealthVol. 22Issue 4
        • Preview
          Valuation of Health and Nonhealth Outcomes from Next-Generation Sequencing: Approaches, Challenges, and Solutions
        • Full-Text
        • PDF
        Open Archive