Highlights
- •Simulation modeling methods such as discrete event simulation may be better suited than traditional state-transition cohort models to address the complexity and specific challenges of economic evaluation of precision medicine interventions.
- •Simulation models can be used for patient-level analyses of care pathways and have the ability to deal with system complexity of multiple tests, diagnostic performance, and testing and treatment sequences that present particular challenges for precision medicine.
Abstract
Objectives
The objective of this article is to describe the unique challenges and present potential solutions and approaches for economic evaluations of precision medicine (PM) interventions using simulation modeling methods.
Methods
Given the large and growing number of PM interventions and applications, methods are needed for economic evaluation of PM that can handle the complexity of cascading decisions and patient-specific heterogeneity reflected in the myriad testing and treatment pathways. Traditional approaches (eg, Markov models) have limitations, and other modeling techniques may be required to overcome these challenges. Dynamic simulation models, such as discrete event simulation and agent-based models, are used to design and develop mathematical representations of complex systems and intervention scenarios to evaluate the consequence of interventions over time from a systems perspective.
Results
Some of the methodological challenges of modeling PM can be addressed using dynamic simulation models. For example, issues regarding companion diagnostics, combining and sequencing of tests, and diagnostic performance of tests can be addressed by capturing patient-specific pathways in the context of care delivery. Issues regarding patient heterogeneity can be addressed by using patient-level simulation models.
Conclusion
The economic evaluation of PM interventions poses unique methodological challenges that might require new solutions. Simulation models are well suited for economic evaluation in PM because they enable patient-level analyses and can capture the dynamics of interventions in complex systems specific to the context of healthcare service delivery.
Keywords
Background
Economic evaluations are now commonly used by health technology assessment (HTA) organizations and payers to assess the value of new healthcare interventions (including medicines, pharmaceuticals, devices, and healthcare programs) and to inform decision makers about the efficient allocation of healthcare resources.
1
Many countries have developed guidance on the key considerations in economic evaluations such as the target population, analytical perspective, choice of comparator, analytical methods, outcome measures, measures of utility, costs to be included, time horizon, discounting, and sensitivity analysis.2
These economic evaluation guidelines commonly also consider the use of economic modeling.3
,4
Traditional modeling approaches, such as decision trees or Markov cohort models, typically examine the value of 1 test compared with standard care, where this test is conducted for a specific clinical reason, and provide 1 result (eg, diagnosis) and a single trajectory of costs and outcomes for cohorts of “average” patients.5
,6
Nevertheless, the notion of the “average” person is challenged in the evaluation of precision medicine (PM) interventions.7
, 8
, 9
From a broad perspective, personalized approaches in health applications can be defined as those using any type of patient-specific information (diagnostic tests, patient-reported outcomes, risk estimates, functional performance) to inform therapy tailored to the patient.
7
,10
Ginsburg and Phillips11
differentiated between personalized and precision medicine with precision medicine going beyond genomics, preferences, beliefs, attitudes, knowledge, and social context.11
They described PM as a model for healthcare delivery that relies on data, analytics, and information with the ability to guide healthcare decisions tailored to a specific patient. It is generally acknowledged that PM interventions have the ability to guide healthcare decisions tailored to a specific patient, with the aim of improving the effectiveness and quality of care, reducing related adverse events, and reducing the need for unnecessary testing and treatment.11
Although advanced diagnostic tests used to target treatments can drive improved outcomes, the result of using these tests is a complex and dynamic set of treatment pathways that varies for each individual and where the downstream consequences and outcomes are difficult to predict. To support decision making in PM, models need to reflect the dynamic treatment pathways, the underlying clinical evidence for improved outcomes, and the uncertainties around the evidence.
12
In the past decade, HTA organizations have considered a large and growing number of PM interventions for approval and reimbursement. For example, pembrolizumab was the first solid cancer therapy approved for use by the Food and Drug Administration in 2017 based on the presence of a specific biomarker rather than the location of a particular tumor.
13
Methods are needed for economic evaluation of PM that can handle this complexity of cascading decisions, reflected in the myriad treatment pathways, while presenting results that provide meaningful information to support health policy decisions.Given that many of the current health economic modeling approaches are not likely able to capture this complexity, the objective of this article is to present dynamic simulation modeling as another modeling paradigm that can manage this complexity. The article first describes the challenges of economic evaluation in PM and how simulation modeling methods can potentially address some of these challenges. Then we provide an overview of common simulation modeling methods in health. Finally, empirical examples are used to illustrate how simulation modeling methods could be and have been used to capture the individual care pathways as seen in PM.
Specific Technical Challenges of Economic Evaluation in PM
The economic evaluation of PM interventions epitomizes the need for methods that reflect a system view to healthcare that captures the upstream and downstream consequences of changes in healthcare with multiple decision points based on patient-level characteristics. PM typically involves multiple diagnostic tests and treatments over time, with the sequence of these tests and treatments differing between individual patients.
11
These pathways or clinical trajectories are complex, dynamic, and specific to the context of the delivery of healthcare services.8
Cohort and state-transition models have limited ability to deal with such pathways and typically report results for an “average” patient.6
,14
The challenges of conducting economic evaluations of PM have been synthesized by Phillips et al
9
using next-generation sequencing (NGS) as an example. NGS tests including targeted gene panels, whole-exome sequencing, and whole-genome sequencing, which simultaneously examine multiple genes and can contribute to making treatment recommendations. The top-priority challenges include (1) the requirement for a complex model structure that incorporates multiple pathways, results, and testing uses to reflect the fact that NGS tests evaluate multiple genes and to consider interactive effects across multiple conditions and (2) the need to specify a time frame that captures both the upstream and downstream costs and outcomes specific to NGS and the need to capture downstream consequences for patients and families.Additional conceptual and methodological challenges in PM relate to the valuation of outcomes including preferences for genomic information, multiple layers of uncertainty, and risks.
15
Patient heterogeneity is critically important in the context of economic evaluation in PM as individual care pathways and decisions about treatment and care need to be considered in the analysis. Although current guidelines for health economic modeling recognize the importance of patient heterogeneity,3
,16
there is little specific guidance on how to incorporate heterogeneity in cost-effectiveness models.Nonetheless, in a systematic literature review of health economic models specifically in personalized and PM (in general and including patient stratification by other means than genetics) in all disease areas, Degeling et al
8
reported that decision tree models and Markov models remain the most frequently used approach (n = 21 of 31 publications, 66%). When analyzed over time, there was an observed increase in patient-level simulation modeling methods such as discrete event simulation (DES) since 2011. This review also provides a checklist of the 10 main challenges for health economic modeling in PM, and these challenges are summarized in Table 1 along with a description of how these challenges can potentially be addressed by simulation modeling methods rather than traditional health economic modeling methods. Only a minority of these health economic models of PM addressed the 10 items in the checklist or otherwise identified and reported the challenge. The most commonly addressed challenge was the diagnostic performance of the test (n = 20 of 31 studies, 65%) and the different combinations of test(s) and treatments(s) (n = 22 of 31 studies, 71%). Although 11 of the models (35%) included combinations of tests, all assumed a fixed sequence of tests. Only 6 of the models (19%) were defined on a patient level. These findings suggest that economic models of PM have not yet fully embraced these more advanced simulation modeling methods to address these challenges.Table 1Checklist of challenges for health economic modeling in the context of personalized medicine and how these challenges can potentially be addressed by simulation modeling.
8
| Challenge | Specification of challenge in the checklist | How these challenges are addressed by simulation modeling and limitations compared with traditional health economic modeling |
|---|---|---|
| 1. Modeling patient-level processes | Is the model defined on a patient level? | Patient-level models reflecting care pathways considering context of delivery |
| 2. Modeling patients’ preferences | Are patients’ preferences modeled to take their effect on the outcomes into account? | Incorporate at decision nodes the probability of uptake based on patient preferences; issue of availability of the data and the attributes from preferences need to align with variables in the model/care pathway |
| 3. Modeling physicians’ preferences | Are physicians’ preferences modeled to take their effect on the outcomes into account? | Incorporate at decision nodes the probability of uptake based on physician preferences; issue of availability of the data and the attributes from preferences need to align with variables in the model/care pathway |
| 4. Taking into account the diagnostic performance of tests | Is the effect of the sensitivity, specificity, positive predictive value, and/or negative predictive value on the outcomes taken into account? | Include compound probabilities based on patient-specific pathways considering context of care delivery |
| 5. Modeling combinations of tests | Does the modeled process include combinations of tests and/or prediction models? | Include compound probabilities based on patient-specific pathways |
| 6. Modeling companion diagnostics | Does the modeled process include combinations of test(s) and treatment(s)? | Include compound probabilities based on patient-specific pathways |
| 7. Study-specific outcome measures | Does the modeled process include study-specific outcomes, such as disease-specific adverse events? | Patient-level models reflecting care pathways and patient-specific outcomes based on patient characteristics |
| 8. Data gaps | Do the authors mention any evidence gaps? If so, do they mention that these evidence gaps exist because of stratification of patients based on risk models and/or test results? | Simulation models offer greater flexibility to include patient-specific pathways and account for stratification of patients based on risk models and/or test results |
| 9. Greater uncertainty due to more complex analysis | Do the authors mention greater uncertainty with respect to the outcomes, due to more complex analysis, as a result of personalization of the model? | Simulation models offer greater flexibility to include patient-specific pathways and account for uncertainty at a patient level; there remain challenges to aggregate these findings |
| 10. Absence of guidelines | Do the authors mention any difficulties related to the absence of guidelines for health economic modeling in the context of personalized medicine? | There is guidance for simulation modeling from the operations research literature and emerging in health |
Overview of Simulation Modeling Methods in Health Economic Evaluation
Health economic evaluation models have commonly used cohort-based Markov or state-transition modeling to reflect the clinical evidence and estimate the cost-effectiveness of an intervention compared with usual care in the form of an incremental cost-effectiveness ratio. For example, of 58 publications of economic evaluations modeling therapies for rheumatoid arthritis identified between 1996 and 2012, 38 (66%) were decision trees (n = 13) or Markov models (n = 25).
17
In these models, a limited number of health states represent the care pathway for a group of patients. These models typically aggregate the complexity in the real world into a few distinct health states and therefore heavily rely on assumptions about how patients move from one health state to another and how overall outcomes are synthesized or extrapolated, and importantly, they neglect the heterogeneity at the patient level.The use of cohort models has been partly driven by the use of economic evaluation in the context of HTA, which has historically focused on single interventions (often a new medicine).
18
Further, many of these models are trial based, and they therefore model the relatively straightforward head-to-head comparison of 2 interventions.Nevertheless, a different paradigm is needed as HTA increasingly breaks out of the “adoption addiction” and moves toward health technology management with a broader systems view of innovation, adoption, and disinvestment throughout the technology life cycle to support healthcare decision making.
18
,19
Dynamic simulation modeling is a collection of methods developed primarily in the context of business operations that may be better suited than cohort-based models to reflect the dynamics of the health system and delivery of services.20
Dynamic simulation models use mathematical representations of complex systems with multiple intervention scenarios and evaluate their consequences over time from a systems perspective.
20
Complex systems consist of tasks that are relationally dependent events with unpredictable outcomes.21
As a consequence, simulation models are generally nonlinear and described implicitly through rules or equations, whereas state-transition models are typically linear and defined by transition matrices between health states that are indexed by time. The nonlinearity in simulation models implies that the individual must be simulated in context of the broader context of other individuals and the healthcare system across time such that emergent behavior reflects the system rather than individuals.20
,22
The literature on the applications of simulation modeling in healthcare is growing rapidly, but given its roots, most applications of these methods remain in the traditional areas of operations research including scheduling, transportation, and allocation of resources. In this article, we focus on 2 types of dynamic simulation models, DES and agent-based models, and discuss their potential for use in PM (see Box 1 and Table 2).
5
,, 24
, 25
, 26
, , 28
Box 1
Descriptive overview of discrete event simulation and agent-based modeling methods.
22
Tabled
1
| Discrete event simulation (DES) | Agent-based model (ABM) |
|---|---|
| DES is a simulation modeling method used to represent processes at an individual level where individuals may be subject to events, whether they be decisions or occurrences over time. DES captures individual-level heterogeneity and is used to characterize and analyze queuing processes and networks of queues where there is an emphasis on the utilization of resources. | ABM is a simulation modeling method used to represent individual objects called “agents” and describe their local behavior with local rules. Agents are social and interact with others and their environment, and they may learn and adapt themselves on the basis of experience. ABM is useful to discover patterns of emergence in dynamic and adaptive systems by using deductive and inductive reasoning. |
Table 2Description of simulation model characteristics.
| Discrete event simulation (DES) | Agent-based modeling (ABM) | |
|---|---|---|
| Type of problem | Operational, tactical | Strategic at the policy level (eg, to inform program implementation) Operational at the management level (eg, tactical at the level of logistics, such as scheduling) |
| Perspective | Process oriented, emphasis on detail complexity (top down) | Individual oriented, dynamic and detail complexity (bottom up) |
| Handling of time | Discrete | Discrete |
| Approach | Explanatory | Exploratory and explanatory |
| Basic building blocks | Entities, events, queues | Autonomous agents, decision rules |
| Data sources | Numerical with some judgmental elements | Broadly drawn: qualitative and quantitative |
| Unit of analysis | Queues, events | Decision rules, emergent behavior |
| Mathematical formulation | Mathematically described with logic operators | Mathematically described with logic operators and decision rules |
| Outputs | Point predictions, performance measures | Detailed and aggregate key indicators, understanding of emergence due to individual behavior, point predictions |
| Advantages |
|
|
| Disadvantages | Compared with traditional health economic models, DES models are data intensive and require more time to obtain data and data analysis to prepare model inputs compared with traditional health economic models; programming and calibration are usually time-consuming | Compared with traditional health economic models, ABM models are data intensive and require more time to obtain data and data analysis to prepare model inputs; programming and calibration are usually time-consuming |
∗ Adapted from Marshall et al
22
and Caro and Moler.24
Discrete Event Simulation
DES is the most commonly used dynamic simulation modeling approach in the healthcare context.
25
In DES models, the behavior of a system is captured in an ordered sequence of defined events (eg, a test is done, or a treatment is provided). Other events include points in time with respect to changes in health such as the detection or recurrence of a disease or delivery of services such as admission to a health facility and receipt of medical or surgical treatment. DES has the flexibility to map care pathways that incorporate different testing strategies and services that may affect uptake of treatments and downstream patient outcomes.20
There is a considerable body of literature on the application of DES in health to address a variety of healthcare issues.
5
,29
,30
A recent systematic review by Zhang31
identified 211 studies using DES as the main modeling technique in the context of healthcare delivery or public health scenarios (not specific to PM). The authors noticed a significant expansion of publications in this area especially after 2010. The studies were categorized into 4 main classes of applications of DES: health and care systems operations, disease progression modeling, screening modeling, and health behavior modeling. Zhang31
reported that most DES models (93%) belong to the first 2 categories (ie, health and care systems operations and disease progression modeling) and aimed to evaluate the effects of operational changes, health economic evaluations, and patient scheduling.In comparison with state-transition models that do not capture dynamic interactions in the delivery of care, DES is more suited to modeling complex systems by reflecting patient flows through the system.
24
For instance, DES models can incorporate attributes of individuals that can affect or even determine responses to events, including age, sex, health status, illness history, duration of disease, and other demographics. These attributes can also vary over time.Agent-Based Modeling
Agent-based modeling (ABM) is another dynamic simulation method for modeling dynamic, adaptive, and autonomous systems in which the agent (for example, the patient or doctor) serves as the entity of simulation.
28
The agents in an ABM model interact within an environment based on a set of decision rules describing the agent’s behavior and operationalized using mathematical logic operators.26
An agent’s behavior is typically nonlinear and dependent on previous interactions. For instance, the likelihood of seeing a doctor increases after contamination with a virus resulting in a fever. Agents can also be programmed to change health states based on an interaction with the health system, which implies we can accumulate costs and outcomes attached to an agent’s state as long as the simulation is running. ABM emphasizes agent–agent (for example, patient and care providers) and agent–environment (for example, patient and hospital clinic) interactions and is therefore well suited to examining patterns of health and behavior of populations over time, for example, modeling infectious disease outbreaks.20
,22
In summary, simulation models such as DES and ABM can enable decision makers to better understand the behavior of complex systems—characterized by these nonlinearities and spatial relationships among entities, multiple agents, feedback loops, and variables that evolve dynamically over time—and predict their response to changes with intended and unintended consequences.
Applications and Case Examples of Simulation Modeling for PM
In this final section, we present several examples to illustrate how complex genomic testing affects clinical pathways that would be better addressed in dynamic simulation modeling. These case examples represent a range of applications of simulation models that address some of the 10 challenges identified in the checklist, including clinical risk stratification (which may not be based on genetics) at a patient level, diagnostic performance of the test, combinations of tests, and different testing and treatment sequences and study-specific outcome measures.
Molecular Profiling to Inform Treatment Decisions in Patients With Cancer
One of the most prominent changes in the treatment landscape in cancer is the availability of complex genomic tests in combination with targeted treatments or immunotherapy. Nevertheless, the diagnostic component is usually not considered in health economic models.
32
For instance, women with early-stage breast cancer are faced with challenging treatment decisions determined by stage of disease and the use of neoadjuvant or adjuvant systemic therapy. Patients with overexpressed HER2 (HER2 positive) are eligible for anti-HER2 targeted therapies, such as trastuzumab.33
Likewise, testing for estrogen receptor will identify eligibility for hormone therapy. Although these tests, either through immunohistochemistry or fluorescence in situ hybridization, have been available for quite some time, health economic models have rarely considered the diagnostic performance of the assays,32
despite the differences in test accuracy between methods to assess HER2 status reported in a recently published meta-analysis.34
Likewise, clinical guidelines recommend determination of HER2, estrogen receptor, and progesterone receptor expression status on tissue biopsy and excision material after surgery. Recent studies have shown discordance of up to 15% for progesterone receptor between both samples, which affects optimal treatment decisions.Grazziotin LR, Dada BR, de la Rosa Jaimes C, Cheung WY, Marshall DA. Chromogenic and silver in situ hybridization for identification of HER 2 overexpression in breast cancer patients: a systematic review and meta-analysis [published online May 23, 2019]. Appl Immunohistochem Mol Morphol. https://doi.org/10.1097/PAI.0000000000000773.
35
Although these examples illustrate only how a simple immunohistochemistry or fluorescence in situ hybridization test can change treatment pathways, it will soon become more complex in terms of building the clinical and health economic evidence base. Several international studies have tested complex genomic profiling (NGS panels, whole exome sequencing, and whole genome sequencing) in metastatic cancers to direct treatment. Although earlier work on HER2 was for only 1 specific treatment, complex gene panels (140 genes) used in lung cancer can guide decisions for a dozen of targeted treatments, including ALK-TKI and EGRF-TKIs, and immunotherapy. The recently published PROFILER study presented the evidence for using complex genomic testing using 2 gene panels in nearly 2600 patients in France, showing that 27% were recommended a molecularly based therapy, whereas only 6% actually received a molecularly based therapy.
36
Modeling the health economic impact in this study is far more complex and may require more granularity than possible even through simulation modeling.Several examples using DES in personalized oncology have been published, including comparisons to state-transition models in case of colorectal cancer.
37
Although the previous study was a direct head-to-head economic evaluation of 2 treatment strategies based on a clinical trial, DES has also been used to model the health economic impact of using circulating tumor cells to monitor progressive metastatic castration resistant prostate cancer. The model accounted for 2 lines of treatment in which several biomarkers could be used to optimize treatment strategies.38
Simulation modeling can also be useful in situations in which there is decisional conflict, such as when guidelines recommend adjuvant chemotherapy but chances of recurrence are estimated to be low. In this scenario, personalized prediction tools such as gene expression profiling (eg, OncotypeDX [Genomic Health Inc., Redwood City, CA] or Mammaprint [Agendia Inc., Amsterdam, the Netherlands]) can calculate the likelihood of cancer recurrence and can help to risk stratify women who may not benefit from chemotherapy, sparing them from associated toxicity. Jahn et al
39
developed a DES model to evaluate the cost-effectiveness of OncotypeDx used both with and without the Adjuvant! Online (AO) (Peter M Ravdin, University of Texas, Austin, TX) score, a web-based decision aid to guide decisions about the use of adjuvant chemotherapy. The combination of OncotypeDx and AO results resulted in 12 distinct risk groups of patients. A DES model was selected for this analysis for multiple reasons: (1) individual patient pathways were influenced by multiple characteristics and test results, (2) there were time-dependent functional relationships, (3) OncotypeDx and AO could be modeled as companion tests, and (4) the desired to track and report individual patient pathways. In another publication using the same model, the authors moved even further toward personalized treatment strategies by evaluating the cost-effectiveness of adjuvant chemotherapy for women in specific risk groups according to the joint results of gene expression profiling and AO. Using the DES model, the authors developed a flexible tool for the evaluation of several test-treatment strategies.40
PM Treatment Options in Chronic Obstructive Pulmonary Disease
As noted earlier, state transition models have a limited ability to deal with patient-specific pathways of diagnosis and treatment. The article by Hoogendorn et al
41
explores the suitability of models using more traditional approaches to evaluate PM treatment options for chronic obstructive pulmonary disease (COPD). The authors assessed COPD models to determine which types of patient heterogeneity (risk stratification by patient characteristics) were included. The authors concluded that all of the currently available models are capable of running simulations for different age and COPD severity classes, and most models also have the ability to run analyses separately by sex and smoking status. Nevertheless, the validity of subgroup analyses within the models was questioned because important input parameters were not specified by sex, age, or smoking status. Thus, the challenge for the evaluation of PM treatment options in COPD is that treatment is more likely to be personalized on the basis of clinical parameters rather than age, sex, and smoking status, such that the existing models are likely of limited usefulness. Information on the effectiveness and cost-effectiveness of treatment options for these subgroups is needed to guide clinical guideline development and decisions for reimbursement. It was recommended that future models should include all clinical patient characteristics considered to influence disease severity, prognosis, and treatment response in COPD.42
To address PM in COPD, for which treatment is tailored to specific clinical phenotypes, such as patients with frequent exacerbations, Hoogendoorn et al
42
subsequently developed a patient-level DES simulation model for COPD that was able to estimate the incremental costs and effects of different treatments for many subgroups of patients. The model included 14 patient characteristics (eg, current smoking status, the number of pack-years smoked, history of heart failure, presence of asthma, bronchodilator responsiveness, diabetes, history of depression), 10 intermediate outcomes (eg, exacerbations, pneumonias, lung function), and 3 final outcomes (death, quality-adjusted life-years, and costs). This new model was unique because none of the previously published models had modeled time to events simultaneously with changes in clinical variables for individual patients.Other Considerations
Although there are some distinct advantages of simulation modeling over traditional state-transition models for the economic evaluation of PM, this approach can also present challenges for both the analysts who construct these models (eg, data requirements, computational requirements, requirements for modeling skills, model reproducibility) and the decision makers who use these models (eg, additional expertise required by evidence review groups) (Table 2).
The additional complexity that can be captured using these modeling methods come at the cost of more complex structures and level of detail.
24
Consequently, analysts with specialized modeling and biostatistical skill sets are required to design, develop, program, and conduct analyses using simulation modeling software. These complex models require more data, both in terms of the number of parameters that need to be estimated and the volume of observations needed to inform these estimates. Simulation models require a great deal of data that may not be available in the detail required to populate the model, especially for early stage analyses of PM interventions.This complexity can also present challenges for reviewers and users of these models. The complexity and transparency of cost-effectiveness modeling have been long-standing concerns because decisions about the use of drugs and other therapeutic interventions were first influenced by economic analyses using modeling approaches.
43
,44
More recently, transparency in modeling has been promoted through reporting standards, reference models, collaboration, model registration, peer review, and open-source modeling.45
With the additional complexity associated with dynamic simulation models, further challenges in communicating the structure, assumptions, and outcomes from these models may be encountered. The uptake of such methods may therefore be lower than is optimal, and it may be more difficult to comprehend, assess the validity, and interpret the findings from these models.Summary
In this article, we discuss how simulation modeling methods such as DES may be better suited than traditional Markov or state-transition cohort models to address the complexity and specific challenges of economic evaluation of PM interventions. Among the core advantages of simulation models are the ability for patient-level analyses of care pathways and the ability to deal with system complexity of multiple tests, diagnostic performance and testing, and treatment sequences that present challenges for PM. Although the use of simulation modeling in health is growing in general, there remain few examples in PM to date.
Another modeling approach that we have not addressed in this article and that is potentially relevant for PM is constrained optimization. Constrained optimization is a mathematical approach to finding the best solution to a problem, subject to constraints.
46
Optimization methods may be useful for designing optimal treatment pathways that are relevant in the context of PM, where a specific treatment may be used in a specific subset of patients because of their genotypic or clinical phenotypic profile. Optimization modeling approaches are specifically suited to address constraints in the system (most commonly, fixed budgets and time), which have resource allocation implications for decision makers.46
,47
An example of constrained optimization is the work by the Alliance for Paired Donation to make improved matches between kidney disease sufferers and potential kidney donors.48
The Alliance developed and implemented an innovative technique called nonsimultaneous extended altruistic donor chains that permits better-optimized matching of potential donors to patients, greatly increasing the number of possible transplants.Although there are benefits from applying dynamic simulation modeling methods for PM interventions, implementing these models to support policy decisions may be challenging. Dynamic simulation models are not a “one size fits all” solution, and continuing research, education, and testing of these methods is required to understand when these methods should be applied. We offer 3 suggestions regarding the application of simulation modeling in economic evaluation of PM interventions:
- 1.Simulation modeling methods should be considered part of the tool kit for economic evaluations in PM given the need to model a cascade of testing and treatment sequences.
- 2.Although simulation modeling may be an appropriate modeling approach for economic evaluation in PM, in general, models should aim to represent the decision problem and the decision context in which the results will be interpreted and applied in as simple a manner as possible.
- 3.Providing sufficient transparency to achieve understanding by decision makers and reflecting the robustness of the model may be an even greater challenge for simulation modeling than other types of models. Modelers should explicitly document the rationale for applying simulation modeling, the modeling assumptions, and the strength of the data used to populate the model, and they should conduct an appropriate exploration of the uncertainty around the model.
Acknowledgments
Source of financial support: This study was funded in part by grants from the Canadian Institutes for Health Research (381280), Genome Canada, Arthritis Society, ZonMw (Netherlands Organisation for Health Research and Development), and ReumaNederlands, through the Understanding Childhood Arthritis Network Canada-Netherlands (UCAN CAN-DU) project, the National Human Genome Research Institute (U01 HG009599) and from the National Cancer Institutes (R01 CA221870). D.A.M. is supported by the Arthur J. E. Child Chair in Rheumatology and a Canada Research Chair in Health Systems and Services Research (2008-2018). L.R.G. is funded as a doctoral trainee by the Arthritis Society (TGP-18-0244) and UCAN CAN-DU project. J.B. and S.W. were partly funded by the National Institute for Health Research (NIHR) and Oxford Biomedical Research Centre (BRC). K.P. receives consulting fees from Illumina. M.I. received travel funding by Illumina to attend the 2019 International Summit on Population Health Genomics. D.A.R., K.P., J.B., S.W., and D.A.M. received travel funding by Illumina to attend the Global Economics and Evaluation of Clinical Genomics Sequencing Working Group (GEECS) meeting.
References
- Recent developments in decision-analytic modelling for economic evaluation.Pharmacoeconomics. 2006; 24: 1043-1053
- Health care system information sharing: a step toward better health globally.Value Health Reg Issues. 2012; 1: 118-120
- Guidelines for the Economic Evaluation of Health Technologies: Canada.4th ed. Canadian Agency for Drugs and Technologies in Health, Ottawa, Canada2017
- Methods for the Economic Evaluation of Health Care Programmes.4th ed. Oxford University Press, Oxford, UK2015
- Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–4.Value Health. 2012; 15: 821-827
- State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–3.Value Health. 2012; 15: 812-820
- Health technology assessment and personalized medicine: are economic evaluation guidelines sufficient to support decision making?.Int J Technol Assess Health Care. 2014; 30: 179-187
- A systematic review and checklist presenting the main challenges for health economic modeling in personalized medicine: towards implementing patient-level models.Expert Rev Pharmacoecon Outcomes Res. 2017; 17: 17-25
- Methodological issues in assessing the economic value of next-generation sequencing tests: many challenges and not enough solutions.Value Health. 2018; 21: 1033-1042
- Priorities for personalized medicine.(Accessed September 4, 2019)
- Precision Medicine: From Science To Value.Health Aff (Millwood). 2018; 37: 694-701
- Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–6.Value Health. 2012; 15: 835-842
- Optimizing precision medicine for public health.Front Public Health. 2019; 7: 42
- Accounting for parameter uncertainty in the definition of parametric distributions used to describe individual patient variation in health economic models.BMC Med Res Methodol. 2017; 17: 170
- Valuation of health and nonhealth outcomes from next-generation sequencing: approaches, challenges, and solutions.Value Health. 2018; 21: 1043-1047
- Guide to the methods of technology appraisal.(Accessed September 4, 2019)
- Modeling rheumatoid arthritis using different techniques—a review of model construction and results.Health Econ Rev. 2014; 4: 18
- Breaking the addiction to technology adoption.Health Econ. 2014; 23: 379-383
- CADTH 2018–2021 strategic plan—transforming how we manage health technologies in support of better health, better patient experience, and better value.https://cadth.ca/sites/default/files/corporate/planning_documents/CADTH_2018-2021_Strategic_Plan.pdf(Accessed October 27, 2019)Date: 2018
- Applying dynamic simulation modeling methods in health care delivery research-the SIMULATE checklist: report of the ISPOR simulation modeling emerging good practices task force.Value Health. 2015; 18: 5-16
- Integrating systems engineering practice with health-care delivery.Health Systems. 2014; 3: 159-164
- Selecting a dynamic simulation modeling method for health care delivery research-part 2: report of the ISPOR Dynamic Simulation Modeling Emerging Good Practices Task Force.Value Health. 2015; 18: 147-160
- Industrial dynamics.J Oper Res Soc. 1997; 48: 1037-1041
- Advantages and disadvantages of discrete-event simulation for health economic analyses.Expert Rev Pharmacoecon Outcomes Res. 2016; 16: 327-329
- Discrete event simulation: the preferred technique for health economic evaluations?.Value Health. 2010; 13: 1056-1060
- Agent-based modeling: methods and techniques for simulating human systems.Proc Natl Acad Sci U S A. 2002; 99: 7280-7287
- Models of segregation.Am Econ Rev. 1969; 59: 488-493
- Economic evaluations with agent-based modelling: an introduction.Pharmacoeconomics. 2015; 33: 423-433
- Sequential treatment approaches in the management of BRAF wild-type advanced melanoma: a cost-effectiveness analysis.Immunotherapy. 2018; 10: 1241-1252
- Improving patient access to chemotherapy treatment at Duke Cancer Institute.INFORMS Journal on Applied Analytics. 2013; 43: 449-461
- Application of discrete event simulation in health care: a systematic review.BMC Health Serv Res. 2018; 18: 687
- Do economic evaluations of targeted therapy provide support for decision makers?.J Oncol Pract. 2011; 7: 36s-45s
- Trastuzumab after adjuvant chemotherapy in HER2-positive breast cancer.N Engl J Med. 2005; 353: 1659-1672
Grazziotin LR, Dada BR, de la Rosa Jaimes C, Cheung WY, Marshall DA. Chromogenic and silver in situ hybridization for identification of HER 2 overexpression in breast cancer patients: a systematic review and meta-analysis [published online May 23, 2019]. Appl Immunohistochem Mol Morphol. https://doi.org/10.1097/PAI.0000000000000773.
- Real-world data on discordance between estrogen, progesterone, and HER2 receptor expression on diagnostic tumor biopsy versus tumor resection material.Breast Cancer Res Treat. 2019; 175: 451-458
- Molecular screening program to select molecular-based recommended therapies for metastatic cancer patients: analysis from the ProfiLER trial.Ann Oncol. 2019; 30: 757-765
- Matching the model with the evidence: comparing discrete event simulation and state-transition modeling for time-to-event predictions in a cost-effectiveness analysis of treatment in metastatic colorectal cancer patients.Cancer Epidemiol. 2018; 57: 60-67
- Comparison of timed automata with discrete event simulation for modeling of biomarker-based treatment decisions: an illustration for metastatic castration-resistant prostate cancer.Value Health. 2017; 20: 1411-1419
- Cost effectiveness of personalized treatment in women with early breast cancer: the application of OncotypeDX and Adjuvant! Online to guide adjuvant chemotherapy in Austria.SpringerPlus. 2015; 4: 752
- Personalized treatment of women with early breast cancer: a risk-group specific cost-effectiveness analysis of adjuvant chemotherapy accounting for companion prognostic tests OncotypeDX and Adjuvant!Online.BMC Cancer. 2017; 17: 685
- Patient heterogeneity in health economic decision models for chronic obstructive pulmonary disease: are current models suitable to evaluate personalized medicine?.Value Health. 2016; 19: 800-810
- Broadening the perspective of cost-effectiveness modeling in chronic obstructive pulmonary disease: a new patient-level simulation model suitable to evaluate stratified medicine.Value Health. 2019; 22: 313-321
- The journal’s policy on cost-effectiveness analyses.N Engl J Med. 1994; 331: 669-670
- Modelling in economic evaluation: an unavoidable fact of life.Health Econ. 1997; 6: 217-227
- Transparency in decision modelling: what, why, who and how?.Pharmacoeconomics. 2019; 37: 1355-1369
- Constrained optimization methods in health services research—an introduction: report 1 of the ISPOR Optimization Methods Emerging Good Practices Task Force.Value Health. 2017; 20: 310-319
- Resource modelling: the missing piece of the HTA jigsaw?.Pharmacoeconomics. 2015; 33: 193-203
- Kidney exchange and the alliance for paired donation: operations research changes the way kidneys are transplanted.INFORMS Journal on Applied Analytics. 2015; 45: 26-42
Article info
Publication history
Published online: March 26, 2020
Identification
Copyright
© 2020 ISPOR-The Professional Society for Health Economics and Outcomes Research. Published by Elsevier Inc.
User license
Elsevier user license | How you can reuse 
Elsevier's open access license policy
Elsevier user license
Permitted
For non-commercial purposes:
- Read, print & download
- Text & data mine
- Translate the article
Not Permitted
- Reuse portions or extracts from the article in other works
- Redistribute or republish the final article
- Sell or re-use for commercial purposes
Elsevier's open access license policy
