Advertisement

The Prime Diabetes Model: Novel Methods for Estimating Long-Term Clinical and Cost Outcomes in Type 1 Diabetes Mellitus

Open ArchivePublished:May 04, 2017DOI:https://doi.org/10.1016/j.jval.2016.12.001

      Abstract

      Background

      Recent publications describing long-term follow-up from landmark trials and diabetes registries represent an opportunity to revisit modeling options in type 1 diabetes mellitus (T1DM).

      Objectives

      To develop a new product-independent model capable of predicting long-term clinical and cost outcomes.

      Methods

      After a systematic literature review to identify clinical trial and registry data, a model was developed (the PRIME Diabetes Model) to simulate T1DM progression and complication onset. The model runs as a patient-level simulation, making use of covariance matrices for cohort generation and risk factor progression, and simulating myocardial infarction, stroke, angina, heart failure, nephropathy, retinopathy, macular edema, neuropathy, amputation, hypoglycemia, ketoacidosis, mortality, and risk factor evolution. Several approaches novel to T1DM modeling were used, including patient characteristics and risk factor covariance, a glycated hemoglobin progression model derived from patient-level data, and model averaging approaches to evaluate complication risk.

      Results

      Validation analyses comparing modeled outcomes with published studies demonstrated that the PRIME Diabetes Model projects long-term patient outcomes consistent with those reported for a number of long-term studies. Macrovascular end points were reliably reproduced across five different populations and microvascular complication risk was accurately predicted on the basis of comparisons with landmark studies and published registry data.

      Conclusions

      The PRIME Diabetes Model is product-independent, available online, and has been developed in line with good practice guidelines. Validation has indicated that outcomes from long-term studies can be reliably reproduced. The model offers new approaches to long-standing challenges in diabetes modeling and may become a valuable tool for informing health care policy.

      Keywords

      Introduction

      Type 1 diabetes mellitus (T1DM) is a serious, chronic condition that is associated with significant morbidity and mortality, and its incidence is increasing annually [
      • Daneman D.
      Type 1 diabetes.
      ,
      • Lung T.W.C.
      • Hayes A.J.
      • Herman W.H.
      • et al.
      A meta-analysis of the relative risk of mortality for type 1 diabetes patients compared to the general population: exploring temporal changes in relative mortality.
      ,
      • Patterson C.C.
      • Dahlquist G.G.
      • Gyürüs E.
      • et al.
      EuroDiab Study Group
      Incidence trends for childhood type 1 diabetes in Europe during 1989–2003 and predicted new cases 2005–2020—a multicentre prospective registration study.
      ]. Daily management of the condition is demanding, placing a burden on patients, their caregivers, and health care providers, requiring attention to insulin administration, blood glucose monitoring, meal planning, and screening for comorbid conditions and diabetes-related complications [
      • Daneman D.
      Type 1 diabetes.
      ]. Landmark studies such as the Diabetes Control and Complications Trial (DCCT), its follow-up study—the Epidemiology of Diabetes and Its Complications (EDIC) trial, and the Wisconsin Epidemiologic Study of Diabetic Retinopathy (WESDR) have provided evidence that optimizing therapy, particularly in terms of glycemic control, is key to minimizing the risk of long-term microvascular and macrovascular complications [
      • Nathan D.M.
      DCCT/EDIC Research Group
      The diabetes control and complications trial/epidemiology of diabetes interventions and complications study at 30 years: overview.
      ,
      • Klein R.
      • Klein B.E.
      Relation of glycemic control to diabetic complications and health outcomes.
      ]. There have been several advances in insulin therapy in the last 20 years responding to this need. The first insulin analog, designed to overcome the problems of poor stability and erratic absorption profile of human insulin formulations, was launched in 1996 (insulin lispro) and has been followed by several other short-acting and long-acting insulin analogs designed for prandial and bolus administration in recent years [
      • Aathira R.
      • Jain V.
      Advances in management of type 1 diabetes mellitus.
      ].
      As new therapies for T1DM become available, tools to evaluate their impact on clinical outcomes and costs are needed to assist decision makers in the efficient allocation of health care resources. Economic evaluation can, however, be challenging in T1DM because it is a chronic condition and it is common for complications to take several decades to develop. Health economic models can play an important role in addressing this challenge, provided they are accepted by health care authorities, which requires them to be transparent, on the basis of robust clinical data, and externally validated in line with published guidelines [
      • Eddy D.M.
      • Hollingworth W.
      • Caro J.J.
      • et al.
      ISPOR-SMDM Modeling Good Research Practices Task Force
      Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7.
      ,
      American Diabetes Association Consensus Panel
      Guidelines for computer modeling of diabetes and its complications.
      ].
      Although advances have been made with new models and validation studies in type 2 diabetes mellitus (T2DM), progress has been less notable in the T1DM modeling space [
      • Palmer A.J.
      • Clarke P.
      • Gray A.
      • et al.
      Mount Hood 5 Modeling Group
      Computer modeling of diabetes and its complications: a report on the Fifth Mount Hood Challenge Meeting.
      ]. Historically, health economic models of T1DM have been adapted from their T2DM counterparts and have relied on data from mixed and/or T2DM populations (as well as T1DM studies) to make long-term projections of outcomes [
      Mount Hood 4 Modeling Group
      Computer modeling of diabetes and its complications: a report on the Fourth Mount Hood Challenge Meeting.
      ]. Literature review has revealed numerous publications from landmark trials and registries of T1DM in recent years. These studies represent an opportunity to revisit health economic modeling in T1DM. Our objective was, therefore, to develop a product-independent, computer simulation model of T1DM on the basis of timely data and using new approaches to long-standing challenges in T1DM modeling to provide a resource that can support decision making and inform health care policy for various users and audiences.

      Methods

       Literature Review

      Development of the PRIME Diabetes Model was based on a comprehensive review of relevant literature and expert medical input. To identify published data that could inform diabetes model development, a literature review was conducted to find publications on existing models and clinical results relevant to complications, disease progression, and mortality risk. The search strategy was designed on the basis of high-level Medical Subject Heading terms and keywords to identify trials in diabetes, systematic reviews, meta-analyses, reviews of clinical trials, evidence-based medicine, consensus development conferences, and guidelines. Searches were limited to publications in English since January 1, 2004, and publications relating to animals were excluded. After the exclusion of duplicates, a total of 12,360 unique hits remained, the titles and abstracts of which were then screened to exclude publications that were not related to T1DM that described studies with duration of follow-up of less than 1 year, or that described studies with fewer than 100 patients (Appendix 1). The review identified a total of 58 publications in T1DM. Manual searches were then performed in January 2014 on the basis of the reference lists from these publications with no time limits applied, and a further 96 articles were identified for review. On the basis of this literature review, availability of patient-level data, and two advisory board meetings with data, clinical, and modeling experts, the model concept was finalized and the model, was developed.

       Model Structure and Functionality

      The model is programmed in Java Standard Edition 8 (Oracle Corp., Redwood City, CA), relying on the Apache Commons Mathematics Library for random number generation and distribution sampling (The Apache Software Foundation, Forest Hill, MD). It is architected as a patient-level simulation in which a simulated cohort of patients is generated by the model (for each of two simulation arms) on the basis of user-defined parameters and a covariance matrix derived from the DCCT data (Fig. 1). Each patient in the generated cohort is first assigned a treatment specific to the simulation arm and is then passed between a series of model “controllers,” which have various roles, including risk factor progression, evaluation of diabetes complication or adverse event risk, and calculation of per-cycle costs and quality-adjusted life expectancy (QALE). The order in which patients are passed between the complication and adverse event controllers is randomized for each patient in each 1-year model cycle. Each controller evaluates every living patient in every cycle and, in the case of the complication and adverse event controllers, has the opportunity to trigger and record an event on the basis of sampling from a uniform probability distribution and thereby creating a random walk through the model. In evaluating the risk of a given diabetes complication or adverse event, controllers have access to all patient characteristics, including risk factors over time (e.g., age or glycated hemoglobin [HbA1c]) and history of diabetes complications. Details on each of the complication controllers are provided in Appendix 2.
      Fig. 1
      Fig. 1Schematic diagram of the PRIME Diabetes Model. Interactions between complication controllers are indicated by red arrows. X, risk of mortality is associated with this complication controller; ▲, SBP is a direct risk factor; •, HbA1c is a direct risk factor; ■, BMI is a direct risk factor. BMI, body mass index; HbA1c, glycated hemoglobin; SBP, systolic blood pressure.
      The model is accessible over the Internet and has a Web interface written in JavaScript and HTML5 that allows a two-arm simulation to be configured using an easy-to-use drag-and-drop interface. The model is free for academic organizations and health technology assessment bodies to use and can be accessed at www.prime-diabetes-model.com. The model reports costs, life expectancy, QALE, cumulative incidence of all modeled diabetes complications, mean hypoglycemia and ketoacidosis rates in each simulation arm, along with differences in cost, life expectancy, and QALE outcomes between simulation arms and the incremental cost-effectiveness ratio. To run simulations, the model creates a set, or cohort, of individual patients in which each patient is associated with a set of characteristics that define them (e.g., age, age at diagnosis of T1DM, sex, body mass index [BMI], HbA1c, systolic blood pressure [SBP], total cholesterol, high-density lipoprotein cholesterol, and smoking status). Patients’ characteristics can be based directly on patient-level data (e.g., from a clinical trial) or generated by sampling from user-defined distributions to reflect the characteristics of a given cohort. An option to include covariance of patient characteristics (which is unique among T1DM models), based on a covariance matrix derived from individual patient profiles in the DCCT, is available to create simulated cohorts that are more representative of real-life cohorts. During the course of a simulation, progression of patient characteristics (risk factors, including HbA1c) over time is modeled in line with data from published large-scale studies. Covariance also applies to changes in patient characteristics; for example, a decrease in total cholesterol will increase the likelihood of observing a decrease in SBP on the basis of relationships observed in the DCCT data set. During simulations, treatment-specific effects on patient characteristics of HbA1c, SBP, BMI, high-density lipoprotein cholesterol, and total cholesterol, as well as adverse event rates for hypoglycemia and ketoacidosis, can be applied. As a cohort, the mean impact on patient characteristics will match that specified by the user; each patient, however, will experience a different effect sampled from a normal distribution. As with assignment and progression of patient characteristics, individual treatment effects can covary with other treatment effects and with patient characteristics. Nondiabetes medications, including antihypertensives, antithrombotics, and cholesterol drugs, can also influence the value and progression of patient characteristics. In combination, the properties of simulated patients, the impact of covariance, and the changes in characteristics over time result in the model population in the PRIME Diabetes Model progressing in a manner analogous to a clinical trial. The model is user-editable in terms of costs and utility weights used for any given set of simulations. Direct costs to the health care payer included intervention costs (those associated with therapy, consumables such as needles and/or syringes, self-monitoring of blood glucose, visits, and/or training) and management costs associated with diabetes-related complications. Default utility weights are based on a systematic literature review with QALE estimated using either a multiplicative approach or an additive approach (for simulated patients with multiple complications, the lowest utility weight of all chronic complications is used along with utility decrements for acute events to estimate utility for each year).
      The PRIME Diabetes Model includes support for conducting probabilistic sensitivity analysis (PSA). When PSA is implemented, in addition to sampling from distributions around all baseline cohort characteristics and treatment effects, the model samples from distributions around an additional 397 internal model coefficients and parameters, including beta coefficients, hazard ratios, odds ratios, transition probabilities, and cost and quality-of-life inputs. Distributions were selected on the basis of known approximations of distribution shapes (e.g., the lognormal distribution of ratios) and assumed data bounds (e.g., the impossibility of negative complication costs or positive complication quality-of-life utilities). All distributions are implemented as abstract real distributions using the Apache Commons AbstractRealDistribution Java interface, ultimately allowing any distribution of real numbers to be used for each sampled parameter. When PSA is implemented, the PRIME Diabetes Model reports results based on a nonparametric bootstrapping approach, in which a user-defined number of population samples of user-defined size are drawn from the full patient data set at the end of a simulation (with replacement). The population mean and confidence intervals are then calculated by rerunning the cost and quality-of-life estimators on each sampled population and generating a set of descriptive statistics.

       Validation Analysis

      To evaluate model performance relative to real-life data from T1DM populations, model validation was performed in line with best practice guidelines [
      • Eddy D.M.
      • Hollingworth W.
      • Caro J.J.
      • et al.
      ISPOR-SMDM Modeling Good Research Practices Task Force
      Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7.
      ]. First, face validity of the model was established by review by clinical and diabetes modeling experts as well as by presentation of the model structure, data sources, assumptions, and model outputs at two full-day technical advisory board meetings. Internal validity was addressed by an external third-party performing a code audit of the PRIME Diabetes Model. The model source code (Java) was delivered to an independent consultancy (HealthMetrics Outcomes Research, Bonita Springs, FL). The program code was reviewed for syntax errors and tested in a separate software environment, and all input data were checked against source references. The process also included using null and extreme input values to test whether the expected outputs were produced, and tests of replication using equivalent end-point values were performed to identify and remedy any errors in the model code.
      The model includes inbuilt functionality to run a set of validation simulations automatically against sets of published clinical data. This feature may be unique among current models of T1DM and has been designed to offer a quick and convenient approach to validation after model updates and/or incorporation of new data or features into the model in years ahead. The same functionality has been used to perform validation simulations comparing model predictions with outcomes observed in the studies used to build the model (referred to as internal validation) and with outcomes observed in studies that were not used to build the model (referred to as external validation). Studies identified by the literature review were selected on the basis of those that reported long-term cumulative incidence for a complication using similar end-point definitions to the model, that included a large cohort of individuals with T1DM, and in which sufficient information on baseline characteristics and treatment regimens were reported to allow predictions for that population to be evaluated using the model. Because of space limitations, only selected validation analyses are shown in the present publication.
      After the validation simulations were run in the model, two approaches were used to evaluate the similarities of modeled outcomes with published clinical data, namely, the root mean squared deviation (RMSD) and the area under the curve (AUC). RMSD is a measure of the differences between two values, generally the value predicted by the model and the actual value observed:
      RMSD=n=1NPnAnN,


      where N is the number of predictions, Pn is the predicted value of prediction n, and An is the actual observed value of n. Given the longitudinal nature of the model output data, it is important to consider the performance of the model over time. For a particular end point, the AUC of the cumulative incidence curve is related to the total number of event-years (the number of patient-years over the model time horizon that the end point is present). The AUC is, therefore, an estimate of event exposure rather than a point estimate. Because of limitations in the number of data points available for validation and the fact that the actual cumulative incidence curve is generally unknown, it was assumed that known data points are connected by straight lines (linear increase in cumulative incidence) and the AUC is calculated using the trapezoidal rule:
      AUC=n=1N([(CIn+CIn-12)×(tn1tn)]),


      where N is the number of points on the cumulative incidence curve and CIn is the cumulative incidence at time point tn. Differences in AUC were standardized by setting the AUC of the actual cumulative incidence curve to 1, and the difference was then annualized. This provided a percentage difference in exposure per model-year.

      Results

      Validation across all diabetes-related complications indicated that the model is capable of closely matching real-life clinical outcomes. For internal validations (against published data used to estimate complication risk in the model), all RMSD values were less than 3%, suggesting a close match between modeled and published data. External validations (against data sources that were not used in the development of the model) were associated with higher RMSDs on occasions. Nevertheless, all validation analyses performed to date have been associated with annualized AUC differences of less than 1% between published and modeled data, indicating that modeled and published curves correlate closely.
      To assess model performance in the prediction of cardiovascular disease (CVD) risk, the model was used to evaluate the cumulative incidence of first CVD events (myocardial infarction, stroke, and unstable angina) in the five populations from which risk estimates were derived for the model averaging approach used in the PRIME Diabetes Model (Swedish National Diabetes Register [SNDR], EuroDiab, FinnDiane Registry, the WESDR, and the DCCT). Reproducing the cohort characteristics from each of the studies and running simulations with the model to mimic the trial interventions or treatments showed that the model can accurately predict CVD events in all five populations (Fig. 2). Predictions for all five derivation populations (with two different end points assessed for the DCCT population) were associated with P values of 0.99 (indicating a close match). Four of the six RMSD values were less than 1% and the remaining two were less than 3%, and all six annualized differences in standardized AUC were less than 1%.
      Fig. 2
      Fig. 2Modeled and actual cumulative incidence of first CVD events in multiple patient populations: (A) SNDR; (B) EuroDiab; (C) FinnDiane Registry; (D) WESDR; (E) DCCT (myocardial infarction and stroke only); and (F) DCCT. Each graph shows the cumulative incidence of first events (myocardial infarction, angina, and stroke [with the exception of subpart (E), in which only myocardial infarction and stroke are shown]) on the y-axis, plotted against time in years on the x-axis. CVD, cardiovascular disease; DCCT, Diabetes Control and Complications Trial; SNDR, Swedish National Diabetes Register; WESDR, Wisconsin Epidemiologic Study of Diabetic Retinopathy.
      Model outcomes for renal disease have been validated against data from the DCCT and the EDIC trial published by de Boer et al. [
      • de Boer I.H.
      • Rue T.C.
      • Cleary P.A.
      • et al.
      Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications Study Research Group
      Long-term renal outcomes of patients with type 1 diabetes mellitus and microalbuminuria: an analysis of the Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications cohort.
      ]. The model was used to recreate outcomes for a group of patients with persistent microalbuminuria at baseline in the DCCT (n = 325) with more than 20 years of follow-up (Fig. 3). The analysis was performed to test the model in a group of higher risk patients than those in data set used to estimate risk in the development of the model. The RMSD for the prevalence of microalbuminuria was 8.5% and for overt nephropathy it was 11.8%. Standardized differences in annualized AUC values were 0.1% for microalbuminuria and 0.6% for overt nephropathy.
      Fig. 3
      Fig. 3Comparison of nephropathy outcomes between DCCT data and the PRIME Diabetes Model. PRIME Diabetes Model results are presented as an area graph by year in the background. Data taken from de Boer et al.
      [
      • de Boer I.H.
      • Rue T.C.
      • Cleary P.A.
      • et al.
      Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications Study Research Group
      Long-term renal outcomes of patients with type 1 diabetes mellitus and microalbuminuria: an analysis of the Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications cohort.
      ]
      are overlaid as a stacked bar graph with data in 2-y blocks. Micro, microalbuminuria; normal, normal renal status; overt, overt nephropathy.
      An external validation of retinopathy outcomes showed that the model provided accurate predictions of DCCT/EDIC data over long-term follow-up (Fig. 4). The model was used to simulate outcomes for the DCCT intensive treatment cohort over 30 years and compared with published data on the cumulative incidence of proliferative diabetic retinopathy (PDR) or worse [
      • Nathan D.M.
      • Zinman B.
      • Cleary P.A.
      • et al.
      Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications (DCCT/EDIC) Research Group
      Modern-day clinical course of type 1 diabetes mellitus after 30 years’ duration: the diabetes control and complications trial/epidemiology of diabetes interventions and complications and Pittsburgh epidemiology of diabetes complications experience (1983–2005).
      ]. The RMSD between the model outcomes and the published data was 2.8% and the annualized difference in standardized AUC was 0.7%. Modeled outcomes showed a close match to the published follow-up data over 25 years with slight divergence noted in the last 5 years of the analysis. Validation analysis against published WESDR data also showed a close match to published outcomes, with an RMSD of 3.8% for remaining retinopathy-free over 25 years and an annualized AUC difference of 0.7% (data not shown).
      Fig. 4
      Fig. 4Performance of the PRIME Diabetes Model in predicting retinopathy outcomes (PDR or worse) in the DCCT intensive treatment cohort. External validation of the PRIME Diabetes Model (blue line) vs. the DCCT intensive treatment arm (black dots) of the cumulative incidence of PDR or worse. DCCT, Diabetes Control and Complications Trial; PDR, proliferative diabetic retinopathy.
      Validation of lower extremity amputation was performed separately for male and female cohorts to align with published data from the SNDR [
      • Jonasson J.M.
      • Ye W.
      • Sparén P.
      • et al.
      Risks of nontraumatic lower-extremity amputations in patients with type 1 diabetes: a population-based cohort study in Sweden.
      ]. Simulations with the model over a long-term time horizon showed a close match to the published trial data in this internal validation (Fig. 5). The RMSDs between the model and the published data were 0.8% for males and 0.6% for females. Analysis of AUC values showed that annualized difference for both males and females was 0.03%, indicating a close match between the model and the registry data.
      Fig. 5
      Fig. 5Performance of the PRIME Diabetes Model in predicting amputation outcomes in the SNDR. Internal validation of the model vs. SNDR data for lower extremity amputation. Cumulative probabilities from the model were indexed by patient age to generate outcomes analogous to the published registry data. Model outcomes are shown in blue (male) and red (female), and broken (male) and solid (female) gray lines indicate the registry data. CEA, cost-effectiveness analysis; SNDR, Swedish National Diabetes Register.

      Discussion

      The PRIME Diabetes Model has been developed on the basis of a systematic review of the literature on T1DM and its complications. The model has been developed in line with the Modeling Good Research Practices from the International Society for Pharmacoeconomics and Outcomes Research, including the conceptualization phase (incorporating input from external clinical and modeling experts), the development phase (incorporating best practices for individual-level state-transition models), as well as internal and external validation [
      • Caro J.J.
      • Briggs A.H.
      • Siebert U.
      • Kuntz K.M.
      ISPOR-SMDM Modeling Good Research Practices Task Force
      Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1.
      ]. The model incorporates controllers for nine complications of T1DM, three treatment-related adverse events (ketoacidosis, nonsevere and severe hypoglycemia), and competing mortality. The risk of complication onset is informed exclusively by data taken from populations with T1DM, which may be unique among health economic models of the disease, with recent data used when available and appropriate, including several publications from 2014 and 2015. Model transparency and validation of results are crucial in the acceptance and uptake of any new model. The PRIME Diabetes Model code has been both audited by a third-party and validated against studies used in the development of the model and, when available, studies that did not inform model development (external validation). Validation results have been encouraging, with all simulations producing outcomes with annualized AUC differences of less than 1% when compared with published data.
      Through the use of techniques novel to diabetes modeling, efforts have been made to make the model as representative and accurate as possible across a range of settings and populations. For example, studies have shown that prediction of CVD risk in populations with T1DM is population-specific [
      • Zgibor J.C.
      • Piatt G.A.
      • Ruppert K.
      • et al.
      Deficiencies of cardiovascular risk prediction models for type 1 diabetes.
      ]. CVD risk estimation in the PRIME Diabetes Model is based on a model averaging approach weighted by mean cohort characteristics. An analogous approach has been used in other modeling fields [
      • Wang Y.
      • Wu X.
      • Mo X.
      A novel adaptive-weighted-average framework for blood glucose prediction.
      ], and its application in the PRIME Diabetes Model has recently been published. In other areas, where two data sources were available or end-point definition was not clear, multimodels were incorporated such that uncertainty about disease progression and model structure could be accounted for. The PRIME Diabetes Model is also set apart from other models of T1DM, being a patient-level simulation and incorporating covariance of baseline patient characteristics and risk factor progression. The influence of covariance on model outcomes is an ongoing area of interest, and initial evaluations have suggested that although the impact on incremental cost-effectiveness ratios is limited, it does influence patient survival [

      Valentine WJ, Van Brunt K, Boye KS, Pollock RF. Treating type 1 diabetes with a rapid-acting analog insulin regimen versus regular human insulin in Germany: a long-term cost-effectiveness evaluation. Appl Health Econ Health Policy. In press.

      ]. Furthermore, it is hypothesized that covariance may have a more substantial role in model discrimination when compared with calibration.
      The PRIME Diabetes Model was designed as a health economic model and is primarily aimed at informing the decisions of health technology assessment groups, health care payers, and health care providers. In this context, it may also interest associated medical and insurance companies and academia. A number of novel features and positive validation data make the PRIME Diabetes Model an attractive option for modeling studies. Like all models, however, it has limitations. Although the risk of complication onset is taken solely from publications specific to T1DM, certain outcomes such as mortality associated with hypoglycemia are taken from populations with diabetes (mixed T1DM and T2DM). The model is only as accurate and transferable across populations as the data that inform it. As demonstrated for CVD, transferability of risk estimates between T1DM populations in different settings is not always accurate [
      • Zgibor J.C.
      • Piatt G.A.
      • Ruppert K.
      • et al.
      Deficiencies of cardiovascular risk prediction models for type 1 diabetes.
      ].
      For certain complications, no data were available for external validation, meaning that complication risk had been validated only internally against source data. Source data were also taken from the published literature, and were generally available as mean cohort characteristics, with or without a measure of variance. These cohort summary statistics were used to produce a cohort of individual patients within the model that shared the same mean characteristics. Whether the model population truly reflected that underlying the source data is unknown. Furthermore, given that the PRIME Diabetes Model captures a large number of risk factors, not all risk factors were always available in the source data and were, in these cases, derived from other publications in T1DM. In general, patient-level simulation is seen as a benefit in health economic modeling [
      • Davis S.
      • Stevenson M.
      • Tappenden P.
      • Wailoo A.
      NICE DSU Technical Support Document 15.
      ]. Its implementation, however, can preclude the use of common modeling software such as Microsoft Excel and TreeAge (TreeAge Software, Inc., Williamstown, MA) for more complex diseases, which are recommended by reimbursement authorities in some countries. The model is coded in Java, which is a comparatively fast and flexible language that facilitates simulations on a cohort of 1,000,000 patients, with PSA, within 18 minutes (or <2 minutes without PSA).
      The PRIME Diabetes Model is a validated and externally audited patient-level, discrete event simulation model for populations with T1DM. The model is product-independent, available online, and has been developed in line with good practice guidelines. It has been developed using recent data specific to populations with T1DM and validation has indicated that outcomes from long-term T1DM studies can be reliably reproduced. The model offers new approaches to long-standing challenges in diabetes modeling and may become a valuable tool for informing health care policy in T1DM.

      Acknowledgments

      We are grateful to Professor Michael Brändle for reviewing an early version of the manuscript, and to all the expert advisors who provided valuable input during the development of the model at the advisory board meetings. We are also grateful to Eli Lilly and Company for funding to support the development of the model.
      Source of financial support: This study was supported by funding from Eli Lilly and Company.

      Supplemental Materials

      References

        • Daneman D.
        Type 1 diabetes.
        Lancet. 2006; 367: 847-858
        • Lung T.W.C.
        • Hayes A.J.
        • Herman W.H.
        • et al.
        A meta-analysis of the relative risk of mortality for type 1 diabetes patients compared to the general population: exploring temporal changes in relative mortality.
        PLoS One. 2014; 9: e113635
        • Patterson C.C.
        • Dahlquist G.G.
        • Gyürüs E.
        • et al.
        • EuroDiab Study Group
        Incidence trends for childhood type 1 diabetes in Europe during 1989–2003 and predicted new cases 2005–2020—a multicentre prospective registration study.
        Lancet. 2009; 373: 2027-2033
        • Nathan D.M.
        • DCCT/EDIC Research Group
        The diabetes control and complications trial/epidemiology of diabetes interventions and complications study at 30 years: overview.
        Diabetes Care. 2014; 37: 9-16
        • Klein R.
        • Klein B.E.
        Relation of glycemic control to diabetic complications and health outcomes.
        Diabetes Care. 1998; 21: C39-C43
        • Aathira R.
        • Jain V.
        Advances in management of type 1 diabetes mellitus.
        World J Diabetes. 2014; 5: 689-696
        • Eddy D.M.
        • Hollingworth W.
        • Caro J.J.
        • et al.
        • ISPOR-SMDM Modeling Good Research Practices Task Force
        Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7.
        Med Decis Making. 2012; 32: 733-743
        • American Diabetes Association Consensus Panel
        Guidelines for computer modeling of diabetes and its complications.
        Diabetes Care. 2004; 27: 2262-2265
        • Palmer A.J.
        • Clarke P.
        • Gray A.
        • et al.
        • Mount Hood 5 Modeling Group
        Computer modeling of diabetes and its complications: a report on the Fifth Mount Hood Challenge Meeting.
        Value Health. 2013; 16: 670-685
        • Mount Hood 4 Modeling Group
        Computer modeling of diabetes and its complications: a report on the Fourth Mount Hood Challenge Meeting.
        Diabetes Care. 2007; 30: 1638-1646
        • de Boer I.H.
        • Rue T.C.
        • Cleary P.A.
        • et al.
        • Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications Study Research Group
        Long-term renal outcomes of patients with type 1 diabetes mellitus and microalbuminuria: an analysis of the Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications cohort.
        Arch Intern Med. 2011; 171: 412-420
        • Nathan D.M.
        • Zinman B.
        • Cleary P.A.
        • et al.
        • Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications (DCCT/EDIC) Research Group
        Modern-day clinical course of type 1 diabetes mellitus after 30 years’ duration: the diabetes control and complications trial/epidemiology of diabetes interventions and complications and Pittsburgh epidemiology of diabetes complications experience (1983–2005).
        Arch Intern Med. 2009; 169: 1307-1316
        • Jonasson J.M.
        • Ye W.
        • Sparén P.
        • et al.
        Risks of nontraumatic lower-extremity amputations in patients with type 1 diabetes: a population-based cohort study in Sweden.
        Diabetes Care. 2008; 31: 1536-1540
        • Caro J.J.
        • Briggs A.H.
        • Siebert U.
        • Kuntz K.M.
        • ISPOR-SMDM Modeling Good Research Practices Task Force
        Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1.
        Value Health. 2012; 15: 796-803
        • Zgibor J.C.
        • Piatt G.A.
        • Ruppert K.
        • et al.
        Deficiencies of cardiovascular risk prediction models for type 1 diabetes.
        Diabetes Care. 2006; 29: 1860-1865
        • Wang Y.
        • Wu X.
        • Mo X.
        A novel adaptive-weighted-average framework for blood glucose prediction.
        Diabetes Technol Ther. 2013; 15: 792-801
      1. Valentine WJ, Van Brunt K, Boye KS, Pollock RF. Treating type 1 diabetes with a rapid-acting analog insulin regimen versus regular human insulin in Germany: a long-term cost-effectiveness evaluation. Appl Health Econ Health Policy. In press.

        • Davis S.
        • Stevenson M.
        • Tappenden P.
        • Wailoo A.
        NICE DSU Technical Support Document 15.
        Cost-effectiveness Modelling using Patient-level Simulation. April, 2014; (Available at: http://www.nicedsu.org.uk/TSD15_Patient-level_simulation.pdf. Accessed on April 10, 2017)