Advertisement

Addressing Regional Differences in Diabetes Progression: Global Calibration for Diabetes Simulation Model

Open ArchivePublished:October 11, 2019DOI:https://doi.org/10.1016/j.jval.2019.08.007

      Abstract

      Objectives

      To develop a practical solution for modeling diabetes progression and account for the variations in risks of diabetes complications in different regions of the world, which is critical for model-based evaluations on the value of diabetes intervention across populations from different regions globally.

      Methods

      A literature search was conducted to identify eligible clinical trials to support calibration. The Building, Relating, Assessing, and Validating Outcomes (BRAVO) model was employed to simulate diabetes complications using the baseline characteristics of each clinical trial cohort. We utilized regression methods to estimate regional variations across the United States, Europe, Asia, and other regions (eg, Latin America, Africa) in 6 outcomes: myocardial infarction (MI), congestive heart failure (CHF), stroke, angina, revascularization, and mortality.

      Results

      Regional variations were detected in 4 outcomes. Compared with other regions, individuals from the United States had higher risks of MI (hazard ratio [HR] 1.64; 95% confidence interval [CI]1.41–1.91) and revascularization (HR 3.6; 95% CI 2.94–4.41). Individuals from Europe had a lower risk of stroke (HR 0.61; 95% CI 0.46–0.81), and individuals from other regions outside of the United States, Europe, and Asia had a lower risk of CHF (HR 0.18; 95% CI 0.06–0.58). Finally, the simulated outcomes were regressed on observed outcomes using an ordinary least squares model, with an intercept (0.026), slope (1.005), and R-squared value (0.789) indicating good prediction accuracy.

      Conclusion

      Recalibrating the BRAVO model’s diabetes risk engine to account for regional differences shows improved prediction accuracy when the model is applied to multi-region populations commonly recruited for clinical trials.

      Keywords

      Introduction

      Individuals living with diabetes have a higher risk of developing macrovascular and microvascular complications.
      • Laakso M.
      Heart in diabetes: a microvascular disease.
      • Shah A.D.
      • Langenberg C.
      • Rapsomaniki E.
      • et al.
      Type 2 diabetes and incidence of cardiovascular diseases: a cohort study in 1· 9 million people.
      Nevertheless, because of the chronic nature of diabetes, the long-term escalated risks for complications are often hard to observe in clinical trials, which usually have limited follow-up time or are from relatively selected populations. In addition, although type 2 diabetes is reversible,
      • Lean M.E.
      • Leslie W.S.
      • Barnes A.C.
      • et al.
      Primary care-led weight management for remission of type 2 diabetes (DiRECT): an open-label, cluster-randomised trial.
      people usually live with the disease for life. Thus understanding lifetime benefits becomes critical when evaluating the effectiveness of an intervention program, drug, or policy targeted at diabetes. Researchers and policy makers often rely on computer simulation to produce lifetime estimates.
      • Zhuo X.
      • Zhang P.
      • Barker L.
      • Albright A.
      • Thompson T.J.
      • Gregg E.
      The lifetime cost of diabetes and its implications for diabetes prevention.
      • Zhuo X.
      • Zhang P.
      • Hoerger T.J.
      Lifetime direct medical costs of treating type 2 diabetes and diabetic complications.
      • Deng J.
      • Gu S.
      • Shao H.
      • Dong H.
      • Zou D.
      • Shi L.
      Cost-effectiveness analysis of exenatide twice daily (BID) vs insulin glargine once daily (QD) as add-on therapy in Chinese patients with type 2 diabetes mellitus inadequately controlled by oral therapies.
      • Shao H.
      • Zhai S.
      • Zou D.
      • et al.
      Cost-effectiveness analysis of dapagliflozin versus glimepiride as monotherapy in a Chinese population with type 2 diabetes mellitus.
      The field of diabetes modeling has evolved significantly over the last 2 decades. Numerous diabetes simulation models have been developed to estimate the long-term clinical outcomes based on individuals’ current health status and treatment regimen.
      • Hayes A.J.
      • Leal J.
      • Gray Am
      • Holman R.R.
      • Clarke P.M.
      UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82.
      • Hoerger T.J.
      • Segel J.E.
      • Zhang P.
      • Sorensen S.W.
      Validation of the CDC-RTI diabetes cost-effectiveness model. RTI Press.
      • Stern M.
      • Williams K.
      • Eddy D.
      • Kahn R.
      Validation of prediction of diabetes by the Archimedes model and comparison with other predicting models.
      • Shao H.
      • Fonseca V.
      • Stoecker C.
      • Liu S.
      • Shi L.
      Novel risk engine for diabetes progression and mortality in USA: building, relating, assessing, and validating outcomes (BRAVO).
      The data sources used to estimate hazard rates of cardiovascular events and mortality vary across different models. For example, the United Kingdom Prospective Diabetes Study (UKPDS) Outcomes Model
      • Hayes A.J.
      • Leal J.
      • Gray Am
      • Holman R.R.
      • Clarke P.M.
      UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82.
      used data from the UKPDS trial, which is a UK-based cohort with a follow-up time of more than 30 years. The Building, Relating, Assessing, Validating Outcomes (BRAVO) diabetes model
      • Shao H.
      • Fonseca V.
      • Stoecker C.
      • Liu S.
      • Shi L.
      Novel risk engine for diabetes progression and mortality in USA: building, relating, assessing, and validating outcomes (BRAVO).
      relied on estimates from the Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial, one of the largest US-based diabetes trials. Meanwhile, the Centers for Disease Control–Research Triangle Institute (CDC-RTI) diabetes cost-effectiveness model used meta-analysis to synthesize transitional probabilities from studies globally.
      • Hoerger T.J.
      • Segel J.E.
      • Zhang P.
      • Sorensen S.W.
      Validation of the CDC-RTI diabetes cost-effectiveness model. RTI Press.
      These data sources were either region specific or too aggregated to properly adjust for regional variations. To date, no model has adjusted for regional variations in cardiovascular diseases (CVD) and mortality rates. An implicit assumption is made when applying models to simulate populations outside of the region of the training data set. It is assumed that no regional variations of diabetes complication risks exist, after controlling for other factors such as race, age, and education.
      The validity of such an assumption requires further examination. According to numerous studies, variations in CVD risk exist across diabetes populations from different regions globally. For example, a literature review synthesizing evidence on regional variation of CVD risk across diabetes populations found that North American and Caribbean regions have the highest CVD prevalence (46.0%) whereas the Western Pacific region has the lowest (33.5%).
      • Einarson T.R.
      • Acs A.
      • Ludwig C.
      • Panton U.H.
      Prevalence of cardiovascular disease in type 2 diabetes: a systematic literature review of scientific evidence from across the world in 2007–2017.
      In addition, an epidemiology study found that the risk of developing coronary heart disease among diabetes populations of South Asian origin is twice that of Europeans, and the disease occurred 5 to 10 years earlier.
      • Aambø A.
      • Klemsdal T.O.
      Cardiovascular disease and diabetes in patients with African or Asian background.
      Researchers have also found that patients with diabetes from Eastern Europe have a higher risk of developing major coronary events, congestive heart failure (CHF), peripheral vascular events, and death when compared with Asians, even after controlling for baseline differences.
      • Clarke P.M.
      • Glasziou P.
      • Patel A.
      • et al.
      Event rates, hospital utilization, and costs associated with major complications of diabetes: a multicountry comparative analysis.
      • Woodward M.
      • Patel A.
      • Zoungas S.
      • et al.
      Does glycemic control offer similar benefits among patients with diabetes in different regions of the world? Results from the ADVANCE trial.
      Lastly a literature review on the trials examining the effectiveness of intensive glycemic control has found that trials conducted in North America had significantly higher all-cause mortality, cardiovascular mortality, and severe hypoglycemia, compared with studies conducted in the rest of the world.
      • Sardar P.
      • Udell J.A.
      • Chatterjee S.
      • Bansilal S.
      • Mukherjee D.
      • Farkouh M.E.
      Effect of intensive versus standard blood glucose control in patients with Type 2 diabetes mellitus in different regions of the world: systematic review and meta-analysis of randomized controlled trials.
      This study aims to develop a globalization module to calibrate the BRAVO diabetes model to apply it to disease progression in different regions and demonstrate how to estimate a set of regional modifiers to capture the regional variations in CVD risks and mortality.

      Methods

      This study had three key steps: (1) conduct a literature search to identify clinical trials that have sufficient information to support the calibration process, (2) simulate each trial based on its baseline characteristics, and (3) estimate regional multipliers through regressions.

       Literature Search and Data Extraction

      The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist was used to guide the searching process.
      • Moher D.
      • Liberati A.
      • Tetzlaff J.
      • Altman D.G.
      Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PRISMA Group.
      The literature on diabetes clinical trials was searched on PubMed (January 1, 2000-February 21, 2018). We only included trials that began after 2000 to minimize the effects of technological advancement on complication survival rates.
      We applied 4 inclusion criteria: (1) studies with randomized controlled trials (RCTs), (2) studies with a target population of individuals with type 2 diabetes, (3) studies with at least 1 CVD outcome or mortality, and (4) studies reported in English.
      We applied 4 exclusion criteria: (1) studies with a sample size of less than 1000 person-years, (2) studies with no data on individuals’ biomarkers, (3) studies with no information on the geographical location of the study sample, and (4) studies begun before 2000.
      A standardized data extraction form was used to collect the data (see Appendix 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2019.08.007), and data were independently extracted. Two reviewers (Shao and Yang) independently carried out the selection process. Any discrepancies among reviewers’ records were addressed by a mutual check and resolved by consensus within the research team. Three rounds of reviews were carried out: (1) title review, (2) abstract review, and (3) full-text review. Details for the reviewing process are summarized in Figure 1. Population parameters extracted to support model calibration included demographic characteristics, biomarkers, disease histories, and geographical location. We also extracted the incidence rates for myocardial infarction (MI), stroke, congestive heart failure (CHF), revascularization, angina, mortality, and median follow-up time from each identified study (see Appendix 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2019.08.007). A PRISMA checklist is presented in Appendix 2 in Supplemental Materials (found at https://doi.org/10.1016/j.jval.2019.08.007).

       Simulation Using the BRAVO Diabetes Model

      In this study, we used the BRAVO diabetes model as an example to demonstrate the proposed calibration process.
      • Shao H.
      • Fonseca V.
      • Stoecker C.
      • Liu S.
      • Shi L.
      Novel risk engine for diabetes progression and mortality in USA: building, relating, assessing, and validating outcomes (BRAVO).
      The BRAVO diabetes model is a patient-level/cohort-level discrete-time microsimulation model, which predicts risks of macrovascular/microvascular complications and mortality over a user-specified time horizon (Fig. 2). The model considered the time-varying characteristics of the target population and translated it into lifetime risks of complications. The simulation was conducted on an annual cycle, and the risk of each event in a single year was predicted by a patient’s current biomarkers (eg, blood glucose, blood pressure, BMI, and cholesterol), sociodemographic characteristics (eg, gender, race, education), smoking status, and clinical characteristics (eg, macrovascular history, microvascular history). Events occurred in each year when the calculated risk of that event was greater than a random number generated from a uniform distribution between 0 and 1. We ran the simulation 100 000 times for each study cohort to achieve stable estimation. The simulation flow for the BRAVO model is similar to a large number of current models (eg, UKPDS, CORE model
      • Palmer A.J.
      • Roze S.
      • Valentine W.J.
      • et al.
      The CORE Diabetes Model: projecting long-term clinical outcomes, costs and cost-effectiveness of interventions in diabetes mellitus (types 1 and 2) to support clinical and reimbursement decision-making.
      ) and thus is a suitable method for demonstrating our approach to handling regional variation.
      Figure thumbnail gr2
      Figure 2The simulation flow of the Building, Relating, Assessing, and Validating Outcomes diabetes model.
      In this study, owing to the lack of patient-level data, we conducted the simulation at a cohort level (ie, the study-arm level). Each study arm was simulated by entering its baseline characteristics into the BRAVO model, and the incidence rate (cases per 100 person-year) for MI, stroke, CHF, revascularization, angina, and mortality were predicted and recorded using the median follow-up time in each trial as the time horizon. Each cohort occupied a single line in the final data set, which contained information on predicted outcomes, observed outcomes, and population distribution.

       Model Specification and Regression Methods

      Six risk equations from the BRAVO model were calibrated: nonfatal MI, stroke, CHF, angina, revascularization surgery, and all-cause mortality.
      hazardit=ht*e(βD.B.T.HD.B.T.Hit+γRegioni+εit)
      Eq (1)


      Equation 1 is the proposed function for calculating the risk of event, which is a proportion hazard equation often used to map event risks over time. The hazard rate is denoted by the term hazardit and is the rate of encountering the event at time t for individual i. The term ht is the baseline hazard at time t. The vector βD.B.T.H is comprised of variables commonly used in diabetes simulation modeling, including demographics, biomarkers, treatments, and histories of complications. The vector βD.B.T.H is composed of coefficients for corresponding risk factors in D.B.T.Hit. The vector Regioni is comprised of regional identifiers (ie, the United States, Europe, Asia, and other regions), denoting the regional allocation of the individual i, and γ is a vector of regional multipliers for each regional, which this study aims to estimate.
      Because of the nature of single-region trials such as the ACCORD trial (the data source of the BRAVO model), γ could not be directly estimated. Instead, the regional multiplier of the source region was included in the baseline hazard (Eq. 2).
      hazardit=ht*e(βD.B.T.HD.B.T.Hit+γUS+εit)=[hteγUS]*e(βD.B.T.HD.B.T.Hit+εit)
      Eq (2)


      γUS denotes the regional multiplier of the United States (the source region of the BRAVO model), and hteγUS constitutes the baseline hazard of the BRAVO model. The relative bias when applying the BRAVO model on another region is
      RelativeBiasit=e(γtargetγUS)
      Eq (3)


      where γtarget denotes the regional multiplier of the target region.
      In this study, we grouped geographic regions under the categories of the United States, Europe, Asia, and the rest of the world (called “others”). Thus Equation 1 can be rewritten as
      hazardit=ht*e(βD.B.T.HD.B.T.Hit+γEUEUi+γUSUSi+γAsiaAsiai+γothersothersi+εit)
      Eq (4)


      where EUi, USi, Asiai, othersi denote proportions of study populations in each region for each study cohort, and γEU,γUS,γAsia and γothers are the corresponding regional multipliers. Equation 4 can be further written as
      hazardit=hte(γUS+βD.B.T.HD.B.T.Hit+εBRAVO)e(εBRAVO+γEUEUi+γUSUSiγUS+γAsiaAsiai+γothersothersi+εit)
      Eq (5)


      where εBRAVO denotes the error term of the BRAVO model. The term hte(γUS+βD.B.T.HD.B.T.Hit+εBRAVO) is the predicted hazard from the BRAVO model for cohort i, and hazardit is the observed hazard of the cohort i. Thus Equation 6 can be derived from Equation 5, assuming the hazard ratios were constant over time:
      Ln (Observedipredictedi)=β0+γEUEUi+γUS(USi1)+γAsiaAsiai+γothersothersi+εt
      Eq (6)


      where Obsevedi is the observed incidence rate of the outcome in study arm i, measured per 100 person-years, and Predictedi is the BRAVO-simulated incidence rate. The coefficient β0 measures the systematic bias of the original baseline hazard of the BRAVO model with the calibrated baseline characteristics.
      Because of the small number of identified trials, further variable selections were needed in Equation 6 to avoid overfitting. Several regression methods were applied to estimate Equation 6, including an ordinary least squares (OLS) model, Ridge regression, and a least absolute shrinkage and selection operator (LASSO) regression.
      • Tibshirani R.
      Regression shrinkage and selection via the lasso.
      Mean squared prediction error (MSPE) was used as a measurement for model performance when choosing the final model and variable combination. Details can be found in Tables A1 to A6. Finally, calibration plots were drawn, using observed incidence against predicted incidence for each event type and all events combined. An OLS model of observed incidence was fitted on predicted incidence both before and after the calibration. An ideal model fit is represented by a regression with slope 1 and intercept 0.

      Results

      The flow chart of the literature search is provided in Figure 1. Overall, 467 publications were identified from the initial search, among which 338 were excluded after the title review. Of the remaining 129 studies, 42 were selected after the abstract review. After the full-text review, 18 studies were identified as eligible studies for data extraction. Table 1 summarizes the list of identified trials.
      • Marso S.P.
      • Daniels G.H.
      • Brown-Frandsen K.
      • et al.
      Liraglutide and cardiovascular outcomes in type 2 diabetes.
      • Neal B.
      • Perkovic V.
      • Matthews D.R.
      Canagliflozin and cardiovascular and renal events in type 2 diabetes.
      • Zinman B.
      • Wanner C.
      • Lachin J.M.
      • et al.
      Empagliflozin, cardiovascular outcomes, and mortality in type 2 diabetes.
      • Scirica B.M.
      • Bhatt D.L.
      • Braunwald E.
      • et al.
      Saxagliptin and cardiovascular outcomes in patients with type 2 diabetes mellitus.
      • Griffin S.J.
      • Borch-Johsen K.
      • Davies M.J.
      • et al.
      Effect of early intensive multifactorial therapy on 5-year cardiovascular outcomes in individuals with type 2 diabetes detected by screening (ADDITION-Europe): a cluster-randomised trial.
      • Pfeffer M.A.
      • Claggett B.
      • Diaz R.
      • et al.
      Lixisenatide in patients with type 2 diabetes and acute coronary syndrome.
      • Dormandy J.A.
      • Charbonnel B.
      • Eckland D.J.
      • et al.
      Secondary prevention of macrovascular events in patients with type 2 diabetes in the PROactive Study (PROspective pioglitAzone Clinical Trial In macroVascular Events): a randomised controlled trial.
      • Haller H.
      • Ito S.
      • Izzo Jr., J.L.
      • et al.
      Olmesartan for the delay or prevention of microalbuminuria in type 2 diabetes.
      • Colhoun H.M.
      • Betteridge D.J.
      • Durrington P.N.
      • et al.
      Primary prevention of cardiovascular disease with atorvastatin in type 2 diabetes in the Collaborative Atorvastatin Diabetes Study (CARDS): multicentre randomised placebo-controlled trial.
      • Patel A.
      • MacMahon S.
      • et al.
      ADVANCE Collaborative Group
      Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes.
      • Home P.D.
      • Pocock S.J.
      • Beck-Nielsen H.
      • et al.
      Rosiglitazone evaluated for cardiovascular outcomes in oral agent combination therapy for type 2 diabetes (RECORD): a multicentre, randomised, open-label trial.
      • Knopp R.H.
      • d’Emden M.
      • Smilde J.G.
      • Pocock S.J.
      Efficacy and safety of atorvastatin in the prevention of cardiovascular end points in subjects with type 2 diabetes: the Atorvastatin Study for Prevention of Coronary Heart Disease Endpoints in non-insulin-dependent diabetes mellitus (ASPEN).
      • Marre M.
      • Lievre M.
      • Chatellier G.
      • et al.
      Effects of low dose ramipril on cardiovascular and renal outcomes in patients with type 2 diabetes and raised excretion of urinary albumin: randomised, double blind, placebo controlled trial (the DIABHYCAR study).
      • Green J.B.
      • Bethel M.A.
      • Armstrong P.W.
      • et al.
      Effect of sitagliptin on cardiovascular outcomes in type 2 diabetes.
      • Keech A.
      • Simes R.J.
      • Barter P.
      • et al.
      Effects of long-term fenofibrate therapy on cardiovascular events in 9795 people with type 2 diabetes mellitus (the FIELD study): randomised controlled trial.
      • Duckworth W.
      • Abraira C.
      • Moritz T.
      • et al.
      Glucose control and vascular complications in veterans with type 2 diabetes.
      • Gerstein H.C.
      • Bosch J.
      • et al.
      ORIGIN Trial Investigators
      Basal insulin and cardiovascular and other outcomes in dysglycemia.
      Action to Control Cardiovascular Risk in Diabetes Study Group, Gerstein HC, Miller ME, et al. Effects of intensive glucose lowering in type 2 diabetes.
      Table 1List of studies enrolled in simulation process.
      Trial acronymFull nameRegionsSample sizeFollow-up yearsReference no.
      LEADERThe Liraglutide Effect and Action in Diabetes: Evaluation of Cardiovascular Outcome Results TrialEU, US, Asia, Others93403.821
      CANVASThe Canagliflozin Cardiovascular Assessment StudyEU, US, Others10 142422
      EMPA-REG OUTCOMEThe Empagliflozin Cardiovascular Outcome Event Trial in Type 2 Diabetes Mellitus Patients–Removing Excess Glucose StudyEU, US, Asia, Others70203.123
      TIMI-53The Saxagliptin Assessment of Vascular Outcomes Recorded in Patients with Diabetes Mellitus (SAVOR)–Thrombolysis in Myocardial Infarction 53 trialEU, US, Asia, Others16 4922.124
      ADDITION-EuropeAnglo-Danish-Dutch Study of Intensive Treatment In People with Screen Detected Diabetes in Primary CareEU30575.325
      ELIXAThe Evaluation of Lixisenatide in Acute Coronary Syndrome TrialEU, US, Asia, Others60682.126
      PROactiveProspective Pioglitazone Clinical Trial in Macrovascular EventsEU52382.927
      ROADMAPRandomized Olmesartan and Diabetes Microalbuminuria Prevention StudyEU44473.228
      CARDSCollaborative Atorvastatin Diabetes StudyEU28383.929
      AdvanceThe Action in Diabetes and Vascular Disease: Preterax and Diamicron Modified Release Controlled Evaluation TrialEU, US, Asia, Others11 140530
      RECORDRosiglitazone Evaluated for Cardiac Outcomes and Regulation of Glycaemia in DiabetesEU44475.531
      ASPENThe Atorvastatin Study for Prevention of Coronary Heart Disease Endpoints in Non-Insulin-Dependent Diabetes MellitusEU, US, Asia, Others2410432
      DIABHYCARNon-insulin-dependent Diabetes, Hypertension, Microalbuminuria or Proteinuria, Cardiovascular Events, and Ramipril StudyEU, Others4912433
      TECOsTrial Evaluating Cardiovascular Outcomes with SitagliptinEU, US, Asia, Others14 671334
      FIELDFenofibrate Intervention and Event Lowering in Diabetes StudyEU, Others9795535
      VADTVeterans Affairs Diabetes TrialUS17915.636
      ORIGINThe Outcome Reduction with an Initial Glargine Intervention TrialEU, US, Others12 5376.237
      ACCORDThe Action to Control Cardiovascular Risk in Diabetes TrialUS10 2513.538
      Simulations using the BRAVO model were conducted based on baseline population information from each study arm (total N = 36), and the model/variable selection process was conducted on a range of regression methods using cross-validated mean squared prediction error as the measurement of prediction accuracy. Details of the variable selection process for each risk equation are summarized in Appendix Tables 1 to 6 in Supplementary Materials (found at https://doi.org/10.1016/j.jval.2019.08.007). Across alternative model specifications, the OLS model had the best prediction accuracy in this study.
      Regional multipliers were generated for 4 risk equations (Table 2). An empty cell indicates that the corresponding variable was excluded by the model selection process. Individuals from the United States were found to have a higher risk of a nonfatal MI event (hazard ratio [HR] 1.64; 95% confidence interval [CI] 1.41–1.91), compared with other regions. Individuals from Europe were found to have a lower risk of a stroke event (HR 0.61; 95% CI 0.46–0.81), compared with other regions. Individuals from regions other than the United States, Europe, and Asia were found to have a lower risk of CHF (HR 0.18; 95% CI 0.06–0.58). Individuals from the United States had a higher risk of revascularization surgery (HR 3.6; 95% CI 2.94–4.41). Intercept terms indicating the regional baseline-risk variations from the baseline risk of the ACCORD cohort were included in 4 risk equations: stroke (HR 2.03; 95% CI 1.70–2.42), CHF (HR 1.28; 95% CI 1.05–1.55), angina (HR 0.81; 95% CI 0.68–0.96), and all-cause mortality (HR 1.83; 95% CI 1.60–2.08). The estimated coefficients for each equation can be found in Appendix 3 in Supplemental Materials (found at https://doi.org/10.1016/j.jval.2019.08.007).
      Table 2Hazard ratios for each region on six major event types.
      Nonfatal MIStrokeCHFAnginaRevascularizationAll-cause Mortality
      United States1.64 (1.41–1.91)3.60 (2.94–4.41)
      Europe0.61 (0.46–0.81)
      Asia
      Others0.18 (0.06–0.58)
      Intercept2.03 (1.70–2.42)1.28 (1.05–1.55)0.81 (0.68–0.96)1.83 (1.60–2.08)
      Model typeOLSOLSOLSOLSOLSOLS
      Note. An empty cell indicates the corresponding region variable was excluded from the calibration equation. 95% confidence intervals are included in parentheses.
      CHF indicates congestive heart failure; MI, myocardial infarction; OLS, ordinary least squares.
      Calibration plots with slope 1 as perfect prediction accuracy were drawn using the observed incidence against the predicted incidence (before vs after calibration with regional risk multipliers) (Fig. 3). An OLS model of the observed incidence was fit on the predicted incidence, and the results of intercepts and slopes are recorded in Table A7. Among 6 outcomes, 3 of them were found to have an improvement on prediction accuracy (slope closer to 1.00) after applying regional multipliers: MI (slope: 0.84 vs 1.01), revascularization (slope: 0.60 vs 1.04), and all-cause mortality (slope: 1.53 vs 0.89). For angina, CHF, and stroke, we did not observe prediction improvement after applying regional multipliers.
      Figure thumbnail gr3
      Figure 3Calibration plots for risk equations. Perfect predictions are indicated by points located on the 45-degree line. Hollow dots denote predictions before calibration, whereas solid dots denote predictions after adding regional multipliers to the Building, Relating, Assessing, and Validating Outcomes model.
      Figure 4 further summarizes the final external validation plots, using observed incidence against calibrated prediction on all events. We regressed the observed incidence on predicted incidence before and after calibration, as compared with perfect prediction accuracy (intercept = 0 and slope = 1). The globalization module using regional multipliers improved the prediction accuracy as measured by intercept (0.85 before vs 0.026 after) and slope (0.276 before vs 1.005 after) and regression model fit as measured by adjusted R-squared value (0.287 before vs 0.789 after).
      Figure thumbnail gr4
      Figure 4Validation plots for all 6 outcomes. Slope and intercepts for “before” and “after” regional calibration are generated by weighting all 6 outcomes equally. The solid 45-degree line indicates a perfect prediction.

      Discussions

      Regional variations contribute to heterogeneities in CVD event incidence among individuals with diabetes. Even after controlling for characteristics of the population, regional differences still affect the risk of CVD events through other pathways, such as the characteristics of health systems, lifestyle, and the policy environment. Consistent with findings from Einarson et al,
      • Einarson T.R.
      • Acs A.
      • Ludwig C.
      • Panton U.H.
      Prevalence of cardiovascular disease in type 2 diabetes: a systematic literature review of scientific evidence from across the world in 2007–2017.
      our study also identified a higher risk of nonfatal MI and revascularization among the US population. Because this study already controls for a wide range of risk factors associated with patients’ health conditions, the escalated rate of nonfatal MI and revascularization surgery in the US population is likely due to accessibility of specialty care or a difference in clinical practice patterns. In other words, revascularization surgery is more frequently performed in the United States compared with other regions, which in turn leads to a higher survival rate from complication events
      • Hochman J.S.
      • Boland J.
      • Sleeper L.A.
      • et al.
      Current spectrum of cardiogenic shock and effect of early revascularization on mortality: results of an international registry.
      (eg, higher risk for nonfatal MI). Also, we found that individuals from Europe had a lower risk for stroke compared with individuals from the rest of the world. This might be attributable to social economic factors
      • Feigin V.L.
      • Lawes C.M.
      • Bennett D.A.
      • Barker-Collo S.L.
      • Parag V.
      Worldwide stroke incidence and early case fatality reported in 56 population-based studies: a systematic review.
      and access to vascular prevention strategies among those at high risk.
      • Feigin V.L.
      • Lawes C.M.
      • Bennett D.A.
      • Anderson C.S.
      Stroke epidemiology: a review of population-based studies of incidence, prevalence, and case-fatality in the late 20th century.
      The findings in this study shed further light into variations in global CVD outcome studies across different regions of the world, such as the BI 10773 (Empagliflozin) Cardiovascular Outcome Event Trial in Type 2 Diabetes Mellitus Patients (EMPA-REG OUTCOME)
      • Fitchett D.
      • Zinman B.
      • Wanner C.
      • et al.
      Heart failure outcomes with empagliflozin in patients with type 2 diabetes at high cardiovascular risk: results of the EMPA-REG OUTCOME® trial.
      and the Canagliflozin Cardiovascular Assessment Study (CANVAS).
      • Neal B.
      • Perkovic V.
      • Matthews D.R.
      Canagliflozin and cardiovascular and renal events in type 2 diabetes.
      More importantly, the regional variations in diabetes complications may play a greater role in the design and conduct of real-world outcome studies in diabetes (eg, the Comparative Effectiveness of Cardiovascular Outcomes in New Users of SGLT-2 Inhibitors [CVD-REAL] study
      • Birkeland K.I.
      • Jørgensen M.E.
      • Carstensen B.
      • et al.
      Cardiovascular mortality and morbidity in patients with type 2 diabetes following initiation of sodium-glucose co-transporter-2 inhibitors versus other glucose-lowering drugs (CVD-REAL Nordic): a multinational observational analysis.
      ).
      Economic evaluations for health technology assessments have been conducted, relying on source data from outside the targeted region. For example, Shao and colleagues used the Cardiff diabetes model (Europe-based
      • Scirica B.M.
      • Bhatt D.L.
      • Braunwald E.
      • et al.
      Saxagliptin and cardiovascular outcomes in patients with type 2 diabetes mellitus.
      ) to compare the cost-effectiveness of alternative antidiabetic medications among the Chinese population.
      • Deng J.
      • Gu S.
      • Shao H.
      • Dong H.
      • Zou D.
      • Shi L.
      Cost-effectiveness analysis of exenatide twice daily (BID) vs insulin glargine once daily (QD) as add-on therapy in Chinese patients with type 2 diabetes mellitus inadequately controlled by oral therapies.
      • Shao H.
      • Zhai S.
      • Zou D.
      • et al.
      Cost-effectiveness analysis of dapagliflozin versus glimepiride as monotherapy in a Chinese population with type 2 diabetes mellitus.
      This study improves on that approach by providing a novel way to estimate regional multipliers for a diabetes model developed based on data from a single region. As illustrated in Figure 4, this approach can improve the prediction accuracy, both in term of reducing systematic bias (ie, the slope and intercept) and improving explanation power (ie, the R-squared value) of the model. This method is not limited to particular diabetes model structures (eg, Markov or discrete-time microsimulation) or the functional form of the original risk equations, thus it can be easily adopted by other diabetes simulation models beyond the BRAVO model. We encourage current diabetes models to adopt a similar approach to improve their performance and better guide policy making and practice change globally.
      Another contribution of this study is the recalibration of the baseline hazard across different RCT populations, especially for predicting revascularization and all-cause mortality (see Fig. 3 and Appendix Table 7 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2019.08.007). The ACCORD population is a highly selected population with high risk of CVD complication. Although the BRAVO risk equations have been adjusted for a wide range of risk factors, it is still possible that there are selection issues separating the ACCORD population from the general diabetes clinical trial population. Our calibration module has categorized the sources of prediction error into 2 types: those owing to regional variations and those owing to unobserved differences caused by the RCT population selection criteria. For example, our findings suggest that the overestimation of revascularization rates is attributed to regional variation because the ACCORD cohort from the United States tends to use more procedures than other clinical trial regions. The underestimation of all-cause mortality rates in the ACCORD cohort, which tends to have lower mortality rates than comparable clinical trial cohorts, is attributed to unobserved baseline risk. This prediction error caused by the baseline-risk difference for a CVD outcome can be significantly reduced by introducing an intercept term in the regional multiplier function.
      The number of diabetes RCT databases on which this study draws is relatively small, which limits our capacity to build a saturated model (Eq. 6) or to further disaggregate regions into smaller areas (eg, countries). It is possible that different countries within the same region have different risks of CVD events and death. If patient-level data were available, we could further segregate regional effects into country effects. Studies intending to further calibrate the BRAVO model by estimating country-specific multipliers are warranted for better prediction accuracy. Also, the calibration was conducted at the cohort level. It is possible that the regional variation in CVD is not constant across different subgroups within the same region, and the magnitude of regional variation estimated in this study should better be generalized to a cohort that has similar characteristics as the included cohorts. In addition, populations selected by RCT studies often have more severe diabetes than the general diabetes population. This may limit the generalizability of models based on data from RCTs, even after calibration, when targeting general individuals with diabetes. Although the BRAVO model took into account a wide range of baseline characteristics, including diabetes duration and disease history, unobserved characteristics can still lead to systematic bias when predicting incidence among the general population. Moreover, the literature searching process did not include other databases such as Embase. It is possible that eligible studies might be exclusively reported on another database besides PubMed. Nevertheless, this study only includes RCTs with a large number of participants, and to the best of our knowledge, we did not miss relevant large RCTs. Lastly, although the calibration equations improve the model prediction according to our validation of study cohorts in the published clinical trials, further validation is warranted to explore the performance of the proposed method.

      Conclusion

      The globalization module for the BRAVO diabetes model shows an improved prediction accuracy when the diabetes simulation model accounts for different disease progression risks across populations from different regions. This novel calibration approach can be used not only to improve other diabetes models in international transferability but also to better interpret the findings from global diabetes outcomes trials and real-world studies that use data sources from more than one region.

      Acknowledgments

      Shuang Yang and Hui Shao conducted the literature search and data extraction from each identified clinical trial. Hui Shao analyzed the data for the globalization module. Vivian Fonseca provided a clinical interpretation of the regional variations in diabetes complications. Charles Stoecker reviewed the econometrics in the risk engine during the model development. Lizheng Shi, as the principal investigator, initiated the project and worked with Hui Shao in developing the article. All authors were extensively involved in writing the article.
      Guarantor: Hui Shao.
      Source of financial support: No sources of funding were received for the preparation of this article.
      Data availability: The data sets used for this study are publicly available and can be requested.

      Supplemental Material

      References

        • Laakso M.
        Heart in diabetes: a microvascular disease.
        Diabetes Care. 2011; 34: S145-S149
        • Shah A.D.
        • Langenberg C.
        • Rapsomaniki E.
        • et al.
        Type 2 diabetes and incidence of cardiovascular diseases: a cohort study in 1· 9 million people.
        Lancet. 2015; 3: 105-113
        • Lean M.E.
        • Leslie W.S.
        • Barnes A.C.
        • et al.
        Primary care-led weight management for remission of type 2 diabetes (DiRECT): an open-label, cluster-randomised trial.
        Lancet. 2018; 391: 541-551
        • Zhuo X.
        • Zhang P.
        • Barker L.
        • Albright A.
        • Thompson T.J.
        • Gregg E.
        The lifetime cost of diabetes and its implications for diabetes prevention.
        Diabetes Care. 2014; 37: 2557-2564
        • Zhuo X.
        • Zhang P.
        • Hoerger T.J.
        Lifetime direct medical costs of treating type 2 diabetes and diabetic complications.
        Am J Prev Med. 2013; 45: 253-261
        • Deng J.
        • Gu S.
        • Shao H.
        • Dong H.
        • Zou D.
        • Shi L.
        Cost-effectiveness analysis of exenatide twice daily (BID) vs insulin glargine once daily (QD) as add-on therapy in Chinese patients with type 2 diabetes mellitus inadequately controlled by oral therapies.
        J Med Econ. 2015; 18: 974-989
        • Shao H.
        • Zhai S.
        • Zou D.
        • et al.
        Cost-effectiveness analysis of dapagliflozin versus glimepiride as monotherapy in a Chinese population with type 2 diabetes mellitus.
        Curr Med Res Opin. 2017; 33: 359-369
        • Hayes A.J.
        • Leal J.
        • Gray Am
        • Holman R.R.
        • Clarke P.M.
        UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82.
        Diabetologia. 2013; 56: 1925-1933
        • Hoerger T.J.
        • Segel J.E.
        • Zhang P.
        • Sorensen S.W.
        Validation of the CDC-RTI diabetes cost-effectiveness model. RTI Press.
        (Accessed April 4, 2017)
        • Stern M.
        • Williams K.
        • Eddy D.
        • Kahn R.
        Validation of prediction of diabetes by the Archimedes model and comparison with other predicting models.
        Diabetes Care. 2008; 31: 1670-1671
        • Shao H.
        • Fonseca V.
        • Stoecker C.
        • Liu S.
        • Shi L.
        Novel risk engine for diabetes progression and mortality in USA: building, relating, assessing, and validating outcomes (BRAVO).
        Pharmacoeconomics. 2018; 36: 1125-1134
        • Einarson T.R.
        • Acs A.
        • Ludwig C.
        • Panton U.H.
        Prevalence of cardiovascular disease in type 2 diabetes: a systematic literature review of scientific evidence from across the world in 2007–2017.
        Cardiovasc Diabetol. 2018; 17: 83
        • Aambø A.
        • Klemsdal T.O.
        Cardiovascular disease and diabetes in patients with African or Asian background.
        Tidsskr Nor Laegeforen. 2017; 137
        • Clarke P.M.
        • Glasziou P.
        • Patel A.
        • et al.
        Event rates, hospital utilization, and costs associated with major complications of diabetes: a multicountry comparative analysis.
        PLoS Med. 2010; 7: e1000236
        • Woodward M.
        • Patel A.
        • Zoungas S.
        • et al.
        Does glycemic control offer similar benefits among patients with diabetes in different regions of the world? Results from the ADVANCE trial.
        Diabetes Care. 2011; 34: 2491-2495
        • Sardar P.
        • Udell J.A.
        • Chatterjee S.
        • Bansilal S.
        • Mukherjee D.
        • Farkouh M.E.
        Effect of intensive versus standard blood glucose control in patients with Type 2 diabetes mellitus in different regions of the world: systematic review and meta-analysis of randomized controlled trials.
        J Am Heart Assoc. 2015; 4: e001577
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PRISMA Group.
        PLoS Med. 2009; 6: e 1000097
        • Palmer A.J.
        • Roze S.
        • Valentine W.J.
        • et al.
        The CORE Diabetes Model: projecting long-term clinical outcomes, costs and cost-effectiveness of interventions in diabetes mellitus (types 1 and 2) to support clinical and reimbursement decision-making.
        Curr Med Res Opin. 2004; 20: S5-S26
        • Tibshirani R.
        Regression shrinkage and selection via the lasso.
        J Royal Statistical Society. Series B (Methodological). 1996; 58: 267-288
        • Marso S.P.
        • Daniels G.H.
        • Brown-Frandsen K.
        • et al.
        Liraglutide and cardiovascular outcomes in type 2 diabetes.
        N Engl J Med. 2016; 375: 311-322
        • Neal B.
        • Perkovic V.
        • Matthews D.R.
        Canagliflozin and cardiovascular and renal events in type 2 diabetes.
        N Engl J Med. 2017; 377: 644-657
        • Zinman B.
        • Wanner C.
        • Lachin J.M.
        • et al.
        Empagliflozin, cardiovascular outcomes, and mortality in type 2 diabetes.
        N Engl J Med. 2015; 373: 2117-2128
        • Scirica B.M.
        • Bhatt D.L.
        • Braunwald E.
        • et al.
        Saxagliptin and cardiovascular outcomes in patients with type 2 diabetes mellitus.
        N Engl J Med. 2013; 369: 1317-1326
        • Griffin S.J.
        • Borch-Johsen K.
        • Davies M.J.
        • et al.
        Effect of early intensive multifactorial therapy on 5-year cardiovascular outcomes in individuals with type 2 diabetes detected by screening (ADDITION-Europe): a cluster-randomised trial.
        Lancet. 2011; 378: 156-167
        • Pfeffer M.A.
        • Claggett B.
        • Diaz R.
        • et al.
        Lixisenatide in patients with type 2 diabetes and acute coronary syndrome.
        N Engl J Med. 2015; 373: 2247-2257
        • Dormandy J.A.
        • Charbonnel B.
        • Eckland D.J.
        • et al.
        Secondary prevention of macrovascular events in patients with type 2 diabetes in the PROactive Study (PROspective pioglitAzone Clinical Trial In macroVascular Events): a randomised controlled trial.
        Lancet. 2005; 366: 1279-1289
        • Haller H.
        • Ito S.
        • Izzo Jr., J.L.
        • et al.
        Olmesartan for the delay or prevention of microalbuminuria in type 2 diabetes.
        N Engl J Med. 2011; 364: 907-917
        • Colhoun H.M.
        • Betteridge D.J.
        • Durrington P.N.
        • et al.
        Primary prevention of cardiovascular disease with atorvastatin in type 2 diabetes in the Collaborative Atorvastatin Diabetes Study (CARDS): multicentre randomised placebo-controlled trial.
        Lancet. 2004; 364: 685-696
        • Patel A.
        • MacMahon S.
        • et al.
        • ADVANCE Collaborative Group
        Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes.
        N Engl J Med. 2008; 358: 2560-2572
        • Home P.D.
        • Pocock S.J.
        • Beck-Nielsen H.
        • et al.
        Rosiglitazone evaluated for cardiovascular outcomes in oral agent combination therapy for type 2 diabetes (RECORD): a multicentre, randomised, open-label trial.
        Lancet. 2009; 373: 2125-2135
        • Knopp R.H.
        • d’Emden M.
        • Smilde J.G.
        • Pocock S.J.
        Efficacy and safety of atorvastatin in the prevention of cardiovascular end points in subjects with type 2 diabetes: the Atorvastatin Study for Prevention of Coronary Heart Disease Endpoints in non-insulin-dependent diabetes mellitus (ASPEN).
        Diabetes Care. 2006; 29: 1478-1485
        • Marre M.
        • Lievre M.
        • Chatellier G.
        • et al.
        Effects of low dose ramipril on cardiovascular and renal outcomes in patients with type 2 diabetes and raised excretion of urinary albumin: randomised, double blind, placebo controlled trial (the DIABHYCAR study).
        BMJ. 2004; 328: 495
        • Green J.B.
        • Bethel M.A.
        • Armstrong P.W.
        • et al.
        Effect of sitagliptin on cardiovascular outcomes in type 2 diabetes.
        N Engl J Med. 2015; 373: 232-242
        • Keech A.
        • Simes R.J.
        • Barter P.
        • et al.
        Effects of long-term fenofibrate therapy on cardiovascular events in 9795 people with type 2 diabetes mellitus (the FIELD study): randomised controlled trial.
        Lancet. 2005; 366: 1849-1861
        • Duckworth W.
        • Abraira C.
        • Moritz T.
        • et al.
        Glucose control and vascular complications in veterans with type 2 diabetes.
        N Engl J Med. 2009; 360: 129-139
        • Gerstein H.C.
        • Bosch J.
        • et al.
        • ORIGIN Trial Investigators
        Basal insulin and cardiovascular and other outcomes in dysglycemia.
        N Engl J Med. 2012; 367: 319-328
      1. Action to Control Cardiovascular Risk in Diabetes Study Group, Gerstein HC, Miller ME, et al. Effects of intensive glucose lowering in type 2 diabetes.
        N Engl J Med. 2008; 358: 2545-2559
        • Hochman J.S.
        • Boland J.
        • Sleeper L.A.
        • et al.
        Current spectrum of cardiogenic shock and effect of early revascularization on mortality: results of an international registry.
        Circulation. 1995; 91: 873-881
        • Feigin V.L.
        • Lawes C.M.
        • Bennett D.A.
        • Barker-Collo S.L.
        • Parag V.
        Worldwide stroke incidence and early case fatality reported in 56 population-based studies: a systematic review.
        Lancet Neurol. 2009; 8: 355-369
        • Feigin V.L.
        • Lawes C.M.
        • Bennett D.A.
        • Anderson C.S.
        Stroke epidemiology: a review of population-based studies of incidence, prevalence, and case-fatality in the late 20th century.
        Lancet Neurol. 2003; 2: 43-53
        • Fitchett D.
        • Zinman B.
        • Wanner C.
        • et al.
        Heart failure outcomes with empagliflozin in patients with type 2 diabetes at high cardiovascular risk: results of the EMPA-REG OUTCOME® trial.
        Eur. Heart J. 2016; 37: 1526-1534
        • Birkeland K.I.
        • Jørgensen M.E.
        • Carstensen B.
        • et al.
        Cardiovascular mortality and morbidity in patients with type 2 diabetes following initiation of sodium-glucose co-transporter-2 inhibitors versus other glucose-lowering drugs (CVD-REAL Nordic): a multinational observational analysis.
        Lancet Diabetes Endocrinol. 2017; 5: 709-717