Advertisement

Incorporating Budget Impact Analysis in the Implementation of Complex Interventions: A Case of an Integrated Intervention for Multimorbid Patients within the CareWell Study

Open AccessPublished:October 10, 2016DOI:https://doi.org/10.1016/j.jval.2016.08.002

      Abstract

      Objectives

      To develop a framework for the management of complex health care interventions within the Deming continuous improvement cycle and to test the framework in the case of an integrated intervention for multimorbid patients in the Basque Country within the CareWell project.

      Methods

      Statistical analysis alone, although necessary, may not always represent the practical significance of the intervention. Thus, to ascertain the true economic impact of the intervention, the statistical results can be integrated into the budget impact analysis. The intervention of the case study consisted of a comprehensive approach that integrated new provider roles and new technological infrastructure for multimorbid patients, with the aim of reducing patient decompensations by 10% over 5 years. The study period was 2012 to 2020.

      Results

      Given the aging of the general population, the conventional scenario predicts an increase of 21% in the health care budget for care of multimorbid patients during the study period. With a successful intervention, this figure should drop to 18%. The statistical analysis, however, showed no significant differences in costs either in primary care or in hospital care between 2012 and 2014. The real costs in 2014 were by far closer to those in the conventional scenario than to the reductions expected in the objective scenario. The present implementation should be reappraised, because the present expenditure did not move closer to the objective budget.

      Conclusions

      This work demonstrates the capacity of budget impact analysis to enhance the implementation of complex interventions. Its integration in the context of the continuous improvement cycle is transferable to other contexts in which implementation depth and time are important.

      Keywords

      Introduction

      The increasing prevalence of chronic diseases mainly because of an aging population has led to a profound change in the paradigm of health care. Approximately one in four adults have two or more chronic conditions, and half of older adults have three or more [
      • Boyd C.M.
      • Fortin M.
      Future of multimorbidity research: how should understanding of multimorbidity inform health system design.
      ]. Therefore, health systems have changed in perspective, and health care organizations previously concerned mainly with treating acute problems are now focused on a continuum-of-care approach [
      • Allepuz Palau A.
      • Piñeiro Méndez P.
      • Molina Hinojosa J.C.
      • et al.
      Evaluación económica de un programa de coordinación entre niveles para el manejo de pacientes crónicos complejos.
      ]. That implies profound organizational changes [

      Goodwin N, Smith J, Davies A, et al. Integrated care for patients and populations: improving outcomes by working together. 2012. Available from: http://www.kingsfund.org.uk/publications/future_forum_report.html. [Accessed February 13, 2015].

      ,
      • Ham C.
      Chronic care in the English National Health Service: progress and challenges.
      ]. Nevertheless, organizations are dynamic, and interventions that require behavioral changes are difficult to implement. As first shown in 1943, the adoption curve of an innovation has an S shape, with a slow early phase affecting very few people, a rapid middle phase spreading widely, and a slow third phase ending with incomplete penetration [
      • Berwick D.M.
      Disseminating innovations in health care.
      ]. This means that a substantial “steady-state” period during which the intervention could be evaluated is unlikely to be attained quickly [
      • Drummond M.
      • Griffin A.
      • Tarricone R.
      Economic evaluation for devices and drugs—same or different?.
      ].
      Furthermore, the impact of organizational changes depends not only on the intervention content but also on their implementation. This is similar in pharmacoeconomics to the relationship of the efficacy of drugs to adherence to treatment [
      • Hiligsmann M.
      • Gathon H.J.
      • Bruyère O.
      • et al.
      Cost-effectiveness of osteoporosis screening followed by treatment: the impact of medication adherence.
      ]. Nevertheless, adherence can be managed in randomized controlled trials to study the effectiveness of the drug, whereas the deployment of an organizational change relates to personal behavior. Implementing behavioral changes is not insurmountable, but it makes the economic evaluation of interventions aimed at modifying organizational models challenging [
      • Craig P.
      • Dieppe P.
      • Macintyre S.
      • et al.
      Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance.
      ].
      The Deming cycle, also known as the Plan-Do-Check-Act (PDCA) cycle, is an iterative four-step management method used for the control and continuous improvement of processes and products. A fundamental principle of the scientific method and PDCA is iteration—once a hypothesis is confirmed (or negated), executing the cycle again will extend the knowledge further. Repeating the PDCA cycle can bring the goal closer [

      Moen R, Norman C. Evolution of the PDCA cycle. 2006. Available from: http://pkpinc.com/files/NA01MoenNormanFullpaper.pdf. [Accessed June 25, 2015].

      ], and the process itself helps to create a culture of critical thinkers [
      • Deming W.E.
      Out of the Crisis.
      ]. Compared with more traditional health care research methods, the PDCA cycle presents a pragmatic scientific method to address the implementation of organizational changes [

      Moen R, Norman C. Evolution of the PDCA cycle. 2006. Available from: http://pkpinc.com/files/NA01MoenNormanFullpaper.pdf. [Accessed June 25, 2015].

      ].
      The objective of the present study was to develop a framework for the management of complex interventions within the continuous improvement cycle over the long-term. The approach, although adaptable to other contexts and diseases, was tested with the case of an integrated health care intervention for multimorbid patients in the Donostialdea county in the Basque Country.

      Methods

      The Framework

      The transferability of randomized controlled trials in the context of complex interventions has arisen in the literature in recent years [
      • Craig P.
      • Dieppe P.
      • Macintyre S.
      • et al.
      Developing and evaluating complex interventions: the new Medical Research Council guidance.
      ]. Unlike in the field of pharmacoeconomics, the implementation of the intervention in this study depends on behavioral changes, and therefore the need to assess them in the daily routine emerges. Administrative claims databases can prove to be very useful in measuring resource use and costs [
      • Garrison Jr, L.P.
      • Neumann P.J.
      • Erickson P.
      • et al.
      Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report.
      ]. Furthermore, behavioral changes occur slowly, which implies that the designed framework needs to cover the mid- to long-term vision.
      A budget impact analysis (BIA) projects the burden of the target population in the conventional or baseline scenario and analyzes how this burden would change if the intervention achieved the organizationally defined goal. First, the BIA provides the long-term perspective. This approach also lends understanding of the economic burden of the disease, which is important for estimating future expenditures, especially in environments in which an aging population will make a difference. Finally, it helps explore the potential impact of the intervention [
      • Shaya F.T.
      • Mullins C.D.
      • Wong W.
      Incidence versus prevalence modeling in pharmacoeconomics.
      ,
      • Mauskopf J.A.
      • Sullivan S.D.
      • Annemans L.
      • et al.
      Principles of good practice for budget impact analysis: report of the ISPOR Task Force on Good Research Practices—Budget Impact Analysis.
      ] (Plan stage). Although a BIA can be carried out more simply than by dynamic simulation modeling, this technique is advantageous for representing the complexities of health systems [
      • Marshall D.A.
      • Burgos-Liz L.
      • IJzerman M.J.
      • et al.
      Applying dynamic simulation modeling methods in health care delivery research—the SIMULATE checklist: report of the ISPOR Simulation Modeling Emerging Good Practices Task Force.
      ]. Because discrete event simulation (DES) modeling handles time explicitly [
      • Caro J.J.
      • Möller J.
      • Karnon J.
      • et al.
      Discrete Event Simulation for.
      ], we think it is the most suitable dynamic model for carrying out BIA. Once the intervention is deployed (Do stage), a statistical analysis is needed to ascertain any changes in resource consumption in the subsequent years (Check stage). In addition, the real costs, together with the objective cost fixed in the Plan stage, will determine whether the trend is positive. The statistical analysis alone, although necessary, may not always represent the practical significance of the intervention. Thus, the true economic impact of the intervention can be ascertained by integrating the statistical results in the BIA. This approach provides direct and understandable information for the stakeholders [
      • Mauskopf J.
      Prevalence-based economic evaluation.
      ]. If the intervention achieves the objective, then that becomes the new standard (baseline) for the organization’s actions going forward. On the contrary, if the Check stage shows no improvement, then the existing standard remains and adjustments or correction actions should be done (Act stage). Figure 1 shows graphically the proposed framework for assessing complex interventions. It combines statistical analysis with the analysis of trends on the basis of what would have occurred 1) in the baseline scenario and 2) in an objective scenario.
      Fig. 1
      Fig. 1Description of the approach that integrates simulation modeling and statistical analysis in Deming’s continuous improvement cycle.

      Case Study: Integrated Health Care Intervention for Multimorbid Patients in the Basque Country

      An integrated care approach supported by information and communication technologies is being applied to determine how to best respond to the complex needs of multimorbid patients in the Basque Country as well as in six other European pilot sites participating in the CareWell project [

      CareWell. CareWell project: delivering integrated care to frail patients through ICT. Available from: http://www.carewell-project.eu/home/. [Accessed September 9, 2015].

      ]. The Basque Country approach is focused on a vertical integrated model of health care that refers to the delivery of primary and specialized care in a single health care organization [
      • Strandberg-Larsen M.
      • Krasnik A.
      Measurement of integrated healthcare delivery: a systematic review of methods and future research directions.
      ,
      • MacAdam M.
      Frameworks of Integrated Care for the Elderly: A Systematic Review.
      ]. This is described in depth in Supplemental Materials found at 10.1016/j.jval.2016.08.002.

      Conceptual model

      The natural history of multimorbidity is dynamic in persons, characterized by frequent transitions between stable and unstable states over time. In our study, during the stable state in which the patients stayed at home, they were cared for by primary care professionals. When patients decompensated and required additional attention, they were referred to secondary care [
      • Gill T.M.
      • Gahbauer E.A.
      • Allore H.G.
      • Han L.
      Transitions between frailty states among community-living older persons.
      ]. All patients who used hospital care were initially evaluated by the emergency department and were hospitalized only when the department deemed it necessary. After the patients’ conditions restabilized, they were discharged to their residence (Fig. 2).
      Fig. 2
      Fig. 2Description of the conceptual model. A&E, accident and emergency department; PC, primary care.
      A stratification strategy was set up in the Basque Country to identify those patients among the whole Basque population who were at high risk of hospitalization and to forecast health care utilization costs (costs of resource use and pharmacy). The strategy was based on the Adjusted Clinical Groups, a system that measures the morbidity burden of patient populations on the basis of disease patterns, age, and sex. It relies on the diagnostic and/or pharmaceutical code information in administrative databases to assign to each individual a risk score predicting resource consumption during the next year compared with the total stratified population [

      School of Public Health, Johns Hopkins University. The Johns Hopkins University ACG case-mix system. Available from: http://acg.jhsph.org/index.php?option=com_content&view=article&id=46&Itemid=366. [Accessed July 5, 2016].

      ,
      • Johns Hopkins Bloomberg School of Public Health
      The Johns Hopkins ACG Case-Mix System Technical Reference Guide Manual Version 9.0.
      ]. Higher risk foresees greater costs for the health care system. The process of obtaining the risk score is explained extensively elsewhere [
      • Orueta J.F.
      • Nuño-Solinis R.
      • Mateos M.
      • et al.
      Predictive risk modeling in the Spanish population: a cross-sectional study.
      ]. In our study, risk scores of 6.1 and higher were considered to be suitable for case management. The criteria to select the target population included the presence of two or more chronic conditions, such as diabetes mellitus, heart failure, or chronic obstructive pulmonary disease, and hospitalization at least once in the previous year. In the Donostialdea county, out of about 300,000 people, 1,113 multimorbid patients were eligible for the intervention in 2012.
      The intervention was an integrated care program comprising an interdisciplinary team including a general practitioner and a case manager, with the goal of reducing the risk of patient decompensation measured by accident and emergency service use and hospitalizations avoided. The integrated care model was implemented in 2012 and developed between 2013 and 2015; it has presently achieved 100% deployment [
      • Mora J.
      • de Manuel E.
      • Arratibel P.
      • et al.
      Planning of the local innovation process for the development of Population Intervention Plans in an integrated health system.
      ]. On the basis of studies in the literature [
      • Tappenden P.
      • Campbell F.
      • Rawdin A.
      • et al.
      The clinical effectiveness and cost-effectiveness of home-based, nurse-led health promotion for older people: a systematic review.
      ,
      • McLean S.
      • Nurmatov U.
      • Liu J.L.
      • et al.
      Telehealthcare for chronic obstructive pulmonary disease: Cochrane review and meta-analysis.
      ,
      • Bernabei R.
      • Landi F.
      • Gambassi G.
      • et al.
      Randomised trial of impact of model of integrated care and case management for older people living in the community.
      ], the Donostialdea Health Care Organization set its own objective with Delphi [

      Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. 2007. Available from: http://pareonline.net/pdf/v12n10.pdf. [Accessed June 28, 2015].

      ] methodology. Decision makers, clinicians, and epidemiologists were included in the study and they concluded that the intervention for integrated health care could reduce decompensations by an annual 2% beginning in 2014, with the goal of attaining a total 10% reduction in 5 years.

      Study design

      First, a DES model [
      • Karnon J.
      • Stahl J.
      • Brennan A.
      • et al.
      ISPOR-SMDM Modeling Good Research Practices Task Force. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–4.
      ] was built with the Arena Rockwell software v14 (Rockwell Automation, Milwaukee, WI 53204, EE. UU) to represent the care pathway for multimorbid patients, which was characterized by frequent transitions to decompensation states over time. The model outputs were consumption rates. By multiplying consumption rates in both scenarios by the unit costs (Table 1), we determined the cost of illness of multimorbid patients under both the conventional and integrated organizational systems. Combining the cost of illness under both organizational systems allowed us to calculate costs in the BIA.
      Table 1Unit costs for different services
      ServiceUnit cost (€)
      General practitioner (health center)27.2
      General practitioner (home)38.1
      General practitioner (telephone)13.6
      Primary care nurse (health center)12.0
      Primary care nurse (home)21.8
      Primary care nurse (telephone)6.0
      Emergency168.3
      In-hospitalization mean stay2273.7
      Home hospitalization2400.3
      Second, a statistical analysis was carried out on the basis of resource consumption calculated from each patient’s contacts in terms of rate and cost. A univariate statistical testing approach was first used, and then a multivariate analysis by general linear models was addressed [
      • Glick H.A.
      • Doshi J.A.
      • Sonnad S.S.
      • Daniel Polsky D.
      Economic Evaluation in.
      ].
      Finally, real costs in 2014 were compared with those calculated in the BIA. Comparing subsequent real costs with the calculated ones allowed us to analyze trends.

      Data sources

      Epidemiological data (prevalence and mortality) and resource consumption data were obtained from administrative databases. Incidence rates of multimorbid patients by age and sex could not be directly obtained from administrative databases. Knowing our population prevalence and mortality, we estimated incidence rates by age group using the Dismod II software, which is a tool created by the World Health Organization (Geneva, Switzerland) to measure the consistency of estimates of incidence, prevalence, duration, and case fatality for diseases [

      World Health Organization. EpiGear. Dismod II. Available from: http://www.epigear.com/index_files/dismod_ii.html. [Accessed June 28, 2014].

      ]. Costs were obtained from the Basque Health Service accounting system in 2013 [
      • Drummond M.F.
      Methods for the Economic Evaluation of Health Care Programmes.
      ,
      • Glick H.A.
      • Doshi J.A.
      • Sonnad S.S.
      • Polsky D.
      Economic Evaluation in Clinical Trials.
      ]. Projections of the National Institute of Statistics of Spain [

      INEbase. Demografía y población. Cifras de población y censos demográficos. Available from: http://www.ine.es/inebmenu/mnu_cifraspob.htm. [Accessed April 9, 2014].

      ] were used to determine the Spanish multimorbid population between 2015 and 2020.

      Results

      The main results of the BIA are presented in Table 2. In the first row, we show how the multimorbid patient population will grow according to the aging population. We also show how contacts with primary care (general practitioners and nurses), accident and emergency, and hospitalizations will evolve over time. Finally, we show the costs of these contacts. Considering the aging of the general population, the multimorbid patient population in the Donostialdea county will increase by 8% by 2020. In addition, because the target population is not only larger but also older, the expenses will have increased by 21% under conventional health care. Nevertheless, if interventions were successful and reduced emergencies by an annual 2%, this budget would decrease to 18%, with cumulative savings of more than half a million euros during the study period (Table 2 and Fig. 3A).
      Table 2Extrapolation till 2020 of the target population, resource use, and costs both in standard and in objective scenarios
      Year20132014201520162017201820192020
      Prevalence1,1481,1581,1691,1941,1981,2271,2351,250
      Resource consumption (contacts)
      Traditional health care
       PC
        General practitioner16,09616,89017,36317,18317,89718,07518,29518,408
        Nurse9,7539,99710,29510,85310,68810,97711,38111,743
       A&E3,8703,9654,0774,2834,4414,5994,7764,949
       Hospitalizations8468578749169569921,0311,063
      Integrated health care
       PC
        General practitioner16,09617,77517,85018,17618,35518,48019,08719,079
        Nurse9,7539,85310,0999,91610,05610,61710,58211,145
       A&E3,8703,8943,9914,1654,3034,4194,5424,681
       Hospitalizations8468428568959309529831,011
      Costs
      Traditional health care
       PC591,864637,466642,492648,663657,100661,612666,483672,277
       A&E and hospitalization2,790,6412,815,0392,879,7413,014,7043,144,2433,264,6863,387,9483,501,821
       Total3,382,5053,452,5053,522,2333,663,3673,801,3433,926,2984,054,4314,174,098
      Integrated health care
       PC591,864642,455647,203657,318657,197665,635674,134680,980
       A&E and hospitalization2,790,6412,764,1852,819,5412,947,0983,057,1023,133,8453,234,6303,328,886
       Total3,382,5053,406,6403,466,7443,604,4163,714,2993,799,4803,908,7644,009,866
      A&E, accident and emergency department; PC, primary care.
      Fig. 3
      Fig. 3Budget impact analysis: (A) Plan stage. (B) Check stage including RWD for 2014 and hypothetical costs for the following years. RWD, real-world data.
      By combining the results of the statistical analysis (see Supplemental Materials) that showed no change in the resource consumption by 2014 and that of the BIA, we provided new insights about the implementation of the integrated intervention. Figure 3B shows the real burden in 2014 and how the points representing the following years (2015, 2016, etc.) could hypothetically evolve. Because the points have not moved closer to the objective line, we can state that deployment and/or intervention must be reconsidered to begin the planning process again.

      Conclusions

      The Deming cycle, together with statistical analysis, is a well-known tool for health care management, but to our knowledge this work introduces for the first time the BIA in the PDCA cycle for continuous improvement of complex interventions.
      Complex interventions are different from pharmacological interventions in many ways. First, complex interventions are usually formed by various components, hindering the identification of the target population. Moreover, complex interventions follow a nonlinear pattern, which complicates foreseeing the intervention’s likely harms and benefits. But, probably the most relevant difference is that implementation of complex interventions relies on behavioral changes that are often subjected to learning curves. If the intervention is evaluated too early in time, that is, before it is sufficiently implemented, we may state that the program did not work. Because economic evaluations based only on early appraisals can be misleading [
      • Drummond M.
      • Griffin A.
      • Tarricone R.
      Economic evaluation for devices and drugs—same or different?.
      ], an iterative approach should be taken. The four stages described in Deming’s continuous improvement cycle mirror the scientific experimental method by formulating a BIA, collecting data to test the hypothesis, analyzing and interpreting the results, and making inferences to iterate the hypothesis [
      • Speroff T.
      • O’Connor G.T.
      Study designs for PDSA quality improvement research.
      ]. Moreover, the iterative approach of the continuous improvement cycle should bring us closer to the goal [

      Moen R, Norman C. Evolution of the PDCA cycle. 2006. Available from: http://pkpinc.com/files/NA01MoenNormanFullpaper.pdf. [Accessed June 25, 2015].

      ]. This is consistent with the point of view of Drummond et al. [
      • Drummond M.
      • Griffin A.
      • Tarricone R.
      Economic evaluation for devices and drugs—same or different?.
      ] who suggest taking an interactive approach to the clinical and economic evaluation of devices by revising the expected results as increasing evidence of effectiveness in actual use is collected.
      Scientific conclusions and business or policy decisions should not be based only on statistical significance. Pragmatic considerations often require binary “yes-no” decisions, but this does not mean that P values alone can ensure that a decision is correct or incorrect. Moreover, as reported by the American Statistical Association, statistical significance is not equivalent to scientific, human, or economic significance, and so it does not measure the size of an effect or the importance of the result [
      • Wasserstein R.L.
      • Lazara N.A.
      The ASA’s statement on p-values: context, process, and purpose.
      ]. Decision makers need a better explanation of the practical relevance [
      • Hoffmann C.
      • Graf von der Schulenburg J.M.
      The influence of economic evaluation studies on decision making: a European survey.
      ]. BIA translates the results of the statistical analysis into the budget, providing direct and understandable information for the stakeholders [
      • Mauskopf J.
      Prevalence-based economic evaluation.
      ].
      The inclusion of BIA in Deming’s continuous improvement cycle has a triple aim. First, the BIA will provide understanding of the economic burden of the disease. Second, it will help to explore the potential impact of the intervention, according to organizationally defined goals [
      • Shaya F.T.
      • Mullins C.D.
      • Wong W.
      Incidence versus prevalence modeling in pharmacoeconomics.
      ,
      • Mauskopf J.A.
      • Sullivan S.D.
      • Annemans L.
      • et al.
      Principles of good practice for budget impact analysis: report of the ISPOR Task Force on Good Research Practices—Budget Impact Analysis.
      ]. This may be relevant because, as previously noted, translation of the statistical analysis into the budget directly provides the stakeholders with understandable information [
      • Mauskopf J.
      Prevalence-based economic evaluation.
      ]. Finally, BIA provides a medium- to long-term horizon for analyzing trends. This gives us a broader perspective in assessing whether we are on track. Comparison of the real resource consumption with the expected values over time allows a comparison of the deviation between the goals determined by the BIA and the present events occurring at each of the stages. If the results begin to agree with the objective over time, it will suggest that work is progressing in the right direction. If, however, the results move further away from the objective, as shown in this case, the deployment and/or the intervention should be reconsidered. If the primary statistical analysis shows positive results, a new BIA would have to be performed to compare the conventional and integrated health care interventions. The inclusion of organizationally set objectives using qualitative methods such as Delphi studies has various advantages. On one hand, objective setting is fundamental for continuous improvement [
      • Bodenheimer T.
      • Handley M.A.
      Goal-setting for behavior change in primary care: an exploration and status report.
      ], and on the other hand, it allows the implementation process to be tailored to the characteristics of the organization.
      The BIA may be carried out via several approaches. The simplest one would be to assume that the rate of resource use per person would remain constant over the study period for each age group. Costs would be obtained by multiplying the resource consumption by the number of individuals in each age group. Nevertheless, a more sophisticated approach, such as DES, would provide more accurate results because it would enable the representation of the natural history of the disease. The virtues of dynamic models to represent complex systems have been recently highlighted in a report of the International Society for Pharmacoeconomics and Outcomes Research [
      • Marshall D.A.
      • Burgos-Liz L.
      • IJzerman M.J.
      • et al.
      Applying dynamic simulation modeling methods in health care delivery research—the SIMULATE checklist: report of the ISPOR Simulation Modeling Emerging Good Practices Task Force.
      ]. This application of simulation modeling was also considered in a report addressed to Barack Obama, the president of the United States, in May 2014; an expert task force highlighted the uses of such engineering tools to improve management of health systems [

      Executive Office of the President. President’s Council of Advisors on Science and Technology. Report to the President. Better health care and lower costs: accelerating improvement through systems engineering. Available from: https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_systems_engineering_in_healthcare_-_may_2014.pdf. [Accessed September 10, 2015].

      ]. Among the different dynamic models, DES seems particularly adequate because it handles time explicitly, which is a fundamental requirement for this study. Its flexibility also makes our approach more generalizable, representing models of both simple and complex levels of interaction.
      The case study shown in this article is a practical application of the approach. The BIA allowed us to estimate the burden of multimorbid patients that would surpass €4 million during the study period. Furthermore, it anticipated the increase in cost of care for multimorbid patients because of aging in the Donostialdea county (Basque Country). It also showed the cost savings if the program achieved the organizationally set objective of reducing unstable conditions in patients by an annual rate of 2%. This was quantified in cumulative savings of more than half a million euros. Decision makers were thus able to assess in advance the size of the change they could expect from the deployment of the integrated program in terms of budget expenditure. The fact that the rate of primary care consultation costs did not increase in the study period suggests that the intervention has not been sufficiently implemented. With the passage of time and implementation improvement, it would be possible to analyze trends. Representing the care process and the natural history of multimorbid patients with DES allows prediction of the economic burden associated with that population in the Donostialdea county. This was made possible by the use of data and tools with very different origins. We combined clinical evolution, resource consumption, demographic trends, and epidemiological data obtained with the Dismod II software, parametric survival analysis, economic evaluation, and simulation to carry out a BIA to inform the planning stage of the Deming cycle.
      The framework developed within the CareWell project will help its pilot sites to manage the implementation of interventions aimed at maintaining long-term stability of multimorbid patients and assess their outcomes. By setting objectives based on evidence and including them in the BIA, managers can evaluate whether the integrated health care intervention is having the expected impact. This approach, however, has a broad scope and is not limited to the management of integrated health care interventions focused on improving care for multimorbid patients. In fact, by tailoring the conceptual model of the BIA, this approach could be used to determine the adequacy of any complex intervention for which time and implementation are key issues.

      Acknowledgment

      We acknowledge the English editorial assistance provided by Sally Ebeling.
      Source of financial support: All the authors were funded by their institutions. CareWell project is cofunded by the European Commission within the Information Communication Technologies Policy Support Programme of the Competitiveness and Innovation Framework Programme (grant agreement no. 620983).
      CareWell Project Group (partner: members) Kronikgune, Basque Country, Spain: Esteban de Manuel, Joana Mora, Ane Fullaondo. Osakidetza-Basque Health Service, Spain: Maria Luisa Merino, Angel Faria, Itziar Vergara, Javier Mar. Health Information Management, Belgium: Panagiotis Stafylas, Marco D'Angelantonio, Maite Hurtado. Faculty of Electrical Engineering and Computig, University of Zagreb, Croatia:Mario Kovac, Leon Dragic. Croatian Society for Pharmacoeconomics and Health Economics, Croatia: Vanesa Benkovic. Ericsson Nikola Tesla, Croatia: Karlo Gustin, Mario Ravic. Regional Healthcare Agency of Puglia, Italy: Elisabetta Anna Graps, Francesca Avolio. Local Health Authority nr.2 of Feltre, Italy: Francesco Marchet. Veneto's Research Centre for Health Innovation, Italy: Claudio Dario, Silvia Mancin, Claudio Saccavini. Lower Silesian Marshal’s Office, Poland: Antoni Zwiefka. Wales Powys Teaching Health Board, United Kingdom: Ian Green, Andrew Rogers, Tanya Summerfield. Empirica, Germany: Reinhard Hammerschmidt, Veli Stroetmann. Region of Southern Denmark, Denmark: Signe Daugbjerg. International Foundation for Integrated Care: Leo Lewis.

      Supplemental Materials

      References

        • Boyd C.M.
        • Fortin M.
        Future of multimorbidity research: how should understanding of multimorbidity inform health system design.
        Public Health Rev. 2010; 32: 451-474
        • Allepuz Palau A.
        • Piñeiro Méndez P.
        • Molina Hinojosa J.C.
        • et al.
        Evaluación económica de un programa de coordinación entre niveles para el manejo de pacientes crónicos complejos.
        Aten Primaria. 2015; 47: 134-140
      1. Goodwin N, Smith J, Davies A, et al. Integrated care for patients and populations: improving outcomes by working together. 2012. Available from: http://www.kingsfund.org.uk/publications/future_forum_report.html. [Accessed February 13, 2015].

        • Ham C.
        Chronic care in the English National Health Service: progress and challenges.
        Health Aff (Millwood). 2009; 28: 190-201
        • Berwick D.M.
        Disseminating innovations in health care.
        JAMA. 2003; 289: 1969-1975
        • Drummond M.
        • Griffin A.
        • Tarricone R.
        Economic evaluation for devices and drugs—same or different?.
        Value Health. 2009; 12: 402-404
        • Hiligsmann M.
        • Gathon H.J.
        • Bruyère O.
        • et al.
        Cost-effectiveness of osteoporosis screening followed by treatment: the impact of medication adherence.
        Value Health. 2010; 13: 394-401
        • Craig P.
        • Dieppe P.
        • Macintyre S.
        • et al.
        Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance.
        BMJ. 2008; 337: a1655
      2. Moen R, Norman C. Evolution of the PDCA cycle. 2006. Available from: http://pkpinc.com/files/NA01MoenNormanFullpaper.pdf. [Accessed June 25, 2015].

        • Deming W.E.
        Out of the Crisis.
        Massachusetts Institute of Technology-Center for Advanced Engineering Study, Cambridge, MA1986: 510
        • Craig P.
        • Dieppe P.
        • Macintyre S.
        • et al.
        Developing and evaluating complex interventions: the new Medical Research Council guidance.
        Int J Nurs Stud. 2013; 50: 587-592
        • Garrison Jr, L.P.
        • Neumann P.J.
        • Erickson P.
        • et al.
        Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report.
        Value Health. 2007; 10: 326-335
        • Shaya F.T.
        • Mullins C.D.
        • Wong W.
        Incidence versus prevalence modeling in pharmacoeconomics.
        Expert Rev Pharmacoecon Outcomes Res. 2002; 2: 435-442
        • Mauskopf J.A.
        • Sullivan S.D.
        • Annemans L.
        • et al.
        Principles of good practice for budget impact analysis: report of the ISPOR Task Force on Good Research Practices—Budget Impact Analysis.
        Value Health. 2007; 10: 336-347
        • Marshall D.A.
        • Burgos-Liz L.
        • IJzerman M.J.
        • et al.
        Applying dynamic simulation modeling methods in health care delivery research—the SIMULATE checklist: report of the ISPOR Simulation Modeling Emerging Good Practices Task Force.
        Value Health. 2015; 18: 5-16
        • Caro J.J.
        • Möller J.
        • Karnon J.
        • et al.
        Discrete Event Simulation for.
        Health Technology Assessment. CRC Press, Boca Raton, FL2015
        • Mauskopf J.
        Prevalence-based economic evaluation.
        Value Health. 1998; 1: 251-259
      3. CareWell. CareWell project: delivering integrated care to frail patients through ICT. Available from: http://www.carewell-project.eu/home/. [Accessed September 9, 2015].

        • Strandberg-Larsen M.
        • Krasnik A.
        Measurement of integrated healthcare delivery: a systematic review of methods and future research directions.
        Int J Integr Care. 2009; 9: e01
        • MacAdam M.
        Frameworks of Integrated Care for the Elderly: A Systematic Review.
        CPRN Research Report. Canadian Policy Research Networks/Réseaux canadiens de recherche en politiques publiques, 2008 (Available from: http://www.cprn.org/documents/49813_EN.pdf)
        • Gill T.M.
        • Gahbauer E.A.
        • Allore H.G.
        • Han L.
        Transitions between frailty states among community-living older persons.
        Arch Intern Med. 2006; 166: 418-423
      4. School of Public Health, Johns Hopkins University. The Johns Hopkins University ACG case-mix system. Available from: http://acg.jhsph.org/index.php?option=com_content&view=article&id=46&Itemid=366. [Accessed July 5, 2016].

        • Johns Hopkins Bloomberg School of Public Health
        The Johns Hopkins ACG Case-Mix System Technical Reference Guide Manual Version 9.0.
        Johns Hopkins Bloomberg School of Public Health, Baltimore, MD2009
        • Orueta J.F.
        • Nuño-Solinis R.
        • Mateos M.
        • et al.
        Predictive risk modeling in the Spanish population: a cross-sectional study.
        BMC Health Serv Res. 2013; 13: 269
        • Mora J.
        • de Manuel E.
        • Arratibel P.
        • et al.
        Planning of the local innovation process for the development of Population Intervention Plans in an integrated health system.
        Int J Integr Care. 2013; 13: 1-2
        • Tappenden P.
        • Campbell F.
        • Rawdin A.
        • et al.
        The clinical effectiveness and cost-effectiveness of home-based, nurse-led health promotion for older people: a systematic review.
        Health Technol Assess. 2012; 16: 1-72
        • McLean S.
        • Nurmatov U.
        • Liu J.L.
        • et al.
        Telehealthcare for chronic obstructive pulmonary disease: Cochrane review and meta-analysis.
        Br J Gen Pract. 2012; 62: e739-e749
        • Bernabei R.
        • Landi F.
        • Gambassi G.
        • et al.
        Randomised trial of impact of model of integrated care and case management for older people living in the community.
        BMJ. 1998; 316: 1348-1351
      5. Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. 2007. Available from: http://pareonline.net/pdf/v12n10.pdf. [Accessed June 28, 2015].

        • Karnon J.
        • Stahl J.
        • Brennan A.
        • et al.
        ISPOR-SMDM Modeling Good Research Practices Task Force. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–4.
        Value Health. 2012; 15: 821-827
        • Glick H.A.
        • Doshi J.A.
        • Sonnad S.S.
        • Daniel Polsky D.
        Economic Evaluation in.
        Clinical Trials. 2nd ed. Oxford University Press, Oxford, UK2014
      6. World Health Organization. EpiGear. Dismod II. Available from: http://www.epigear.com/index_files/dismod_ii.html. [Accessed June 28, 2014].

        • Drummond M.F.
        Methods for the Economic Evaluation of Health Care Programmes.
        Oxford University Press, Oxford, UK2005
        • Glick H.A.
        • Doshi J.A.
        • Sonnad S.S.
        • Polsky D.
        Economic Evaluation in Clinical Trials.
        Oxford University Press, Oxford, UK2014
      7. INEbase. Demografía y población. Cifras de población y censos demográficos. Available from: http://www.ine.es/inebmenu/mnu_cifraspob.htm. [Accessed April 9, 2014].

        • Speroff T.
        • O’Connor G.T.
        Study designs for PDSA quality improvement research.
        Qual Manag Health Care. 2004; 13: 17-32
        • Wasserstein R.L.
        • Lazara N.A.
        The ASA’s statement on p-values: context, process, and purpose.
        Am Stat. 2016; 70: 129-133
        • Hoffmann C.
        • Graf von der Schulenburg J.M.
        The influence of economic evaluation studies on decision making: a European survey.
        The EUROMET group. Health Policy. 2000; 52: 179-192
        • Bodenheimer T.
        • Handley M.A.
        Goal-setting for behavior change in primary care: an exploration and status report.
        Patient Educ Couns. 2009; 76: 174-180
      8. Executive Office of the President. President’s Council of Advisors on Science and Technology. Report to the President. Better health care and lower costs: accelerating improvement through systems engineering. Available from: https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_systems_engineering_in_healthcare_-_may_2014.pdf. [Accessed September 10, 2015].