If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Incorporating Budget Impact Analysis in the Implementation of Complex Interventions: A Case of an Integrated Intervention for Multimorbid Patients within the CareWell Study
AP-OSI Research Unit, Alto Deba Integrated Health Care Organization, Mondragon, SpainHealth Services Research on Chronic Patients Network (REDISSEC), Kronikgune Group, Barakaldo, SpainBiodonostia Health Research Institute, San Sebastian-Donostia, Spain
AP-OSI Research Unit, Alto Deba Integrated Health Care Organization, Mondragon, SpainHealth Services Research on Chronic Patients Network (REDISSEC), Kronikgune Group, Barakaldo, SpainBiodonostia Health Research Institute, San Sebastian-Donostia, SpainClinical Management Unit, Alto Deba Integrated Health Care Organization, Mondragon, Spain
To develop a framework for the management of complex health care interventions within the Deming continuous improvement cycle and to test the framework in the case of an integrated intervention for multimorbid patients in the Basque Country within the CareWell project.
Methods
Statistical analysis alone, although necessary, may not always represent the practical significance of the intervention. Thus, to ascertain the true economic impact of the intervention, the statistical results can be integrated into the budget impact analysis. The intervention of the case study consisted of a comprehensive approach that integrated new provider roles and new technological infrastructure for multimorbid patients, with the aim of reducing patient decompensations by 10% over 5 years. The study period was 2012 to 2020.
Results
Given the aging of the general population, the conventional scenario predicts an increase of 21% in the health care budget for care of multimorbid patients during the study period. With a successful intervention, this figure should drop to 18%. The statistical analysis, however, showed no significant differences in costs either in primary care or in hospital care between 2012 and 2014. The real costs in 2014 were by far closer to those in the conventional scenario than to the reductions expected in the objective scenario. The present implementation should be reappraised, because the present expenditure did not move closer to the objective budget.
Conclusions
This work demonstrates the capacity of budget impact analysis to enhance the implementation of complex interventions. Its integration in the context of the continuous improvement cycle is transferable to other contexts in which implementation depth and time are important.
The increasing prevalence of chronic diseases mainly because of an aging population has led to a profound change in the paradigm of health care. Approximately one in four adults have two or more chronic conditions, and half of older adults have three or more [
]. Therefore, health systems have changed in perspective, and health care organizations previously concerned mainly with treating acute problems are now focused on a continuum-of-care approach [
]. Nevertheless, organizations are dynamic, and interventions that require behavioral changes are difficult to implement. As first shown in 1943, the adoption curve of an innovation has an S shape, with a slow early phase affecting very few people, a rapid middle phase spreading widely, and a slow third phase ending with incomplete penetration [
Furthermore, the impact of organizational changes depends not only on the intervention content but also on their implementation. This is similar in pharmacoeconomics to the relationship of the efficacy of drugs to adherence to treatment [
]. Nevertheless, adherence can be managed in randomized controlled trials to study the effectiveness of the drug, whereas the deployment of an organizational change relates to personal behavior. Implementing behavioral changes is not insurmountable, but it makes the economic evaluation of interventions aimed at modifying organizational models challenging [
The Deming cycle, also known as the Plan-Do-Check-Act (PDCA) cycle, is an iterative four-step management method used for the control and continuous improvement of processes and products. A fundamental principle of the scientific method and PDCA is iteration—once a hypothesis is confirmed (or negated), executing the cycle again will extend the knowledge further. Repeating the PDCA cycle can bring the goal closer [
]. Compared with more traditional health care research methods, the PDCA cycle presents a pragmatic scientific method to address the implementation of organizational changes [
The objective of the present study was to develop a framework for the management of complex interventions within the continuous improvement cycle over the long-term. The approach, although adaptable to other contexts and diseases, was tested with the case of an integrated health care intervention for multimorbid patients in the Donostialdea county in the Basque Country.
Methods
The Framework
The transferability of randomized controlled trials in the context of complex interventions has arisen in the literature in recent years [
]. Unlike in the field of pharmacoeconomics, the implementation of the intervention in this study depends on behavioral changes, and therefore the need to assess them in the daily routine emerges. Administrative claims databases can prove to be very useful in measuring resource use and costs [
]. Furthermore, behavioral changes occur slowly, which implies that the designed framework needs to cover the mid- to long-term vision.
A budget impact analysis (BIA) projects the burden of the target population in the conventional or baseline scenario and analyzes how this burden would change if the intervention achieved the organizationally defined goal. First, the BIA provides the long-term perspective. This approach also lends understanding of the economic burden of the disease, which is important for estimating future expenditures, especially in environments in which an aging population will make a difference. Finally, it helps explore the potential impact of the intervention [
] (Plan stage). Although a BIA can be carried out more simply than by dynamic simulation modeling, this technique is advantageous for representing the complexities of health systems [
Applying dynamic simulation modeling methods in health care delivery research—the SIMULATE checklist: report of the ISPOR Simulation Modeling Emerging Good Practices Task Force.
], we think it is the most suitable dynamic model for carrying out BIA. Once the intervention is deployed (Do stage), a statistical analysis is needed to ascertain any changes in resource consumption in the subsequent years (Check stage). In addition, the real costs, together with the objective cost fixed in the Plan stage, will determine whether the trend is positive. The statistical analysis alone, although necessary, may not always represent the practical significance of the intervention. Thus, the true economic impact of the intervention can be ascertained by integrating the statistical results in the BIA. This approach provides direct and understandable information for the stakeholders [
]. If the intervention achieves the objective, then that becomes the new standard (baseline) for the organization’s actions going forward. On the contrary, if the Check stage shows no improvement, then the existing standard remains and adjustments or correction actions should be done (Act stage). Figure 1 shows graphically the proposed framework for assessing complex interventions. It combines statistical analysis with the analysis of trends on the basis of what would have occurred 1) in the baseline scenario and 2) in an objective scenario.
Fig. 1Description of the approach that integrates simulation modeling and statistical analysis in Deming’s continuous improvement cycle.
Case Study: Integrated Health Care Intervention for Multimorbid Patients in the Basque Country
An integrated care approach supported by information and communication technologies is being applied to determine how to best respond to the complex needs of multimorbid patients in the Basque Country as well as in six other European pilot sites participating in the CareWell project [
CareWell. CareWell project: delivering integrated care to frail patients through ICT. Available from: http://www.carewell-project.eu/home/. [Accessed September 9, 2015].
]. The Basque Country approach is focused on a vertical integrated model of health care that refers to the delivery of primary and specialized care in a single health care organization [
]. This is described in depth in Supplemental Materials found at 10.1016/j.jval.2016.08.002.
Conceptual model
The natural history of multimorbidity is dynamic in persons, characterized by frequent transitions between stable and unstable states over time. In our study, during the stable state in which the patients stayed at home, they were cared for by primary care professionals. When patients decompensated and required additional attention, they were referred to secondary care [
]. All patients who used hospital care were initially evaluated by the emergency department and were hospitalized only when the department deemed it necessary. After the patients’ conditions restabilized, they were discharged to their residence (Fig. 2).
Fig. 2Description of the conceptual model. A&E, accident and emergency department; PC, primary care.
A stratification strategy was set up in the Basque Country to identify those patients among the whole Basque population who were at high risk of hospitalization and to forecast health care utilization costs (costs of resource use and pharmacy). The strategy was based on the Adjusted Clinical Groups, a system that measures the morbidity burden of patient populations on the basis of disease patterns, age, and sex. It relies on the diagnostic and/or pharmaceutical code information in administrative databases to assign to each individual a risk score predicting resource consumption during the next year compared with the total stratified population [
]. In our study, risk scores of 6.1 and higher were considered to be suitable for case management. The criteria to select the target population included the presence of two or more chronic conditions, such as diabetes mellitus, heart failure, or chronic obstructive pulmonary disease, and hospitalization at least once in the previous year. In the Donostialdea county, out of about 300,000 people, 1,113 multimorbid patients were eligible for the intervention in 2012.
The intervention was an integrated care program comprising an interdisciplinary team including a general practitioner and a case manager, with the goal of reducing the risk of patient decompensation measured by accident and emergency service use and hospitalizations avoided. The integrated care model was implemented in 2012 and developed between 2013 and 2015; it has presently achieved 100% deployment [
Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. 2007. Available from: http://pareonline.net/pdf/v12n10.pdf. [Accessed June 28, 2015].
] methodology. Decision makers, clinicians, and epidemiologists were included in the study and they concluded that the intervention for integrated health care could reduce decompensations by an annual 2% beginning in 2014, with the goal of attaining a total 10% reduction in 5 years.
ISPOR-SMDM Modeling Good Research Practices Task Force. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–4.
] was built with the Arena Rockwell software v14 (Rockwell Automation, Milwaukee, WI 53204, EE. UU) to represent the care pathway for multimorbid patients, which was characterized by frequent transitions to decompensation states over time. The model outputs were consumption rates. By multiplying consumption rates in both scenarios by the unit costs (Table 1), we determined the cost of illness of multimorbid patients under both the conventional and integrated organizational systems. Combining the cost of illness under both organizational systems allowed us to calculate costs in the BIA.
Second, a statistical analysis was carried out on the basis of resource consumption calculated from each patient’s contacts in terms of rate and cost. A univariate statistical testing approach was first used, and then a multivariate analysis by general linear models was addressed [
Finally, real costs in 2014 were compared with those calculated in the BIA. Comparing subsequent real costs with the calculated ones allowed us to analyze trends.
Data sources
Epidemiological data (prevalence and mortality) and resource consumption data were obtained from administrative databases. Incidence rates of multimorbid patients by age and sex could not be directly obtained from administrative databases. Knowing our population prevalence and mortality, we estimated incidence rates by age group using the Dismod II software, which is a tool created by the World Health Organization (Geneva, Switzerland) to measure the consistency of estimates of incidence, prevalence, duration, and case fatality for diseases [
] were used to determine the Spanish multimorbid population between 2015 and 2020.
Results
The main results of the BIA are presented in Table 2. In the first row, we show how the multimorbid patient population will grow according to the aging population. We also show how contacts with primary care (general practitioners and nurses), accident and emergency, and hospitalizations will evolve over time. Finally, we show the costs of these contacts. Considering the aging of the general population, the multimorbid patient population in the Donostialdea county will increase by 8% by 2020. In addition, because the target population is not only larger but also older, the expenses will have increased by 21% under conventional health care. Nevertheless, if interventions were successful and reduced emergencies by an annual 2%, this budget would decrease to 18%, with cumulative savings of more than half a million euros during the study period (Table 2 and Fig. 3A).
Table 2Extrapolation till 2020 of the target population, resource use, and costs both in standard and in objective scenarios
Year
2013
2014
2015
2016
2017
2018
2019
2020
Prevalence
1,148
1,158
1,169
1,194
1,198
1,227
1,235
1,250
Resource consumption (contacts)
Traditional health care
PC
General practitioner
16,096
16,890
17,363
17,183
17,897
18,075
18,295
18,408
Nurse
9,753
9,997
10,295
10,853
10,688
10,977
11,381
11,743
A&E
3,870
3,965
4,077
4,283
4,441
4,599
4,776
4,949
Hospitalizations
846
857
874
916
956
992
1,031
1,063
Integrated health care
PC
General practitioner
16,096
17,775
17,850
18,176
18,355
18,480
19,087
19,079
Nurse
9,753
9,853
10,099
9,916
10,056
10,617
10,582
11,145
A&E
3,870
3,894
3,991
4,165
4,303
4,419
4,542
4,681
Hospitalizations
846
842
856
895
930
952
983
1,011
Costs
Traditional health care
PC
591,864
637,466
642,492
648,663
657,100
661,612
666,483
672,277
A&E and hospitalization
2,790,641
2,815,039
2,879,741
3,014,704
3,144,243
3,264,686
3,387,948
3,501,821
Total
3,382,505
3,452,505
3,522,233
3,663,367
3,801,343
3,926,298
4,054,431
4,174,098
Integrated health care
PC
591,864
642,455
647,203
657,318
657,197
665,635
674,134
680,980
A&E and hospitalization
2,790,641
2,764,185
2,819,541
2,947,098
3,057,102
3,133,845
3,234,630
3,328,886
Total
3,382,505
3,406,640
3,466,744
3,604,416
3,714,299
3,799,480
3,908,764
4,009,866
A&E, accident and emergency department; PC, primary care.
Fig. 3Budget impact analysis: (A) Plan stage. (B) Check stage including RWD for 2014 and hypothetical costs for the following years. RWD, real-world data.
By combining the results of the statistical analysis (see Supplemental Materials) that showed no change in the resource consumption by 2014 and that of the BIA, we provided new insights about the implementation of the integrated intervention. Figure 3B shows the real burden in 2014 and how the points representing the following years (2015, 2016, etc.) could hypothetically evolve. Because the points have not moved closer to the objective line, we can state that deployment and/or intervention must be reconsidered to begin the planning process again.
Conclusions
The Deming cycle, together with statistical analysis, is a well-known tool for health care management, but to our knowledge this work introduces for the first time the BIA in the PDCA cycle for continuous improvement of complex interventions.
Complex interventions are different from pharmacological interventions in many ways. First, complex interventions are usually formed by various components, hindering the identification of the target population. Moreover, complex interventions follow a nonlinear pattern, which complicates foreseeing the intervention’s likely harms and benefits. But, probably the most relevant difference is that implementation of complex interventions relies on behavioral changes that are often subjected to learning curves. If the intervention is evaluated too early in time, that is, before it is sufficiently implemented, we may state that the program did not work. Because economic evaluations based only on early appraisals can be misleading [
], an iterative approach should be taken. The four stages described in Deming’s continuous improvement cycle mirror the scientific experimental method by formulating a BIA, collecting data to test the hypothesis, analyzing and interpreting the results, and making inferences to iterate the hypothesis [
] who suggest taking an interactive approach to the clinical and economic evaluation of devices by revising the expected results as increasing evidence of effectiveness in actual use is collected.
Scientific conclusions and business or policy decisions should not be based only on statistical significance. Pragmatic considerations often require binary “yes-no” decisions, but this does not mean that P values alone can ensure that a decision is correct or incorrect. Moreover, as reported by the American Statistical Association, statistical significance is not equivalent to scientific, human, or economic significance, and so it does not measure the size of an effect or the importance of the result [
The inclusion of BIA in Deming’s continuous improvement cycle has a triple aim. First, the BIA will provide understanding of the economic burden of the disease. Second, it will help to explore the potential impact of the intervention, according to organizationally defined goals [
]. This may be relevant because, as previously noted, translation of the statistical analysis into the budget directly provides the stakeholders with understandable information [
]. Finally, BIA provides a medium- to long-term horizon for analyzing trends. This gives us a broader perspective in assessing whether we are on track. Comparison of the real resource consumption with the expected values over time allows a comparison of the deviation between the goals determined by the BIA and the present events occurring at each of the stages. If the results begin to agree with the objective over time, it will suggest that work is progressing in the right direction. If, however, the results move further away from the objective, as shown in this case, the deployment and/or the intervention should be reconsidered. If the primary statistical analysis shows positive results, a new BIA would have to be performed to compare the conventional and integrated health care interventions. The inclusion of organizationally set objectives using qualitative methods such as Delphi studies has various advantages. On one hand, objective setting is fundamental for continuous improvement [
], and on the other hand, it allows the implementation process to be tailored to the characteristics of the organization.
The BIA may be carried out via several approaches. The simplest one would be to assume that the rate of resource use per person would remain constant over the study period for each age group. Costs would be obtained by multiplying the resource consumption by the number of individuals in each age group. Nevertheless, a more sophisticated approach, such as DES, would provide more accurate results because it would enable the representation of the natural history of the disease. The virtues of dynamic models to represent complex systems have been recently highlighted in a report of the International Society for Pharmacoeconomics and Outcomes Research [
Applying dynamic simulation modeling methods in health care delivery research—the SIMULATE checklist: report of the ISPOR Simulation Modeling Emerging Good Practices Task Force.
]. This application of simulation modeling was also considered in a report addressed to Barack Obama, the president of the United States, in May 2014; an expert task force highlighted the uses of such engineering tools to improve management of health systems [
]. Among the different dynamic models, DES seems particularly adequate because it handles time explicitly, which is a fundamental requirement for this study. Its flexibility also makes our approach more generalizable, representing models of both simple and complex levels of interaction.
The case study shown in this article is a practical application of the approach. The BIA allowed us to estimate the burden of multimorbid patients that would surpass €4 million during the study period. Furthermore, it anticipated the increase in cost of care for multimorbid patients because of aging in the Donostialdea county (Basque Country). It also showed the cost savings if the program achieved the organizationally set objective of reducing unstable conditions in patients by an annual rate of 2%. This was quantified in cumulative savings of more than half a million euros. Decision makers were thus able to assess in advance the size of the change they could expect from the deployment of the integrated program in terms of budget expenditure. The fact that the rate of primary care consultation costs did not increase in the study period suggests that the intervention has not been sufficiently implemented. With the passage of time and implementation improvement, it would be possible to analyze trends. Representing the care process and the natural history of multimorbid patients with DES allows prediction of the economic burden associated with that population in the Donostialdea county. This was made possible by the use of data and tools with very different origins. We combined clinical evolution, resource consumption, demographic trends, and epidemiological data obtained with the Dismod II software, parametric survival analysis, economic evaluation, and simulation to carry out a BIA to inform the planning stage of the Deming cycle.
The framework developed within the CareWell project will help its pilot sites to manage the implementation of interventions aimed at maintaining long-term stability of multimorbid patients and assess their outcomes. By setting objectives based on evidence and including them in the BIA, managers can evaluate whether the integrated health care intervention is having the expected impact. This approach, however, has a broad scope and is not limited to the management of integrated health care interventions focused on improving care for multimorbid patients. In fact, by tailoring the conceptual model of the BIA, this approach could be used to determine the adequacy of any complex intervention for which time and implementation are key issues.
Acknowledgment
We acknowledge the English editorial assistance provided by Sally Ebeling.
Source of financial support: All the authors were funded by their institutions. CareWell project is cofunded by the European Commission within the Information Communication Technologies Policy Support Programme of the Competitiveness and Innovation Framework Programme (grant agreement no. 620983).
CareWell Project Group (partner: members) Kronikgune, Basque Country, Spain: Esteban de Manuel, Joana Mora, Ane Fullaondo. Osakidetza-Basque Health Service, Spain: Maria Luisa Merino, Angel Faria, Itziar Vergara, Javier Mar. Health Information Management, Belgium: Panagiotis Stafylas, Marco D'Angelantonio, Maite Hurtado. Faculty of Electrical Engineering and Computig, University of Zagreb, Croatia:Mario Kovac, Leon Dragic. Croatian Society for Pharmacoeconomics and Health Economics, Croatia: Vanesa Benkovic. Ericsson Nikola Tesla, Croatia: Karlo Gustin, Mario Ravic. Regional Healthcare Agency of Puglia, Italy: Elisabetta Anna Graps, Francesca Avolio. Local Health Authority nr.2 of Feltre, Italy: Francesco Marchet. Veneto's Research Centre for Health Innovation, Italy: Claudio Dario, Silvia Mancin, Claudio Saccavini. Lower Silesian Marshal’s Office, Poland: Antoni Zwiefka. Wales Powys Teaching Health Board, United Kingdom: Ian Green, Andrew Rogers, Tanya Summerfield. Empirica, Germany: Reinhard Hammerschmidt, Veli Stroetmann. Region of Southern Denmark, Denmark: Signe Daugbjerg. International Foundation for Integrated Care: Leo Lewis.
Applying dynamic simulation modeling methods in health care delivery research—the SIMULATE checklist: report of the ISPOR Simulation Modeling Emerging Good Practices Task Force.
CareWell. CareWell project: delivering integrated care to frail patients through ICT. Available from: http://www.carewell-project.eu/home/. [Accessed September 9, 2015].
Frameworks of Integrated Care for the Elderly: A Systematic Review.
CPRN Research Report. Canadian Policy Research Networks/Réseaux canadiens de recherche en politiques publiques,
2008 (Available from: http://www.cprn.org/documents/49813_EN.pdf)
Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. 2007. Available from: http://pareonline.net/pdf/v12n10.pdf. [Accessed June 28, 2015].
ISPOR-SMDM Modeling Good Research Practices Task Force. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–4.
30010-9&id=gr1.jpg){kind=link}
30010-9&id=gr2.jpg){kind=link}
30010-9&id=gr3.jpg){kind=link}