The Potential Cost-Effectiveness of a Machine Learning Tool That Can Prevent Untimely Intensive Care Unit Discharge

Published:October 21, 2021DOI:


      • Given that Pacmed Critical (PC) or similar tools are not yet on the market and, to date, no cost-effectivenes analysis has been performed, we designed a novel 7-state Markov model based on the pathway for patients in the intensive care unit (ICU) and the application of PC along this pathway.
      • We showed that PC has the potential to be cost-effective compared with standard care in the Dutch ICU health setting.
      • This study is one of the first cost-effectiveness analyses of a machine learning device to be used in healthcare and also contributes to the limited cost-effectiveness analyses in the field of ICU and discharge devices.



      The machine learning prediction model Pacmed Critical (PC), currently under development, may guide intensivists in their decision-making process on the most appropriate time to discharge a patient from the intensive care unit (ICU). Given the financial pressure on healthcare budgets, this study assessed whether PC has the potential to be cost-effective compared with standard care, without the use of PC, for Dutch patients in the ICU from a societal perspective.


      A 1-year, 7-state Markov model reflecting the ICU care pathway and incorporating the PC decision tool was developed. A hypothetical cohort of 1000 adult Dutch patients admitted in the ICU was entered in the model. We used the literature, expert opinion, and data from Amsterdam University Medical Center for model parameters. The uncertainty surrounding the incremental cost-effectiveness ratio was assessed using deterministic and probabilistic sensitivity analyses and scenario analyses.


      PC was a cost-effective strategy with an incremental cost-effectiveness ratio of €18 507 per quality-adjusted life-year. PC remained cost-effective over standard care in multiple scenarios and sensitivity analyses. The likelihood that PC will be cost-effective was 71% at a willingness-to-pay threshold of €30 000 per quality-adjusted life-year. The key driver of the results was the parameter “reduction in ICU length of stay.”


      We showed that PC has the potential to be cost-effective for Dutch ICUs in a time horizon of 1 year. This study is one of the first cost-effectiveness analyses of a machine learning device. Further research is needed to validate the effectiveness of PC, thereby focusing on the key parameter “reduction in ICU length of stay” and potential spill-over effects.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Value in Health
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Haas L.E.
        • Karakus A.
        • Holman R.
        • Cihangir S.
        • Reidinga A.C.
        • de Keizer N.F.
        Trends in hospital and intensive care admissions in The Netherlands attributable to the very elderly in an ageing population.
        Crit Care. 2015; 19: 353
        • Wild C.
        • Narath M.
        Evaluating and planning ICUs: methods and approaches to differentiate between need and demand.
        Health Policy. 2005; 71: 289-301
        • Higgins A.M.
        • Pettila V.
        • Bellomo R.
        • Harris A.H.
        • Nichol A.D.
        • Morrison S.S.
        Expensive care - a rationale for economic evaluations in intensive care.
        Crit Care Resusc. 2010; 12: 62-66
        • Wong E.G.
        • Parker A.M.
        • Leung D.G.
        • Brigham E.P.
        • Arbaje A.I.
        Association of severity of illness and intensive care unit readmission: a systematic review.
        Heart Lung. 2016; 45: 3-9.e2
        • Rosenberg A.L.
        • Watts C.
        Patients readmitted to ICUs∗: a systematic review of risk factors and outcomes.
        Chest. 2000; 118: 492-502
        • Thoral P.J.
        • Peppink J.M.
        • Driessen R.H.
        • et al.
        Sharing ICU patient data responsibly under the Society of Critical Care Medicine/European Society of Intensive Care Medicine joint data science collaboration: the Amsterdam University medical centers database (AmsterdamUMCdb) example.
        Crit Care Med. 2021; 49: e563-e577
        • Thoral P.
        • Fornasa M.
        • Hovenkamp H.
        • Driessen R.
        • Girbes A.
        • Elbers P.
        Right data, right now: developing a big data machine-learning based prediction model to prevent ICU readmission.
        Intensive Care Med Exp. 2018; 6: 40
        • Curth A.
        • Thoral P.
        • van den Wildenberg W.
        • et al.
        Transferring clinical prediction models across hospitals and electronic health record systems.
        in: Cellier P. Driessens K. Machine Learning and Knowledge Discovery in Databases - International Workshops of ECML PKDD 2019, Proceedings. Springer, Wurzburg, Germany2020: 605-621
        • Desautels T.
        • Das R.
        • Calvert J.
        • et al.
        Prediction of early unplanned intensive care unit readmission in a UK tertiary care hospital: a cross-sectional machine learning approach.
        BMJ Open. 2017; 7e017199
        • Rojas J.C.
        • Carey K.A.
        • Edelson D.P.
        • Venable L.R.
        • Howell M.D.
        • Churpek M.M.
        Predicting intensive care unit readmission with machine learning using electronic health record data.
        Ann Am Thorac Soc. 2018; 15: 846-853
        • Lin Y.W.
        • Zhou Y.
        • Faghri F.
        • Shaw M.J.
        • Campbell R.H.
        Analysis and prediction of unplanned intensive care unit readmission using recurrent neural networks with long short-term memory.
        PLoS One. 2019; 14e0218942
        • Buisman L.R.
        • Rutten-Van Mölken M.P.
        • Postmus D.
        • Luime J.J.
        • Uyl-De Groot C.A.
        • Redekop W.K.
        The early bird catches the worm: early cost-effectiveness analysis of new medical tests.
        Int J Technol Assess Health Care. 2016; 32: 46-53
        • Ijzerman M.J.
        • Steuten L.M.
        Early assessment of medical technologies to inform product development and market access: a review of methods and applications.
        Appl Health Econ Health Policy. 2011; 9: 331-347
        • Vallejo-Torres L.
        • Steuten L.M.
        • Buxton M.J.
        • Girling A.J.
        • Lilford R.J.
        • Young T.
        Integrating health economics modeling in the product development cycle of medical devices: a Bayesian approach.
        Int J Technol Assess Health Care. 2008; 24: 459-464
        • Love-Koh J.
        • Peel A.
        • Rejon-Parrilla J.C.
        • et al.
        The future of precision medicine: potential impacts for health technology assessment [published correction appears in Pharmacoeconomics. 2018;37(1):117].
        Pharmacoeconomics. 2018; 36: 1439-1451
        • Wolff J.
        • Pauling J.
        • Keck A.
        • Baumbach J.
        The economic impact of artificial intelligence in health care: systematic review.
        J Med Internet Res. 2020; 22e16866
        • Drummond M.F.
        • Sculpher M.J.
        • Claxton K.
        • Stoddhart G.L.
        • Torrance G.W.
        Methods for the Economic Evaluation of Health Care Programmes.
        4th ed. Oxford University Press, Oxford, United Kingdom2015
        • Hakkaart-van Roijen L.
        • Van der Linden N.
        • Bouwmans C.A.M.
        • Kanters T.
        • Tan S.S.
        Costing Manual: Methodology of Costing Research and Reference Prices for Economic Evaluations in Healthcare [in Dutch: Kostenhandleiding: Methodologie van Kostenonderzoek en Referentieprijzen Voor Economische Evaluaties in de Gezondheidszorg].
        Zorginstituut Nederland, Diemen, The Netherlands2015
      1. Guideline for economic evaluations in healthcare. Zorginstituut Nederland.
        • Brinkman S.
        • De Jonge E.
        • Abu-Hanna A.
        • Arbous M.S.
        • De Lange D.W.
        • De Keizer N.F.
        Mortality after hospital discharge in ICU patients.
        Crit Care Med. 2013; 41: 1229-1236
        • van de Klundert N.
        • Holman R.
        • Dongelmans D.A.
        • de Keizer N.F.
        Data resource profile: the Dutch National Intensive Care Evaluation (NICE) registry of admissions to adult intensive care units.
        Int J Epidemiol. 2015; 44 (1850-1850h)
        • Wilcox M.
        • Rubenfeld G.D.
        Is critical care ready for an economic surrogate endpoint?.
        Crit Care. 2015; 19: 248
        • Briggs A.
        • Claxton K.
        • Sculpher M.
        Decision Modelling for Health Economic Evaluation.
        Oxford University Press, Oxford, United Kingdom2006
        • Dong H.
        • Buxton M.
        Early assessment of the likely cost-effectiveness of a new technology: a Markov model with probabilistic sensitivity analysis of computer-assisted total knee replacement.
        Int J Technol Assess Health Care. 2006; 22: 191-202
      2. The labour force in figures 2017. Centraal Bureau voor de Statistiek (CBS).
      3. Labour participation; key figures. Centraal Bureau voor de Statistiek (CBS).
      4. Over 700 euros co-payments for care in 2015. Centraal Bureau voor de Statistiek (CBS).
      5. Consumer Price Index 2015=100. Centraal Bureau voor de Statistiek (CBS).
        • van Beusekom I.
        • Bakhshi-Raiez F.
        • de Keizer N.F.
        • van der Schaaf M.
        • Busschers W.B.
        • Dongelmans D.A.
        Healthcare costs of ICU survivors are higher before and after ICU admission compared to a population based control group: a descriptive study combining healthcare insurance data and data from a Dutch national quality registry.
        J Crit Care. 2018; 44: 345-351
        • Griffiths J.
        • Hatch R.A.
        • Bishop J.
        • et al.
        An exploration of social and economic outcome and associated health-related quality of life after critical illness in general intensive care unit survivors: a 12-month follow-up study.
        Crit Care. 2013; 17: R100
      6. Transport and mobility 2016. Centraal Bureau voor de Statistiek (CBS).
        • van Beusekom I.
        • Bakhshi-Raiez F.
        • de Keizer N.F.
        • van der Schaaf M.
        • Termorshuizen F.
        • Dongelmans D.A.
        Dutch ICU survivors have more consultations with general practitioners before and after ICU admission compared to a matched control group from the general population.
        PLoS One. 2019; 14e0217225
        • Hill A.D.
        • Fowler R.A.
        • Pinto R.
        • Herridge M.S.
        • Cuthbertson B.H.
        • Scales D.C.
        Long-term outcomes and healthcare utilization following critical illness -- a population-based study.
        Crit Care. 2016; 20: 76
        • van Beusekom I.
        • Bakhshi-Raiez F.
        • de Keizer N.F.
        • Dongelmans D.A.
        • van der Schaaf M.
        Reported burden on informal caregivers of ICU survivors: a literature review.
        Crit Care. 2016; 20: 16
        • Soliman I.W.
        • de Lange D.W.
        • Peelen L.M.
        • et al.
        Single-center large-cohort study into quality of life in Dutch intensive care unit subgroups, 1 year after admission, using EuroQoL EQ-6D-3L.
        J Crit Care. 2015; 30: 181-186
        • Hofhuis J.G.
        • van Stel H.F.
        • Schrijvers A.J.
        • Rommes J.H.
        • Bakker J.
        • Spronk P.E.
        Health-related quality of life in critically ill patients: how to score and what is the clinical impact?.
        Curr Opin Crit Care. 2009; 15: 425-430
        • Vainiola T.
        • Roine R.P.
        • Pettilä V.
        • Kantola T.
        • Räsänen P.
        • Sintonen H.
        Effect of health-related quality-of-life instrument and quality-adjusted life year calculation method on the number of life years gained in the critical care setting.
        Value Health. 2011; 14: 1130-1134
        • Kaarlola A.
        • Tallgren M.
        • Pettilä V.
        Long-term survival, quality of life, and quality-adjusted life-years among critically ill elderly patients.
        Crit Care Med. 2006; 34: 2120-2126
        • Ferguson N.D.
        • Scales D.C.
        • Pinto R.
        • et al.
        Integrating mortality and morbidity outcomes: using quality-adjusted life years in critical care trials.
        Am J Respir Crit Care Med. 2013; 187: 256-261
        • Oeyen S.G.
        • Vandijck D.M.
        • Benoit D.D.
        • Annemans L.
        • Ducruyenaere J.M.
        Quality of life after intensive care: a systematic review of the literature.
        Crit Care Med. 2010; 38: 2386-2400
        • Lone N.I.
        • Gillies M.A.
        • Haddow C.
        • et al.
        Five-year mortality and hospital costs associated with surviving intensive care.
        Am J Respir Crit Care Med. 2016; 194: 198-208
        • Klok R.M.
        • Postma M.J.
        Four quadrants of the cost-effectiveness plane: some considerations on the south-west quadrant.
        Expert Rev Pharmacoecon Outcomes Res. 2004; 4: 599-601
      7. Kosteneffectiviteit in de praktijk. Zorginstituut Nederland.
        • de Windt T.S.
        • Sorel J.C.
        • Vonk L.A.
        • Kip M.M.A.
        • Ijzerman M.J.
        • Saris D.B.F.
        Early health economic modelling of single-stage cartilage repair. Guiding implementation of technologies in regenerative medicine.
        J Tissue Eng Regen Med. 2017; 11: 2950-2959
        • Gravesteijn B.Y.
        • Schluep M.
        • Voormolen D.C.
        • et al.
        Cost-effectiveness of extracorporeal cardiopulmonary resuscitation after in-hospital cardiac arrest: a Markov decision model.
        Resuscitation. 2019; 143: 150-157
        • Sevick L.K.
        • Santana M.J.
        • Ghali W.A.
        • Clement F.
        Prospective economic evaluation of an electronic discharge communication tool: analysis of a randomised controlled trial.
        BMJ Open. 2017; 7e019139
        • Talmor D.
        • Shapiro N.
        • Greenberg D.
        • Stone P.W.
        • Neumann P.J.
        When is critical care medicine cost-effective? A systematic review of the cost-effectiveness literature.
        Crit Care Med. 2006; 34: 2738-2747
        • Sevick L.K.
        • Esmail R.
        • Tang K.
        • et al.
        A systematic review of the cost and cost-effectiveness of electronic discharge communications.
        BMJ Open. 2017; 7e014722