Advertisement

Developing a Measure of Value in Health Care

  • K.H. Ken Lee
    Affiliations
    Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Johns Hopkins University, Baltimore, MD, USA

    Department of Health Policy and Management, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA
    Search for articles by this author
  • J. Matthew Austin
    Correspondence
    Address correspondence to: J. Matthew Austin, 750 East Pratt Street, 15th Floor, Baltimore, MD 21202.
    Affiliations
    Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Johns Hopkins University, Baltimore, MD, USA

    Departments of Anesthesiology and Critical Care Medicine, Johns Hopkins University, Baltimore, MD, USA
    Search for articles by this author
  • Peter J. Pronovost
    Affiliations
    Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Johns Hopkins University, Baltimore, MD, USA

    Department of Health Policy and Management, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA

    Departments of Anesthesiology and Critical Care Medicine, Johns Hopkins University, Baltimore, MD, USA

    Surgery, School of Medicine, Johns Hopkins University, Baltimore, MD, USA

    School of Nursing, Johns Hopkins University, Baltimore, MD, USA
    Search for articles by this author
Open ArchivePublished:January 28, 2015DOI:https://doi.org/10.1016/j.jval.2014.12.009

      Abstract

      There is broad support to pay for value, rather than volume, for health care in the United States. Despite the support, practical approaches for measuring value remain elusive. Value is commonly defined as quality divided by costs, where quality reflects patient outcomes and costs are the total costs for providing care, whether these be costs related to an episode, a diagnosis, or per capita. Academicians have proposed a conceptual approach to measure value, in which we measure outcomes important to patients and costs using time-driven activity-based costing. This approach is conceptually sound, but has significant practical challenges. In our commentary, we describe how health care can use existing quality measures and cost accounting data to measure value. Although not perfect, we believe this approach is practical, valid, and scalable and can establish the foundation for future work in this area.

      Keywords

      There is broad support to pay for value, rather than volume, for health care in the United States [

      Centers for Medicare & Medicaid Services. Hospital value-based purchasing program. Available from: http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/hospital-value-based-purchasing/index.html?redirect=/hospital-value-based-purchasing. [Accessed March 7, 2014].

      ]. Despite palpable support, practical approaches for measuring value remain elusive. Value is commonly defined as quality divided by costs, where quality reflects patient outcomes and costs are the total costs for providing care, whether these be costs related to an episode, a diagnosis, or per capita.
      Academicians, such as Michael Porter and Robert Kaplan at the Harvard Business School, have proposed a conceptual approach to measure value, in which we measure outcomes important to patients and costs using time-driven activity-based costing (TDABC) [
      • Porter M.E.
      What is value in health care?.
      ,
      • Kaplan R.S.
      • Porter M.E.
      The big idea: how to solve the cost crisis in health care.
      ]. TDABC involves following clinicians and staff around with a stopwatch, measuring the time they devote to various activities and multiplying the time spent on each activity by their hourly wage. The approach proposed by Porter and Kaplan is conceptually sound, but has significant challenges. First, although organizations such as the International Consortium for Health Outcomes Measurement have made great progress in defining a standard set of outcome measures for a starter set of diagnoses, the US health care system lacks valid and reliable outcome measures for most diagnoses and, in many cases, lacks the standardized methods and infrastructure needed to collect outcome data [

      Berenson RA, Pronovost PJ, Krumholtz HM. Achieving the potential of health care performance measures: timely analysis of immediate health policy issues. May 2013. Available from: http://www.rwjf.org/content/dam/farm/reports/reports/2013/rwjf406195. [Accessed August 25, 2014].

      ,
      • Pronovost P.J.
      • Miller M.
      • Wachter R.M.
      The GAAP in quality measurement and reporting.
      ,
      • Austin J.M.
      • D’Andrea G.
      • Birkmeyer J.D.
      • et al.
      Safety in numbers: the development of Leapfrog’s composite patient safety score for U.S. hospitals.
      ]. Second, current surveillance for patient outcomes and costs of care is not feasible, scalable, or sustainable. Although researchers should work toward the ideal value measure, we need to measure value now and improve over time.
      In this commentary, we describe how health care can use existing quality measures and cost accounting data to measure value. Although not perfect, we believe this approach is practical, valid, and scalable and can establish the foundation for future work in this area.
      In our measurement approach, in an effort to look at quality more broadly, and recognizing that hospitals often choose to focus on specific aspects of quality at the expense of others, the numerator would be a single composite measure composed of multiple qualities of care, combining existing quality measures spanning various domains, such as complications, patient satisfaction, adherence to recommended processes of care, and clinical and patient-reported outcomes. For example, for surgical services, complications are measured by the rates of surgical site infection; patient satisfaction is measured with the Hospital Consumer Assessment of Healthcare Provider and Systems survey; adherence to recommended care is measured with the Surgical Care Improvement Project core measures; and clinical and patient-reported outcomes are measured with indicators such as return to work and change in functional status. Each service line would select appropriate quality measures for inclusion in a composite and determine the relative weights for each measure. The quality measures must be important, scientifically sound, and usable. The relative weights used in the composite can either represent empirical criteria, such as the measure’s reliability, validity, impact, evidence, and opportunity for improvement, or represent a value judgment, determined with input from patients, providers, and payers.
      Although most of these quality measures are available, the methods to combine them into a single index score are underdeveloped. Any approach that evaluates performance requires a benchmark for comparison. For example, the Leapfrog Hospital Safety Score compares performance to a relative benchmark, the national mean, when calculating its composite patient safety score [
      • Austin J.M.
      • D’Andrea G.
      • Birkmeyer J.D.
      • et al.
      Safety in numbers: the development of Leapfrog’s composite patient safety score for U.S. hospitals.
      ]. The selection of the anchor points to benchmark provider performance against is important for setting expectations, so as to not anchor clinicians to suboptimal performance. We believe one factor in the long time it took hospitals to reduce their rates of central-line–associated bloodstream infections (CLABSI) in intensive care units is the practice of benchmarking against suboptimal performance (e.g., mean) rather than perfect care.
      Before a quality composite score is broadly accepted, health care needs to change how performance measures are reported. First, we need standardized reporting on how well measures perform. For many measures, especially those derived from administrative data (e.g., hospital-acquired conditions), the validity and reliability are unknown or poor [

      Berenson RA, Pronovost PJ, Krumholtz HM. Achieving the potential of health care performance measures: timely analysis of immediate health policy issues. May 2013. Available from: http://www.rwjf.org/content/dam/farm/reports/reports/2013/rwjf406195. [Accessed August 25, 2014].

      ]. For example, The Johns Hopkins Hospital was both congratulated for its rate of CLABSI using clinical surveillance data and criticized for its rate, for the same time period, using administrative data. Second, health care needs performance measures that physicians, patients, payers, and policymakers can use. Usable measures must reflect timely data. Administrative data and the derivative quality measures are often a year old, offering little insights into current practice. Finally, measures need to be presented in an understandable and meaningful way for different audiences.
      The methods used to measure cost for the value equation are also immature. Three methods dominate cost measurement in the provider setting [
      • Tan S.S.
      • Serden L.
      • Geissler A.
      • et al.
      DRGs and cost accounting: which is driving which?.
      ]. One approach takes hospital charges, which are notoriously inaccurate, and multiplies them by a cost-to-charge ratio from the Medicare cost report. The second approach uses a cost accounting system, which incorporates a traditional activity-based costing (ABC), to allocate costs to various resource categories. The third approach uses the TDABC described earlier. These methods have their respective advantages and disadvantages, but the marginal benefits and costs have not been rigorously studied.
      The traditional ABC is the predominant approach in health care, and many health systems have invested in cost accounting systems. Health systems with an ABC system could leverage their existing data for the cost component of the value equation. Although these data cannot perfectly capture costs, they are significantly more accurate than hospital charges and less resource intensive, and, likely, close in accuracy to the TDABC. The traditional ABC approach makes assumptions on effort allocation, whereas the TDABC quantifies the amount of time it takes to complete an activity and uses time as a currency to apportion fixed costs, rather than variable costs. Although the TDABC approach is credible, it will require significant additional resources to be used broadly.
      Further study is needed to evaluate the different approaches to measuring costs. Moreover, cost measures beyond the acute care setting are needed to assess longitudinal costs across the full cycle of care, rather than for an episode or diagnosis. But costs for the acute care setting represent a starting point for this work and something one can build upon. One possibility is to have the Center for Medicare & Medicaid Innovation or the Agency for Healthcare Research and Quality commit 5 to 10 years of funding to a consortium of academic health systems to develop, pilot test, evaluate, and implement measures of longitudinal costs.
      To improve value, its measures should be developed, validated, and used by clinicians, enhancing intrinsic motivation for improvement [
      • Dixon-Woods M.
      • Leslie M.
      • Tarrant C.
      • Bion J.
      Explaining Matching Michigan: an ethnographic study of a patient safety program.
      ]. Current policy approaches for improving quality and value in health care rely largely on extrinsic motivation, such as pay-for-performance programs. Such approaches have had little impact on patient outcomes. In contrast, two large, successful improvement efforts, door to balloon time and the national CLABSI effort, used intrinsic motivation [

      Berenson RA, Pronovost PJ, Krumholtz HM. Achieving the potential of health care performance measures: timely analysis of immediate health policy issues. May 2013. Available from: http://www.rwjf.org/content/dam/farm/reports/reports/2013/rwjf406195. [Accessed August 25, 2014].

      ]. These efforts were led by clinicians, informed by science, guided by valid measures, and effective by working through peer norms and professionalism. This approach may reduce the waste from collecting measures that are deemed unimportant, invalid, and unusable.
      We propose that provider organizations and professional societies partner and learn from patients, providers, and payers what importance each group assigns to individual quality and cost measures. The organizing entities could also use this opportunity to cull stakeholder’s preferences for different quality measures, which could serve as a basis for assigning relative weights in the quality composite. The goal would be to have a standard composite measure of quality for each diagnosis.
      The need to measure value in health care is urgent. Between 10% and 30% of hospitalized patients suffer preventable harm, 2 of 10 patients report receiving disrespectful care, and one-third of health care dollars are wasted—nearly $1 trillion or $9000 per US household [
      • Fineberg H.V.
      Shattuck lecture: a successful and sustainable health system—how to get there from here.
      ,
      • Shekelle P.G.
      • Pronovost P.J.
      • Wachter R.M.
      • et al.
      The top patient safety strategies that can be encouraged for adoption now.
      ]. We can begin this journey using current measures. Despite the existing challenges of measuring quality and costs, we believe our approach to measuring value in health care is the way forward in the near-term. Concurrent to maturing near-term approaches, research should push ahead with developing more ideal value measures.

      Acknowledgments

      We thank Christine G. Holzmueller, BLA, from the Johns Hopkins Medicine Armstrong Institute for Patient Safety and Quality for reviewing and editing the manuscript; she was not compensated for her contribution. The corresponding author obtained written permission from Ms. Holzmueller to acknowledge her contributions.
      Source of financial support: The authors have no other financial relationships to disclose.

      References

      1. Centers for Medicare & Medicaid Services. Hospital value-based purchasing program. Available from: http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/hospital-value-based-purchasing/index.html?redirect=/hospital-value-based-purchasing. [Accessed March 7, 2014].

        • Porter M.E.
        What is value in health care?.
        N Engl J Med. 2010; 363: 2477-2481
        • Kaplan R.S.
        • Porter M.E.
        The big idea: how to solve the cost crisis in health care.
        Harv Bus Rev. 2011; 89 (, 54, 56–61 passim): 46-52
      2. Berenson RA, Pronovost PJ, Krumholtz HM. Achieving the potential of health care performance measures: timely analysis of immediate health policy issues. May 2013. Available from: http://www.rwjf.org/content/dam/farm/reports/reports/2013/rwjf406195. [Accessed August 25, 2014].

        • Pronovost P.J.
        • Miller M.
        • Wachter R.M.
        The GAAP in quality measurement and reporting.
        JAMA. 2007; 298: 1800-1802
        • Austin J.M.
        • D’Andrea G.
        • Birkmeyer J.D.
        • et al.
        Safety in numbers: the development of Leapfrog’s composite patient safety score for U.S. hospitals.
        J Patient Saf. 2014; 10: 64-71
        • Tan S.S.
        • Serden L.
        • Geissler A.
        • et al.
        DRGs and cost accounting: which is driving which?.
        in: Busse R. Geissler A. Quentin W. Wily M. Diagnosis-Related Groups in Europe: Moving Towards Transparency, Efficiency and Quality in Hospitals. Open University Press, McGraw-Hill Education, McGraw-Hill House, Berkshire, England2011
        • Dixon-Woods M.
        • Leslie M.
        • Tarrant C.
        • Bion J.
        Explaining Matching Michigan: an ethnographic study of a patient safety program.
        Implement Sci. 2013; 8: 70
        • Fineberg H.V.
        Shattuck lecture: a successful and sustainable health system—how to get there from here.
        N Engl J Med. 2012; 366: 1020-1027
        • Shekelle P.G.
        • Pronovost P.J.
        • Wachter R.M.
        • et al.
        The top patient safety strategies that can be encouraged for adoption now.
        Ann Intern Med. 2013; 158: 365-368