“Looking Under the Hood” of Anchor-Based Assessment of Clinically Important Change: A Machine Learning Approach


      • The Global Assessment of Change (GAC) primarily reflects change in the physical domain of health-related quality of life among chronically ill patients and their caregivers.
      • It reflects attributions, goals, and patterns of emphasis related to change in health and healthcare.
      • Our findings suggest that commonly unmeasured factors have bearing on GAC scores and are relevant in facilitating the interpretation of patient-reported outcome change.



      The Global Assessment of Change (GAC) item has facilitated the interpretation of change in patient-reported outcomes, providing an anchor for computing minimally important differences. Construct validity has been documented via disease-specific patient-reported outcomes change. We examined what domains, sociodemographic characteristics, attributions of change, and cognitive-appraisal processes are reflected in GAC ratings.


      This secondary analysis examined data from 1,481 chronically ill patients and caregivers surveyed at baseline and 17 months. Items queried change since baseline in overall disease symptoms (GAC) and in physical, emotional, and social functioning. Candidate predictors included sociodemographic factors, health-related quality-of-life domains, change attributions, and quality-of-life appraisal processes. Least absolute shrinkage and selection operator and bootstrapping tested 77 predictors’ effectiveness and stability.


      GAC worsening was notably associated with being disabled (β = −0.24) and having difficulty paying bills (β = −0.13). GAC was better explained by the physical domain than the emotional or social (β = 0.67, 0.10, and 0.03, respectively; R2adj = 0.63) after sociodemographic-covariate adjustment. In a separate model (R2adj = 0.18), GAC variance was explained by attributions about changing health and changing response of one’s health team, goals related to solving healthcare problems and maintaining activities, and appraisal about things getting better (β = −0.14, 0.08, −0.07, 0.05, 0.21, respectively; prange ~0.0005–0.05) after adjustment.


      The GAC primarily reflects the physical domain, and the GAC reflects attributions, goals, and patterns of emphasis related to change in health and healthcare. Commonly unmeasured factors have some bearing on GAC scores and can facilitate the interpretation of change.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic and Personal


      Subscribe to Value in Health
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Juniper E.F.
        • Guyatt G.H.
        • Willan A.
        • Griffith L.E.
        Determining a minimal important change in a disease-specific quality of life questionnaire.
        J Clin Epidemiol. 1994; 47: 81-87
        • Revicki D.A.
        • Cella D.
        • Hays R.D.
        • Sloan J.A.
        • Lenderking W.R.
        • Aaronson N.K.
        Responsiveness and minimal important differences for patient reported outcomes.
        Health Qual Life Outcomes. 2006; 4: 70
        • Guyatt G.H.
        • Osoba D.
        • Wu A.W.
        • Wyrwich K.W.
        • Norman G.R.
        • Clinical Significance Consensus Meeting Group
        Methods to explain the clinical significance of health status measures.
        Mayo Clin Proc. 2002; 77: 371-383
        • Sloan J.A.
        • Aaronson N.
        • Cappelleri J.C.
        • Fairclough D.L.
        • Varricchio C.
        Clinical Significance Consensus Meeting Group. Assessing the clinical significance of single items relative to summated scores.
        Mayo Clin Proc. 2002; 77: 479-487
        • Sprangers M.A.
        • Moinpour C.M.
        • Moynihan T.J.
        • Patrick D.L.
        • Revicki D.A.
        Clinical Significance Consensus Meeting Group. Assessing meaningful change in quality of life over time: a users’ guide for clinicians.
        Mayo Clin Proc. 2002; 77: 561-571
        • Revicki D.
        • Hays R.D.
        • Cella D.
        • Sloan J.
        Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes.
        J Clin Epidemiol. 2008; 61: 102-109
        • Norman G.R.
        • Sloan J.A.
        • Wyrwich K.W.
        Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation.
        Med Care. 2003; 41: 582-592
        • Haywood K.L.
        • Garratt A.M.
        • Jordan K.
        • Dziedzic K.
        • Dawes P.T.
        Spinal mobility in ankylosing spondylitis: reliability, validity and responsiveness.
        Rheumatology (Oxford). 2004; 43: 750-757
        • Norman G.
        • Wyrwich K.W.
        • Patrick D.L.
        The mathematical relationship among different forms of responsiveness coefficients.
        Qual Life Res. 2007; 16: 815-822
        • Kamper S.
        Global rating of change scales.
        Aust J Physiother. 2009; 55: 289
        • Kamper S.J.
        • Maher C.G.
        • Mackay G.
        Global rating of change scales: a review of strengths and weaknesses and considerations for design.
        J Man Manip Ther. 2009; 17: 163-170
        • Ross M.
        Relation of implicit theories to the construction of personal histories.
        Psychol Rev. 1989; 96: 341
        • Herrmann D.
        Reporting current, past, and changed health status. What we know about distortion.
        Med Care. 1995; 33: AS89-AS94
        • Guyatt G.H.
        • Norman G.R.
        • Juniper E.F.
        • Griffith L.E.
        A critical look at transition ratings.
        J Clin Epidemiol. 2002; 55: 900-908
        • Schmitt J.
        • Di Fabio R.P.
        The validity of prospective and retrospective global change criterion measures.
        Arch Phys Med Rehabil. 2005; 86: 2270-2276
        • Kamper S.J.
        • Ostelo R.W.
        • Knol D.L.
        • Maher C.G.
        • de Vet H.C.
        • Hancock M.J.
        Global perceived effect scales provided reliable assessments of health transition in people with musculoskeletal disorders, but ratings are strongly influenced by current status.
        J Clin Epidemiol. 2010; 63: 760-766.e1
        • Norman G.R.
        • Stratford P.
        • Regehr G.
        Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach.
        J Clin Epidemiol. 1997; 50: 869-879
        • Costa L.O.P.
        • Maher C.G.
        • Latimer J.
        • et al.
        Clinimetric testing of three self-report outcome measures for low back pain patients in Brazil: which one is the best?.
        Spine (Phila Pa 1976). 2008; 33: 2459-2463
      1. U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER) Center for Biologics Evaluation and Research (CBER) Center for Devices and Radiological Health (CDRH) Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims.
        (Available at:)
        Date accessed: April 22, 2021
        • Powers 3rd, J.H.
        • Patrick D.L.
        • Walton M.K.
        • et al.
        Clinician-reported outcome assessments of treatment benefit: report of the ISPOR clinical outcome assessment emerging good practices task force.
        Value Health. 2017; 20: 2-14
        • Long A.F.
        • Dixon P.
        Monitoring outcomes in routine practice: defining appropriate measurement criteria.
        J Eval Clini Pract. 1996; 2: 71-78
        • Abrams D.
        • Davidson M.
        • Harrick J.
        • Harcourt P.
        • Zylinski M.
        • Clancy J.
        Monitoring the change: current trends in outcome measure usage in physiotherapy.
        Man Ther. 2006; 11: 46-53
        • Beaton D.E.
        • Tarasuk V.
        • Katz J.N.
        • Wright J.G.
        • Bombardier C.
        “Are you better?” A qualitative study of the meaning of recovery.
        Arthritis Care Res. 2001; 45: 270-279
        • Sangha O.
        • Stucki G.
        • Liang M.H.
        • Fossel A.H.
        • Katz J.N.
        The self-administered comorbidity questionnaire: a new method to assess comorbidity for clinical and health services research.
        Arthritis Rheum. 2003; 49: 156-163
        • Hanmer J.
        • Cherepanov D.
        A single question about a respondent’s perceived financial ability to pay monthly bills explains more variance in health utility scores than absolute income and assets questions.
        Qual Life Res. 2016; 25: 2233-2237
        • Li Y.
        • Rapkin B.D.
        Classification and regression tree uncovered hierarchy of psychosocial determinants underlying quality-of-life response shift in HIV/AIDS.
        J Clin Epidemiol. 2009; 62: 1138-1147
        • Schwartz C.E.
        • Rapkin B.A.
        Understanding appraisal processes underlying the thentest: a mixed methods investigation [published correction appears in Qual Life Res. 2014;23(1):373. Rapkin, Bruce A [corrected to Rapkin, Bruce D]].
        Qual Life Res. 2012; 21: 381-388
        • Rapkin B.D.
        • Garcia I.
        • Michael W.
        • Zhang J.
        • Schwartz C.E.
        Distinguishing appraisal and personality influences on quality of life in chronic illness: introducing the quality-of-life Appraisal Profile version 2.
        Qual Life Res. 2017; 26: 2815-2829
        • Schwartz C.E.
        • Li J.
        • Rapkin B.D.
        Refining a web-based goal assessment interview: item reduction based on reliability and predictive validity.
        Qual Life Res. 2016; 25: 2201-2212
        • Rapkin B.D.
        • Schwartz C.E.
        Distilling the essence of appraisal: a mixed methods study of people with multiple sclerosis.
        Qual Life Res. 2016; 25: 793-805
        • James G.
        • Witten D.
        • Hastie T.
        • Tibshirani R.
        An Introduction to Statistical Learning With Applications in R.
        Springer, New York, NY2014
        • Zar J.
        Biostatistical Analysis.
        2nd ed. Prentice Hall, Englewood Cliffs, NJ1984
        • Harrell Jr., F.E.
        Ordinal logistic regression.
        in: Regression Modeling Strategies. Springer, Berlin, UK2015: 311-325
        • IBM
        Creating decision trees.
        in: Documentation ISSV. 24.0 edition. IBM, Armonk, NY2020
        • Cohen J.
        A power primer.
        Psychol Bull. 1992; 112: 155-159
        • Freedman D.A.
        Bootstrapping regression models.
        Ann Statist. 1981; 9: 1218-1228
      2. SPSS Statistics for Windows [computer program]. Version 26.0. IBMCorp, Armonk, NY2019
      3. R: A Language and Environment for Statistical Computing [computer program]. R Foundation for Statistical Computing, Vienna, Austria2017
      4. Glmnet: Fit a GLM With Lasso or Elasticnet Regularization [computer program]. Version 3.0-2. R Foundation, Vienna, Austria2008
      5. Plotmo: Plot a Model’s Response Over a Range of Predictor Values (the Model Surface) [computer program]. R Foundation for Statistical Computing, Vienna, Austria2019
        • Schwartz C.E.
        • Finkelstein J.A.
        • Rapkin B.D.
        Appraisal assessment in patient-reported outcome research: methods for uncovering the personal context and meaning of quality of life.
        Qual Life Res. 2017; 26: 545-554
        • Schwartz C.E.
        • Zhang J.
        • Rapkin B.D.
        • Finkelstein J.A.
        Reconsidering the minimally important difference: evidence of instability over time and across groups.
        Spine J. 2019; 19: 726-734
        • Rapkin B.D.
        • Schwartz C.E.
        Toward a theoretical model of quality-of-life appraisal: implications of findings from studies of response shift.
        Health Qual Life Outcomes. 2004; 2: 14
        • Rapkin B.D.
        • Schwartz C.E.
        Advancing quality-of-life research by deepening our understanding of response shift: a unifying theory of appraisal.
        Qual Life Res. 2019; 28: 2623-2630
        • Idler E.
        • Cartwright K.
        What do we rate when we rate our health? Decomposing age-related contributions to self-rated health.
        J Health Soc Behav. 2018; 59: 74-93
        • Jylhä M.
        What is self-rated health and why does it predict mortality? Towards a unified conceptual model.
        Soc Sci Med. 2009; 69: 307-316