Advertisement

Estimating Joint Health State Utility Algorithms Under Partial Information

Open AccessPublished:October 25, 2022DOI:https://doi.org/10.1016/j.jval.2022.09.009

      Highlights

      • Co-occurring, or “joint,” health conditions complicate the estimation of health state utilities, motivating studies that empirically assess the performance of joint health state utility estimators.
      • Empirical studies on joint health state utility estimators use the information on the utility of narrowly defined single-condition health states that is often not readily available to researchers.
      • We explored the performance of joint health state utility estimators under a more realistic information set and find that previous methodological guidance may not be appropriate under such conditions of limited information.

      Objectives

      We explored the performance of existing joint health state utility estimators when data are not available on utilities that isolate single-condition health states excluding any co-occurring condition.

      Methods

      Using data from the National Epidemiologic Survey on Alcohol and Related Conditions-III, we defined 2 information sets: (1) a full-information set that includes the narrowly defined health state utilities used in most studies that test the performance of joint health state utility estimators, and (2) a limited information set that includes only the more broadly defined health state utilities more commonly available to researchers. We used an example of alcohol use disorder co-occurring with cirrhosis of the liver, depressive disorder, or nicotine use disorder to illustrate our analysis.

      Results

      We found that the performance of joint health state utility estimators is appreciably different under limited information than under full information. Full-information estimators typically overestimate the joint state utility, whereas limited-information estimators underestimate the joint state utility, except for the minimum estimator, which is overestimated in all cases.

      Conclusions

      Researchers using joint health state utility estimators should understand the information set available to them and use methodological guidance appropriate for that information set. We recommend the minimum estimator under limited information based on its ease of use, consistency (and therefore a predictable direction of bias), and lower root mean squared error.

      Keywords

      Introduction

      Cost-effectiveness analysis (CEA) is a standard element of establishing the evidence base in healthcare.
      Guide to the methods of technology appraisal.
      ,
      • Sanders G.D.
      • Neumann P.J.
      • Basu A.
      • et al.
      Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine.
      CEA provides a quantitative measure of value by simultaneously comparing the costs and outcomes of one intervention to those of another. Best practice guidelines
      Guide to the methods of technology appraisal.
      ,
      • Sanders G.D.
      • Neumann P.J.
      • Basu A.
      • et al.
      Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine.
      recommend that quality-adjusted life-years be used as the outcome measure in CEA, resulting in cost-utility analysis. Quality-adjusted life-years combine both the length of life and quality of life as measured with health utility, which ranges from 0 (for dead) to 1 (for perfect health) and captures the individuals’ preferences for living in discrete health states. Best practice guidelines
      Guide to the methods of technology appraisal.
      ,
      • Sanders G.D.
      • Neumann P.J.
      • Basu A.
      • et al.
      Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine.
      also recommend that health utilities be measured using generic, preference-based measures. Although these measures may not isolate the discrete health state in question, they are often used to define the average health utility associated with a discrete health state in Markov or other state transition models.
      Estimating health utility for a health state defined by a single health condition is relatively straightforward but becomes more complicated when 2 or more health conditions co-occur.
      • Basu A.
      • Dale W.
      • Elstein A.
      • Meltzer D.
      A linear index for predicting joint health-states utilities from single health-states utilities.
      • Dale W.
      What is the best model for estimating joint health states utilities? Comparing the linear index model to the proportional decrement model.
      • Fu A.Z.
      • Kattan M.W.
      Utilities should not be multiplied: evidence from the preference-based scores in the United States.
      • Hu B.
      • Fu A.Z.
      Predicting utility for joint health states: a general framework and a new nonparametric estimator.
      Estimating health utility for so-called “joint health states” is particularly critical for health conditions that commonly co-occur with other conditions or for treatments in which adverse events are common or serious,
      • Basu A.
      • Dale W.
      • Elstein A.
      • Meltzer D.
      A linear index for predicting joint health-states utilities from single health-states utilities.
      ,
      • Ara R.
      • Brazier J.E.
      Populating an economic model with health state utility values: moving toward better practice.
      ,
      • Brazier J.
      • Ara R.
      • Azzabi I.
      • et al.
      Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force Report.
      such as impotence after the treatment of prostate cancer
      • Dale W.
      • Basu A.
      • Elstein A.
      • Meltzer D.
      Predicting utility ratings for joint health States from single health states in prostate cancer: empirical testing of 3 alternative theories.
      ,
      • Loeb S.
      • Zhou Q.
      • Siebert U.
      • et al.
      Active surveillance versus watchful waiting for localized prostate cancer: a model to inform decisions.
      or comorbid mental health and substance use disorders.
      • Wittenberg E.
      • Bray J.W.
      • Gebremariam A.
      • Aden B.
      • Nosyk B.
      • Schackman B.R.
      Joint utility estimators in substance use disorders.
      • Schackman B.R.
      • Gutkind S.
      • Morgan J.R.
      • et al.
      Cost-effectiveness of hepatitis C screening and treatment linkage intervention in US methadone maintenance treatment programs.
      • Avanceña A.L.V.
      • Miller N.
      • Uttal S.E.
      • Hutton D.W.
      • Mellinger J.L.
      Cost-effectiveness of alcohol use treatments in patients with alcohol-related cirrhosis.
      In these cases, ignoring the utility loss associated with the joint health state can bias CEA results and lead to faulty policy recommendations.
      Although the preferred approach is to use empirically estimated utilities for joint health states,
      • Brazier J.
      • Ara R.
      • Azzabi I.
      • et al.
      Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force Report.
      ,
      • Ara R.
      • Brazier J.
      Estimating health state utility values for comorbidities.
      the number of possible joint states often makes this impractical. Theoretically-derived estimation algorithms have, therefore, been proposed to estimate joint health state utilities using the utilities of the relevant single health states, and a growing literature explores how well these algorithms perform in estimating the true joint health state utility.
      • Ara R.
      • Wailoo A.J.
      Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.
      From a theoretical standpoint,
      • Keeney R.L.
      • Raiffa H.
      Decisions With Multiple Objectives: Preferences and Value Tradeoffs.
      a joint health state should be a combination of the attributes that define each of the underlying single health states. Thus, the utility of a joint health state can be assessed using the same stated preference elicitation methods that are used to estimate the utility of single health states. These methods describe health states based on their underlying attributes and assess the utility of any given health condition, or combination of any multiple health conditions, based on the stated preferences for living in such states. Theory predicts that a health state with more negative attributes should have lower utility than a state with fewer negative attributes. By definition, a joint health state has more negative attributes than either of the associated single-condition health states and so should have a lower utility than either single-condition health state. Comparing the utility of the joint health state to the utility of the associated single-condition states, therefore, allows researchers to test key theoretical assumptions, the assessment methods used, or both.
      • Dale W.
      • Bilir S.P.
      • Hemmerich J.
      • Basu A.
      • Elstein A.
      • Meltzer D.
      The prevalence, correlates, and impact of logically inconsistent preferences in utility assessments for joint health states in prostate cancer.
      ,
      • Triantaphyllou E.
      • Yanase J.
      How to identify and treat data inconsistencies when eliciting health-state utility values for patient-centered decision making.
      Although the theoretical anchoring of joint health state utility estimators has many advantages, it results in estimation algorithms that may require more information than is commonly available to researchers using generic, preference-based measures. These measures capture data on the health state attributes included in the measure, and not necessarily the attributes most salient for the underlying health conditions. Furthermore, joint health state utility estimators require information on the utility of each single-condition health state with no comorbidities, and possibly the health state defined by the absence of both health conditions under consideration. Yet, utilities for single-condition health states are more commonly reported for anyone with that health condition, regardless of any other condition they may have. Thus, it can be difficult to find estimates of the “pure” single-condition health state utilities required by most joint health state estimation algorithms.
      This article explores the performance of joint health state utility estimators when only limited information is available. We used data from the National Epidemiologic Survey on Alcohol and Related Conditions-III (NESARC-III) on the health utility associated with alcohol use disorder (AUD) and select co-occurring conditions to empirically test the performance of 4 joint health state utility estimators and to provide practical guidance on the conditions under which each estimator is likely to be optimal.

      Background

      Four joint health state utility estimators have been investigated by the field
      • Ara R.
      • Wailoo A.J.
      Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.
      :
      • 1.
        the minimum estimator, in which the lesser of 2 single states’ utilities is used as an estimate of their joint utility
        • Keeney R.L.
        • Raiffa H.
        Decisions With Multiple Objectives: Preferences and Value Tradeoffs.
        ;
      • 2.
        the additive or constant decrement estimator, in which the sum of the 2 single states’ disutilities (ie, 1 – utility) is subtracted from a baseline utility to estimate their joint utility (to a minimum of 0)
        • Anderson N.
        • Zalinski J.
        Functional measurement approach to self-estimation in multiattribute evaluation.
        ;
      • 3.
        the multiplicative estimator, in which the product of the 2 single states’ utilities is used as an estimate of their joint utility
        • Keeney R.L.
        • Raiffa H.
        Decisions With Multiple Objectives: Preferences and Value Tradeoffs.
        ; and
      • 4.
        the adjusted decrement estimator, a nonparametric model that combines the 2 single states’ utilities in proportion to the difference between them.
        • Hu B.
        • Fu A.Z.
        Predicting utility for joint health states: a general framework and a new nonparametric estimator.
      A fifth estimator, the linear index estimator,
      • Basu A.
      • Dale W.
      • Elstein A.
      • Meltzer D.
      A linear index for predicting joint health-states utilities from single health-states utilities.
      has been proposed but is less commonly used. The linear index estimator is a parametric model that uses the weighted sum of the minimum and maximum of the 2 single states’ utilities and their interaction to estimate the joint state’s utility.
      • Basu A.
      • Dale W.
      • Elstein A.
      • Meltzer D.
      A linear index for predicting joint health-states utilities from single health-states utilities.
      ,
      • Thompson A.J.
      • Sutton M.
      • Payne K.
      Estimating joint health condition utility values.
      The optimal weights for the linear index estimator seem to be specific to the health conditions in question
      • Ara R.
      • Wailoo A.J.
      Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.
      ,
      • Thompson A.J.
      • Sutton M.
      • Payne K.
      Estimating joint health condition utility values.
      and so it is difficult to implement in practice.
      Studies have explored the practical application of these algorithms for estimating joint health state utilities.
      • Basu A.
      • Dale W.
      • Elstein A.
      • Meltzer D.
      A linear index for predicting joint health-states utilities from single health-states utilities.
      ,
      • Fu A.Z.
      • Kattan M.W.
      Utilities should not be multiplied: evidence from the preference-based scores in the United States.
      • Hu B.
      • Fu A.Z.
      Predicting utility for joint health states: a general framework and a new nonparametric estimator.
      • Ara R.
      • Brazier J.E.
      Populating an economic model with health state utility values: moving toward better practice.
      ,
      • Dale W.
      • Basu A.
      • Elstein A.
      • Meltzer D.
      Predicting utility ratings for joint health States from single health states in prostate cancer: empirical testing of 3 alternative theories.
      ,
      • Wittenberg E.
      • Bray J.W.
      • Gebremariam A.
      • Aden B.
      • Nosyk B.
      • Schackman B.R.
      Joint utility estimators in substance use disorders.
      ,
      • Ara R.
      • Brazier J.
      Estimating health state utility values for comorbidities.
      ,
      • Dale W.
      • Bilir S.P.
      • Hemmerich J.
      • Basu A.
      • Elstein A.
      • Meltzer D.
      The prevalence, correlates, and impact of logically inconsistent preferences in utility assessments for joint health states in prostate cancer.
      Most studies use data from large databases that include a preference-weighted quality-of-life measure and data on a variety of health conditions experienced by sample members. Such data can be used to estimate the mean health utility of individuals with a joint health condition as well as that of individuals with each of the pure single health conditions comprising the joint condition. Using the mean utility of the pure single health states, the authors predict the joint health state utility using one or more joint state utility estimators and compare the predicted joint state utility to the directly estimated joint state utility. Based on their review of the literature, Ara and Wailoo
      • Ara R.
      • Wailoo A.J.
      Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.
      recommend the multiplicative estimator; a recommendation that is supported by more recent research
      • Thompson A.J.
      • Sutton M.
      • Payne K.
      Estimating joint health condition utility values.
      and by a recent ISPOR task force.
      • Brazier J.
      • Ara R.
      • Azzabi I.
      • et al.
      Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force Report.
      A key practical consideration this literature has identified is the appropriate choice of baseline utility.
      • Ara R.
      • Wailoo A.J.
      Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.
      Theoretically, the absence of all adverse health conditions should result in a health utility of 1; and so, many algorithms for estimating the utility of a joint health state initially used a baseline of 1 as the utility of having neither condition. However, the absence of the 2 health conditions in question does not imply the absence of other possible health conditions that might impact health utility (except in the case of direct utility elicitation wherein such perfect health can be described). As a result, a baseline other than 1 is more appropriate, such as the age-adjusted health utility of those with neither of the conditions under consideration or the age-adjusted health utility of the general population if more granular data are not available.
      • Brazier J.
      • Ara R.
      • Azzabi I.
      • et al.
      Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force Report.
      ,
      • Ara R.
      • Wailoo A.J.
      Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.

      Empirical Issue

      An empirical issue that has not yet been explored in the literature is the impact of not using pure single-condition utilities in joint state utility predictors. If researchers collect or have access to appropriate individual-level data, they can estimate the pure single-condition utilities, and quite possibly the joint health state utility as well. More commonly, researchers use data from published estimates of the mean utility among individuals with the conditions in question,
      • Brazier J.
      • Ara R.
      • Azzabi I.
      • et al.
      Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force Report.
      resulting in a much more limited information set that does not include the pure single-condition utilities. When published estimates of utility for a health condition are used, the “single-state” utility is typically the mean utility across all individuals with the health condition in question, with limited information on any other health conditions they may have. Thus, researchers using population-level estimates often do not know the pure single-condition utilities.
      More specifically, consider 2 health conditions, A and B, and define terms as population means, as detailed in Table 1. We define these terms as population-level means, not directly observed individual-level utilities because population means are more commonly available. We also define 2 information sets: a full-information set and a limited-information set. The full-information set includes at least the minimal information set needed to implement joint health state algorithms as theoretically derived and as tested in most of the methodological literature: UnA, UnB, UAnB, UBnA, and UnAnB. The limited-information set includes data only on UALL, UA, and UB. For ease of discussion, we will refer to UnA, UnB, UAnB, UBnA, and UnAnB as full-information utilities and UALL, UA, and UB as limited-information utilities.
      Table 1Health state utility notation definitions.
      Utility notationDefinition
       UALLMean utility of all individuals
       UAMean utility of all individuals with condition A
       UBMean utility of all individuals with condition B
       UnAMean utility of all individuals without condition A
       UnBMean utility of all individuals without condition B
       UAnBMean utility of individuals with condition A but not B
       UBnAMean utility of individuals with condition B but not A
       UnAnBMean utility of individuals with neither health condition
       UABMean utility of individuals with both conditions A and B
      Our premise is that researchers more commonly have the limited-information set, and seldom have the full-information set without also knowing the joint health state utility, UAB. Although we acknowledge the possibility of an intermediary-information set, in which the researcher has some additional information beyond UALL, UA, and UB, we contend that it is unlikely that a researcher could know UnA, UnB, UAnB, UBnA, and UnAnB without also knowing UAB, in which case there would be no need to use an algorithm to predict the joint health state utility.
      There are both theoretical and empirical relationships among these utilities. Theoretically, UAnB is a function of the attributes of health condition A only, and UBnA is a function of the attributes of health condition B only. Because UAB is a function of the attributes of both health condition A and health condition B, it should be a combination of the attributes determining UAnB and UBnA. More importantly, there is no reason a priori to assume any precise mathematical relationship among these utilities. Thus, other than suggesting UAnB > UAB and UBnA > UAB, this theoretical relationship provides no evidence of the potential performance of joint health state utility estimators. Furthermore, although aggregate data are often used to estimate the theoretical relationship among the health states, this relationship is fundamentally defined at an individual level.
      The empirical relationship among these health states differs from the theoretical relationship primarily because it exists at a population level. Empirically, the mean utility of all individuals with condition A and the mean utility of the subset of individuals with the joint condition AB depends on the proportion of people in the joint state and the pure single-state AnB. Specifically,
      UA=pB|AUAB+(1pB|A)UAnB


      where pB|A is the proportion of all people with condition A that also have condition B. As the prevalence (pB|A) of the joint condition increases, the joint state utility plays an increasingly important role in determining the utility among all people with condition A. In other words, as pB|A increases, UA becomes a better estimate of UAB until, at pB|A = 1, UA = UAB. The relationship between the mean utility of all people with condition A and the joint health state also depends on the difference in utility between the joint condition (AB) and the single-condition health state (AnB). The smaller the difference in utility among the health states, the less important the joint health state is in determining the overall health state utility: as (UAnB – UAB) gets smaller, UA becomes a better estimate of UAB until, at (UAnB – UAB) = 0, UA = UAB = UAnB. Analogous relationships hold for UB and suggest that, under limited information, the relative performance of joint health state estimators will vary based on the prevalence of the joint condition and the relative utility of the 2 conditions.
      Using our notation, we define algorithms for estimating joint health state utilities under full and limited information, as detailed in Table 2. Based on these algorithms and the relationships in equation (1), we can derive the expected bias of each estimator (ie, the predicted joint utility minus UAB) under limited information, also detailed in Table 2.
      Table 2Joint health state utility estimation algorithms under full and partial information.
      EstimatorFull-information algorithmLimited information
      AlgorithmExpected bias
      Minimummin(UAnB, UBnA)min(UA, UB)(1 − pB|A)(UAnB − UAB), if UA < UB

      (1 – pA|B)(UBnA − UAB), if UB < UA
      AdditiveUnAnB – [(UnA – UAnB) + (UnB – UBnA)]UALL – [(UALL – UA) + (UALL – UB)](1 − pB|A)(UAnB − UAB) − (UALL − UB)
      MultiplicativeUnAnB UAnBUnAUBnAUnBUALL UAUALLUBUALL(1 − pB|A) UBUALL UAnB – (1 − pB|A UBUALL)UAB
      Adjusted decrementmin(UAnB, UBnA) – min(UAnB, UBnA) [(1 – UAnB)(1 – UBnA)]min(UA, UB) – min(UA, UB) [(1 – UA)(1 – UB)](1 − pB|A)ZUAnB − (1 − pB|AZ)UAB, if UA < UB

      (1 – pA|B)ZUBnA − (1 – pA|BZ)UAB, if UB < UA

      Z = 1 – (1 – UA)(1 – UB)
      Several important insights emerged from evaluating the expected bias of the estimators. First, in every case the bias is a weighted average of the full-information health state UAnB (or UBnA) and the joint state, with the weight being a function of the prevalence of the joint state, pB|A (or pA|B). Although relevant for all estimators, the bias formulas for the minimum estimator clearly illustrate the importance of pB|A and (UAnB − UAB). It is also clear that the minimum will always overestimate the joint health state utility—that is, have a positive bias. The bias of the additive estimator starts with the same weighted average as the minimum bias but adjusts it additively based on the utility decrement associated with the comorbid condition, making clear the importance of the baseline utility (UALL in our algorithm) in determining the bias of the additive estimator. Because they use proportional utility decrements, both the multiplicative and adjusted decrement estimators adjust the weighting used in the average rather than subtracting an adjustment like the additive estimator. The extent to which these adjustments under or overcompensate for the positive bias of the minimum estimator depends on the relative utilities of the baseline and conditions A and B, but it is possible for the additive, multiplicative, and adjusted decrement estimators to underestimate the joint state utility under limited information.

      Methods

      To empirically estimate the performance of the 4 joint utility estimators under limited information, we compared the predicted joint state utility to the observed joint state utility for AUD and 3 comorbid conditions: cirrhosis, depressive disorder, and nicotine use disorder.
      We used data from NESARC-III, a nationally representative survey of the civilian noninstitutionalized US population, aged 18 or older, including persons living in noninstitutional group quarters.
      • Grant B.F.
      • Amsbary M.
      • Chu A.
      • et al.
      National Epidemiologic Survey on Alcohol and Related Conditions-III (NESARC-III). NIH.
      NESARC-III classifies respondents as having AUD based on the Alcohol Use Disorder and Associated Disabilities Interview Schedule-5 (AUDADIS-5).
      • Grant B.F.
      • Goldstein R.B.
      • Smith S.M.
      • et al.
      The alcohol use disorder and associated disabilities interview Schedule-5 (AUDADIS-5): reliability of substance use and psychiatric disorder modules in a general population sample.
      Individuals are classified into mild, moderate, or severe AUD in the past 12 months or before the past year. For this analysis, we defined AUD as lifetime AUD, combining mild, moderate, and severe for the past 12 months and before the past year. We estimated health utility using the 6-dimensional health state short form (SF-6D), a community-perspective measure of health utility
      • Brazier J.E.
      • Roberts J.
      The estimation of a preference-based measure of health from the SF-12.
      • Brazier J.
      • Roberts J.
      • Deverill M.
      The estimation of a preference-based measure of health from the SF-36.
      • Craig B.M.
      • Pikcard A.S.
      • Stolk E.
      • Brazier J.E.
      US valuation of the SF-6D.
      available from the 12 item short form, version 2 (SF-12v2) in the NESARC-III.
      NESARC-III also collects data on medical, mental health, and substance use conditions. Medical conditions in the past 12 months are self-reported by respondents as having been “diagnosed by a doctor or other health professional.” We used a self-reported diagnosis of cirrhosis of the liver in the past 12 months as our measure of cirrhosis. We measured depressive disorder as an AUDADIS-5 lifetime diagnosis of major (disorder, episodic), dysthymic, manic, or hypomanic depression or any combination thereof. We measured nicotine use disorder as an AUDADIS-5 lifetime diagnosis of a nicotine use disorder, regardless of severity.
      We chose AUD as the primary health condition (ie, condition A) because it is a prevalent and costly health condition with a high incidence of comorbid conditions,
      • Grant B.F.
      • Goldstein R.B.
      • Saha T.D.
      • et al.
      Epidemiology of DSM-5 alcohol use disorder: results from the national epidemiologic survey on alcohol and related conditions III..
      • Bouchery E.E.
      • Harwood H.J.
      • Sacks J.J.
      • Simon C.J.
      • Brewer R.D.
      Economic costs of excessive alcohol consumption in the US, 2006.
      • Rehm J.
      • Taylor B.
      • Mohapatra S.
      • et al.
      Alcohol as a risk factor for liver cirrhosis: a systematic review and meta-analysis.
      • Rehm J.
      • Sherk A.
      • Shield K.
      • Gmel G.
      and numerous alcohol intervention trials have focused specifically on comorbid conditions.
      • Aharonovich E.
      • Hasin D.S.
      Primary drug use types and intervention-related self-monitoring in HIV patients.
      • Bartels S.J.
      • Coakley E.H.
      • Zubritsky C.
      • et al.
      Improving access to geriatric mental health services: a randomized trial comparing treatment engagement with integrated versus enhanced referral care for depression, anxiety, and at-risk alcohol use.
      • Bartlem K.M.
      • Bowman J.
      • Freund M.
      • et al.
      Effectiveness of an intervention in increasing the provision of preventive care by community mental health services: a non-randomized, multiple baseline implementation trial.
      • Berg K.M.
      • Piper M.E.
      • Smith S.S.
      • Fiore M.C.
      • Jorenby D.E.
      Defining and predicting short-term alcohol use changes during a smoking cessation attempt.
      • Bradizza C.M.
      • Stasiewicz P.R.
      • Paas N.D.
      Relapse to alcohol and drug use among individuals diagnosed with co-occurring mental health and substance use disorders: a review.
      • Korcha R.
      • Polcin D.L.
      • Evans K.
      • Bond J.C.
      • Galloway G.P.
      Intensive motivational interviewing for women with concurrent alcohol problems and methamphetamine dependence.
      • McKay J.R.
      • Drapkin M.L.
      • Van Horn D.H.
      • et al.
      Effect of patient choice in an adaptive sequential randomization trial of treatment for alcohol and cocaine dependence.
      • Schadé A.
      • Marquenie L.A.
      • van Balkom A.J.
      • De Beurs E.
      • van Dyck R.
      • van den Brink W.
      Do comorbid anxiety disorders in alcohol-dependent patients need specific treatment to prevent relapse?.
      • Vrdoljak D.
      • Markovic B.B.
      • Puljak L.
      • Lalic D.I.
      • Kranjcevic K.
      • Vučak J.
      Lifestyle intervention in general practice for physical activity, smoking, alcohol consumption and diet in elderly: a randomized controlled trial.
      Using utility and prevalence estimates from the NESARC-III, we chose our 3 comorbid conditions to explore the empirical implications of the prevalence of comorbidity (pB|A) and the potential health utility impact of the comorbidity (UAnB – UAB). Although commonly associated with AUD,
      • Rehm J.
      • Taylor B.
      • Mohapatra S.
      • et al.
      Alcohol as a risk factor for liver cirrhosis: a systematic review and meta-analysis.
      ,
      • Scaglione S.
      • Kliethermes S.
      • Cao G.
      • et al.
      The epidemiology of cirrhosis in the United States.
      a self-reported diagnosis of cirrhosis is rare among those with a lifetime history of AUD in the NESARC-III (small pB|A) but has a large impact on utility (large UAnB – UAB). In contrast, depressive disorder is moderately common among those with AUD in the NESARC-III (moderate pB|A) and has a large impact on utility (large UAnB – UAB). Finally, nicotine use disorder is very common among those with AUD (large pB|A) but has a small impact on utility (small UAnB – UAB). We did not explore a condition with small pB|A and small UAnB – UAB because such a condition would be less empirically relevant.
      We estimated each of the 9 mean health utilities defined above for lifetime AUD and each comorbid condition (cirrhosis, depressive disorder, and nicotine use disorder). For each joint state and under each information set, we estimated the relevant mean health utilities and used them to derive the joint state utility using each estimator, and then compare the estimator-derived joint state utility with the observed joint state’s mean utility.
      We assessed the performance of the 4 estimators under limited information on the criteria of bias and root mean square error (RMSE). Bias reflects how closely the predicted value matches the observed; it is defined as the predicted mean using the estimator minus the observed mean. RMSE reflects how precise the predicted value is in matching the observed; it is defined as the square root of the mean of the squared prediction error. These performance criteria inform how “right” an estimator is and how “good” it is at prediction. Using parametric bootstrapping
      • Efron B.
      • Tibshirani R.
      An Introduction to the Bootstrap.
      that incorporates the sampling design and weights in the NESARC-III, we created 1000 bootstrap replicates for each estimator’s joint utility and calculated the mean bias for each estimator, the 95% confidence interval (CI) for the bias, and the RMSE. An estimator with a mean bias of 0 would be considered unbiased. The magnitude of the RMSE is relative to the mean and so does not have a threshold value for assessing the performance of an estimator; nevertheless, a smaller RMSE indicates a more precise estimator and so is preferred. All analyses were conducted using Stata Version 17 (StataCorp LLC, College Station, TX). Stata code to replicate our analyses is provided in the Appendix in Supplemental Materials found at https://dx.doi.org/10.1016/j.jval.2022.09.009.

      Results

      Table 3 presents the unweighted sample size and the weighted mean utility and 95% CI for all health states, and the key terms from the expected bias formulas [pB|A, pA|B, (UAnB – UAB), and (UBnA – UAB)] in Table 2. The mean utility among the general population (ie, UALL) was 0.788. The lifetime AUD (ie, UA) had a mean utility of 0.762; all 3 comorbid conditions (ie, UB) had a lower mean utility than AUD, with the lowest being associated with cirrhosis.
      Table 3Health state utilities.
      Unweighted nMean utility (SF-6D)95% CI
      Limited-information health states
      General population36 1630.7880.785-0.790
      Lifetime AUD99740.7620.758-0.767
      Cirrhosis1300.6410.605-0.677
      Depressive disorder47720.6700.665-0.676
      Nicotine use disorder72780.7380.733-0.743
      Full-information health states
      No AUD26 1890.7980.795-0.801
      No cirrhosis35 8910.7880.785-0.791
      No depressive disorder31 3910.8050.802-0.807
      No nicotine use disorder28 8850.8000.797-0.803
      AUD, no cirrhosis98680.7630.759-0.768
      AUD, no depressive disorder79070.7870.783-0.791
      AUD, no nicotine use disorder62180.7820.778-0.787
      Cirrhosis, no AUD720.6550.604-0.707
      Depressive disorder, no AUD27050.6790.672-0.686
      Nicotine use disorder, no AUD35220.7510.744-0.758
      No AUD, no cirrhosis26 0230.7990.795-0.802
      No AUD, no depressive disorder23 4840.8110.808-0.814
      No AUD, no nicotine use disorder22 6670.8050.802-0.808
      Joint health states
      AUD and cirrhosis580.6220.564-0.679
      AUD and depressive disorder20670.6590.652-0.666
      AUD and nicotine use disorder37560.7280.721-0.734
      Key terms from expected bias formulas in Table 2
      Assumes AUD is condition A and the other conditions listed are condition B.
      pB|ApA|B(UAnB – UAB)(UBnA – UAB)
      AUD and cirrhosis0.0050.4380.1420.034
      AUD and depressive disorder0.1920.4690.1280.013
      AUD and nicotine use disorder0.5320.3660.0610.023
      AUD indicates alcohol use disorder; CI, confidence interval; SF-6D, six-dimensional health state short form.
      Assumes AUD is condition A and the other conditions listed are condition B.
      Table 4 presents the 4 joint health state estimators for each of the comorbid conditions under both full and limited information. As expected, the minimum estimator overestimated the joint health state utility in all cases, but less so under limited information. The additive and multiplicative estimators overestimated the joint health state utility under full information but underestimated it under limited information. The adjusted decrement estimator always underestimated the joint health state utility. This underestimation was worse under limited information than under full information.
      Table 4Joint health state estimators under full and limited information.
      AUD and cirrhosisAUD and depressive disorderAUD and nicotine use disorder
      Full informationLimited informationFull informationLimited informationFull informationLimited information
      Observed0.6220.6590.728
      Minimum0.6540.6410.6790.6700.7510.738
      Additive0.6300.6160.6750.6450.7400.713
      Multiplicative0.6340.6210.6750.6490.7410.715
      Adjusted decrement0.6020.5860.6330.6180.7100.692
      AUD indicates alcohol use disorder.
      Table 5 presents bootstrap results for the limited-information estimators and Figure 1 presents these results graphically. For cirrhosis, which had the lowest conditional prevalence but the highest utility impact, the 95% CI for the bias included 0 for all estimators. The difference between the upper and lower confidence bounds and the RMSE, however, reveals that all estimators were very imprecise. This is illustrated in the relatively broad empirical distribution of the bias for the AUD and cirrhosis joint estimators shown in Figure 1. In contrast, the joint health state estimators for depression and nicotine use disorder were much more precise. For both conditions, the 95% CIs excluded 0 across all estimators and so all were biased: the minimum was biased upwards whereas all other estimators were biased downwards. Across all conditions, the adjusted decrement estimator had the highest RMSE, whereas the minimum estimator had the lowest RMSE for all conditions except cirrhosis.
      Table 5Bias and RMSE of joint health state estimators under limited information.
      AUD and cirrhosisAUD and depressive disorderAUD and nicotine use disorder
      Bias95% CIRMSEBias95% CIRMSEBias95% CIRMSE
      Minimum0.021−0.040 to 0.0840.0370.0110.003 to 0.0120.0120.0110.003 to 0.0180.011
      Additive−0.005−0.063 to 0.0590.032−0.014−0.024 to −0.0050.015−0.015−0.024 to −0.0050.015
      Multiplicative−0.000−0.058 to 0.0630.031−0.011−0.019 to −0.0010.012−0.013−0.022 to −0.0040.014
      Adjusted decrement−0.034−0.095 to 0.0300.047−0.042−0.050 to −0.0330.042−0.035−0.043 to −0.0270.035
      AUD indicates alcohol use disorder; CI, confidence interval; RMSE, root mean squared error.
      Figure thumbnail gr1
      Figure 1Empirical distribution of bias for joint health state estimators under limited information.
      AUD indicates alcohol use disorder.

      Discussion

      This article explored joint health state utility estimation using a limited-information set that assumes researchers only know the mean health utility of people with each of the conditions, with no information on other conditions they may have, if any. We also defined a full-information set that included the mean utility among samples defined by the presence and absence of each of the conditions, yielding the “pure” single-condition utilities called for in most joint utility estimators.
      We found that the available information set has potentially important effects on the performance of joint utility estimators. Under full information, all estimators except the adjusted decrement estimator overestimated the joint health state utility. In contrast, under limited information, all estimators except the minimum estimator underestimated the joint health state utility. This behavior results from the full-information set using “pure” health states that isolated each of the single-state health conditions. In contrast, the “single” state utilities used in the limited-information set included both individuals in the “pure” single-state conditions and the joint condition. Thus, despite having less precise information on health states, the limited-information set included data on the joint state utilities that the full-information set intentionally excluded. The inclusion of the joint health state utility caused the underestimation of the joint health state utility under limited information, and its exclusion caused overestimation under full information.
      When we compared the empirical performance of the 4 estimators under limited information, we found that the minimum had the best performance based on RMSE. This conclusion is somewhat undermined by the performance of the minimum estimator in the very rare comorbidity of cirrhosis, but we contend that the better RMSE for depression and nicotine use disorder outweighed the performance for cirrhosis. Across all conditions, we conclusively eliminated the adjusted decrement estimator as defined in the literature. The adjusted decrement estimator was the only estimator that used 1 as the baseline utility. It is well established that this baseline causes an underestimation of joint state utilities because it overstates the decrement in utility associated with the single-condition health states.
      • Ara R.
      • Wailoo A.J.
      Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.
      We speculate that the performance of the adjusted decrement estimator may improve if another baseline utility is considered. The additive and multiplicative estimators were surprisingly similar in all cases, despite recommendations that the multiplicative is preferred under full information.
      • Brazier J.
      • Ara R.
      • Azzabi I.
      • et al.
      Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force Report.
      ,
      • Ara R.
      • Wailoo A.J.
      Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.
      ,
      • Thompson A.J.
      • Sutton M.
      • Payne K.
      Estimating joint health condition utility values.
      The expected bias formulas detailed in Table 1 reveal that the relative performance of these estimators is driven more by the difference in the utility decrement relative to the baseline than by the prevalence of the joint condition. Our empirical evidence suggests that large differences in the relative utility decrement are needed to create an appreciable difference in performance between the additive and multiplicative estimators under limited information.

      Limitations

      Our results are based on a single data set, using a single utility measure, and focused on a single clinical area. Analyses using data from a different population, a different utility measure, or focusing on a different clinical area may yield different results. Furthermore, our study focused on the utility of joint health states defined by 2 co-occurring health conditions and so may not extend to joint health states defined by 3 or more conditions. Studies using full-information utilities to explore joint health states defined by 3 or more conditions
      • Thompson A.J.
      • Sutton M.
      • Payne K.
      Estimating joint health condition utility values.
      suggest that our results may not hold; future research could extend our work to assess the performance of joint health state estimators in such circumstances.
      In addition, our inclusion of cirrhosis as a co-occurring condition is debatable given its very low sample size in the NESARC-III. Nevertheless, its low sample size is why it offers an interesting point of comparison. From a clinical perspective, relatively rare concurrent conditions may be important to include. Indeed, rare conditions can be very expensive and, thus, have important implications for resource allocation. Thus, researchers need guidance on how to include rare conditions in a CEA. Furthermore, our use of cirrhosis reflects the possibility that a concurrent condition may be common among those with a specific condition but not well measured in epidemiological data: research suggests that 20% of heavy drinkers will develop cirrhosis
      • Rehm J.
      • Taylor B.
      • Mohapatra S.
      • et al.
      Alcohol as a risk factor for liver cirrhosis: a systematic review and meta-analysis.
      but that it is undiagnosed in 69% of those with cirrhosis.
      • Scaglione S.
      • Kliethermes S.
      • Cao G.
      • et al.
      The epidemiology of cirrhosis in the United States.
      Thus, we feel it offers information on a possibly important analytical scenario.

      Conclusions

      Our analysis revealed that researchers using joint health state utility estimators need to understand the information set they have at their disposal. In most cases, researchers will either have empirical data on the joint health state or be forced to use limited-information health states—meaning those utilities that do not isolate single-condition health states to exclude the presence of any co-occurring condition—to estimate the joint health state utility. All estimators performed differently under partial information than under full information. The joint utility estimator literature is dominated by studies that assume full-information utilities are available yet use the mean of a preference-based measure that captures all conditions experienced by the respondent when testing the empirical performance of varying estimators. Additional studies on this issue are needed and current methodological guidance needs to be reconsidered. In the absence of these additional studies, we recommend using the minimum of the single-state utilities as the joint health state utility estimate. The minimum has 3 distinct advantages: first, it is precise, as indicated by its relatively low RMSE; second, it has a consistent upward bias, which makes correction predictable; and third, it is easy to implement, explain, and understand, which is an important consideration for practitioners and decision-makers. Although we found all other estimators underestimated the joint health state in our empirical work, the analytical derivation of their bias revealed that this need not always be the case; the lack of a clear a priori direction of bias for estimators complicates interpretation and potentially confidence in results. The preferred approach is, of course, to use empirically estimated utilities for joint health states, which are becoming increasingly available in the peer-reviewed literature.
      • Wittenberg E.
      • Bray J.W.
      • Gebremariam A.
      • Aden B.
      • Nosyk B.
      • Schackman B.R.
      Joint utility estimators in substance use disorders.
      ,
      • Hanmer J.
      • Kaplan R.M.
      Update to the report of nationally representative values for the noninstitutionalized US adult population for five health-related quality-of-life scores.
      • Hanmer J.
      • Lawrence W.F.
      • Anderson J.P.
      • Kaplan R.M.
      • Fryback D.G.
      Report of nationally representative values for the noninstitutionalized US adult population for 7 health-related quality-of-life scores.
      • Wittenberg E.
      • Barbosa C.
      • Hein R.
      • Hudson E.
      • Thornburg B.
      • Bray J.W.
      Health-related quality of life of alcohol use disorder with co-occurring conditions in the US population.
      But when these estimates are not available, it is imperative that researchers understand the information set they have available to them and follow the guidance that is consistent with that set.

      Article and Author Information

      Author Contributions: Concept and design: Bray, Barbosa, Thornburg, Gebreselassie, LaButte, Wittenberg
      Acquisition of data: Bray
      Analysis and interpretation of data: Bray, Thornburg, Gebreselassie, LaButte, Barbosa, Wittenberg
      Drafting of the manuscript: Bray, Gebreselassie, Barbosa, Wittenberg
      Critical revision of the paper for important intellectual content: Bray, Barbosa, Wittenberg
      Statistical analysis: Bray, Thornburg, Gebreselassie, LaButte
      Obtaining funding: Bray, Barbosa, Wittenberg
      Supervision: Bray
      Conflict of Interest Disclosures: Dr Bray reported receiving grants from the National Institute on Alcohol Abuse and Alcoholism during the conduct of the study. Messrs Thornburg, Gebreselassie, and LaButte reported receiving grants from the National Institute on Alcohol Abuse and Alcoholism during the conduct of the study. Drs Barbosa and Wittenberg reported receiving grants from the National Institute on Alcohol Abuse and Alcoholism, during the conduct of the study. No other disclosures were reported. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
      Funding/support: Research reported in this publication was supported by the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health under award number R15AA027655-01S1 .
      Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

      Acknowledgment

      The authors thank Will Queen for the helpful comments on the bias formulas and Riley Hein and Emma Hudson for research assistance.

      Supplemental Materials

      References

      1. Guide to the methods of technology appraisal.
        (NICE)
        • Sanders G.D.
        • Neumann P.J.
        • Basu A.
        • et al.
        Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine.
        JAMA. 2016; 316: 1093-1103
        • Basu A.
        • Dale W.
        • Elstein A.
        • Meltzer D.
        A linear index for predicting joint health-states utilities from single health-states utilities.
        Health Econ. 2009; 18: 403-419
        • Dale W.
        What is the best model for estimating joint health states utilities? Comparing the linear index model to the proportional decrement model.
        Med Decis Making. 2010; 30: 531-533
        • Fu A.Z.
        • Kattan M.W.
        Utilities should not be multiplied: evidence from the preference-based scores in the United States.
        Med Care. 2008; 46: 984-990
        • Hu B.
        • Fu A.Z.
        Predicting utility for joint health states: a general framework and a new nonparametric estimator.
        Med Decis Making. 2010; 30: E29-E39
        • Ara R.
        • Brazier J.E.
        Populating an economic model with health state utility values: moving toward better practice.
        Value Health. 2010; 13: 509-518
        • Brazier J.
        • Ara R.
        • Azzabi I.
        • et al.
        Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force Report.
        Value Health. 2019; 22: 267-275
        • Dale W.
        • Basu A.
        • Elstein A.
        • Meltzer D.
        Predicting utility ratings for joint health States from single health states in prostate cancer: empirical testing of 3 alternative theories.
        Med Decis Making. 2008; 28: 102-112
        • Loeb S.
        • Zhou Q.
        • Siebert U.
        • et al.
        Active surveillance versus watchful waiting for localized prostate cancer: a model to inform decisions.
        Eur Urol. 2017; 72: 899-907
        • Wittenberg E.
        • Bray J.W.
        • Gebremariam A.
        • Aden B.
        • Nosyk B.
        • Schackman B.R.
        Joint utility estimators in substance use disorders.
        Value Health. 2017; 20: 458-465
        • Schackman B.R.
        • Gutkind S.
        • Morgan J.R.
        • et al.
        Cost-effectiveness of hepatitis C screening and treatment linkage intervention in US methadone maintenance treatment programs.
        Drug Alcohol Depend. 2018; 185: 411-420
        • Avanceña A.L.V.
        • Miller N.
        • Uttal S.E.
        • Hutton D.W.
        • Mellinger J.L.
        Cost-effectiveness of alcohol use treatments in patients with alcohol-related cirrhosis.
        J Hepatol. 2021; 74: 1286-1294
        • Ara R.
        • Brazier J.
        Estimating health state utility values for comorbidities.
        Pharmacoeconomics. 2017; 35: 89-94
        • Ara R.
        • Wailoo A.J.
        Estimating health state utility values for joint health conditions: a conceptual review and critique of the current evidence.
        Med Decis Making. 2013; 33: 139-153
        • Keeney R.L.
        • Raiffa H.
        Decisions With Multiple Objectives: Preferences and Value Tradeoffs.
        Cambridge University Press, Cambridge, England1993
        • Dale W.
        • Bilir S.P.
        • Hemmerich J.
        • Basu A.
        • Elstein A.
        • Meltzer D.
        The prevalence, correlates, and impact of logically inconsistent preferences in utility assessments for joint health states in prostate cancer.
        Med Care. 2011; 49: 59-66
        • Triantaphyllou E.
        • Yanase J.
        How to identify and treat data inconsistencies when eliciting health-state utility values for patient-centered decision making.
        Artif Intell Med. 2020; 106: 101882
        • Anderson N.
        • Zalinski J.
        Functional measurement approach to self-estimation in multiattribute evaluation.
        J Behav Decis Mak. 1988; 1: 191-221
        • Thompson A.J.
        • Sutton M.
        • Payne K.
        Estimating joint health condition utility values.
        Value Health. 2019; 22: 482-490
        • Grant B.F.
        • Amsbary M.
        • Chu A.
        • et al.
        National Epidemiologic Survey on Alcohol and Related Conditions-III (NESARC-III). NIH.
        https://www.niaaa.nih.gov/research/nesarc-iii
        Date accessed: October 19, 2022
        • Grant B.F.
        • Goldstein R.B.
        • Smith S.M.
        • et al.
        The alcohol use disorder and associated disabilities interview Schedule-5 (AUDADIS-5): reliability of substance use and psychiatric disorder modules in a general population sample.
        Drug Alcohol Depend. 2015; 148: 27-33
        • Brazier J.E.
        • Roberts J.
        The estimation of a preference-based measure of health from the SF-12.
        Med Care. 2004; 42: 851-859
        • Brazier J.
        • Roberts J.
        • Deverill M.
        The estimation of a preference-based measure of health from the SF-36.
        J Health Econ. 2002; 21: 271-292
        • Craig B.M.
        • Pikcard A.S.
        • Stolk E.
        • Brazier J.E.
        US valuation of the SF-6D.
        Med Decis Making. 2013; 33: 793-803
        • Grant B.F.
        • Goldstein R.B.
        • Saha T.D.
        • et al.
        Epidemiology of DSM-5 alcohol use disorder: results from the national epidemiologic survey on alcohol and related conditions III..
        JAMA Psychiatry. 2015; 72: 757-766
        • Bouchery E.E.
        • Harwood H.J.
        • Sacks J.J.
        • Simon C.J.
        • Brewer R.D.
        Economic costs of excessive alcohol consumption in the US, 2006.
        Am J Prev Med. 2011; 41: 516-524
        • Rehm J.
        • Taylor B.
        • Mohapatra S.
        • et al.
        Alcohol as a risk factor for liver cirrhosis: a systematic review and meta-analysis.
        Drug Alcohol Rev. 2010; 29: 437-445
        • Rehm J.
        • Sherk A.
        • Shield K.
        • Gmel G.
        Risk relations between alcohol use and non-injury causes of death. CAMH.
        • Aharonovich E.
        • Hasin D.S.
        Primary drug use types and intervention-related self-monitoring in HIV patients.
        Drug Alcohol Depend. 2015; 146 (e203)
        • Bartels S.J.
        • Coakley E.H.
        • Zubritsky C.
        • et al.
        Improving access to geriatric mental health services: a randomized trial comparing treatment engagement with integrated versus enhanced referral care for depression, anxiety, and at-risk alcohol use.
        Am J Psychiatry. 2004; 161: 1455-1462
        • Bartlem K.M.
        • Bowman J.
        • Freund M.
        • et al.
        Effectiveness of an intervention in increasing the provision of preventive care by community mental health services: a non-randomized, multiple baseline implementation trial.
        Implement Sci. 2016; 11: 46
        • Berg K.M.
        • Piper M.E.
        • Smith S.S.
        • Fiore M.C.
        • Jorenby D.E.
        Defining and predicting short-term alcohol use changes during a smoking cessation attempt.
        Addict Behav. 2015; 48: 52-57
        • Bradizza C.M.
        • Stasiewicz P.R.
        • Paas N.D.
        Relapse to alcohol and drug use among individuals diagnosed with co-occurring mental health and substance use disorders: a review.
        Clin Psychol Rev. 2006; 26: 162-178
        • Korcha R.
        • Polcin D.L.
        • Evans K.
        • Bond J.C.
        • Galloway G.P.
        Intensive motivational interviewing for women with concurrent alcohol problems and methamphetamine dependence.
        J Subst Abuse Treat. 2014; 46: 113-119
        • McKay J.R.
        • Drapkin M.L.
        • Van Horn D.H.
        • et al.
        Effect of patient choice in an adaptive sequential randomization trial of treatment for alcohol and cocaine dependence.
        J Consult Clin Psychol. 2015; 83: 1021-1032
        • Schadé A.
        • Marquenie L.A.
        • van Balkom A.J.
        • De Beurs E.
        • van Dyck R.
        • van den Brink W.
        Do comorbid anxiety disorders in alcohol-dependent patients need specific treatment to prevent relapse?.
        Alcohol Alcohol. 2003; 38: 255-262
        • Vrdoljak D.
        • Markovic B.B.
        • Puljak L.
        • Lalic D.I.
        • Kranjcevic K.
        • Vučak J.
        Lifestyle intervention in general practice for physical activity, smoking, alcohol consumption and diet in elderly: a randomized controlled trial.
        Arch Gerontol Geriatr. 2014; 58: 160
        • Scaglione S.
        • Kliethermes S.
        • Cao G.
        • et al.
        The epidemiology of cirrhosis in the United States.
        J Clin Gastroenterol. 2015; 49: 690-696
        • Efron B.
        • Tibshirani R.
        An Introduction to the Bootstrap.
        57. Chapman & Hall/CRC, New York, NY1993
        • Hanmer J.
        • Kaplan R.M.
        Update to the report of nationally representative values for the noninstitutionalized US adult population for five health-related quality-of-life scores.
        Value Health. 2016; 19: 1059-1062
        • Hanmer J.
        • Lawrence W.F.
        • Anderson J.P.
        • Kaplan R.M.
        • Fryback D.G.
        Report of nationally representative values for the noninstitutionalized US adult population for 7 health-related quality-of-life scores.
        Med Decis Making. 2006; 26: 391-400
        • Wittenberg E.
        • Barbosa C.
        • Hein R.
        • Hudson E.
        • Thornburg B.
        • Bray J.W.
        Health-related quality of life of alcohol use disorder with co-occurring conditions in the US population.
        Drug Alcohol Depend. 2021; 221: 108558