If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Using a Discrete-Choice Experiment Involving Cost to Value a Classification System Measuring the Quality-of-Life Impact of Self-Management for Diabetes
Address correspondence to: Donna Rowen, School of Health and Related Research, University of Sheffield, Regent Court, 30 Regent Street, Sheffield S1 4DA, UK.
To describe the use of a novel approach in health valuation of a discrete-choice experiment (DCE) including a cost attribute to value a recently developed classification system for measuring the quality-of-life impact (both health and treatment experience) of self-management for diabetes.
Methods
A large online survey was conducted using DCE with cost on UK respondents from the general population (n = 1497) and individuals with diabetes (n = 405). The data were modeled using a conditional logit model with robust standard errors. The marginal rate of substitution was used to generate willingness-to-pay (WTP) estimates for every state defined by the classification system. Robustness of results was assessed by including interaction effects for household income.
Results
There were some logical inconsistencies and insignificant coefficients for the milder levels of some attributes. There were some differences in the rank ordering of different attributes for the general population and diabetic patients. The WTP to avoid the most severe state was £1118.53 per month for the general population and £2356.02 per month for the diabetic patient population. The results were largely robust.
Conclusions
Health and self-management can be valued in a single classification system using DCE with cost. The marginal rate of substitution for key attributes can be used to inform cost-benefit analysis of self-management interventions in diabetes using results from clinical studies in which this new classification system has been applied. The method shows promise, but found large WTP estimates exceeding the cost levels used in the survey.
Discrete choice experiment (DCE) methods are increasingly being applied as a means to value the benefits of health care interventions. DCEs have usually been used to value the process of health care (either in isolation of or in combination with health outcomes) using bespoke or study-specific attributes developed for individual studies [
]. Recent DCE applications in diabetes include, for example, an investigation of patient preferences for insulin therapy and clinical outcomes in type 2 diabetes [
]. Recent work has extended the use of DCE to valuing classification systems for measuring health-related quality of life (HRQOL) such as the five-level EuroQol five-dimensional questionnaire (EQ-5D-5L) [
] on the 0 to 1 quality-adjusted life-year (QALY) scale by adding an additional attribute for duration. This approach has been referred to as the DCETTO (TTO, time trade-off) approach in the literature [
]. These health state utility values can then be used to estimate QALYs for use in cost-utility analysis and for submission to regulatory agencies such as the National Institute of Health and Care Excellence in the United Kingdom [
An alternative to cost-utility analysis is cost-benefit analysis (CBA), in which the benefits of interventions are represented by monetary values. This approach has been used to capture the benefits of interventions beyond the health outcomes achieved including benefits gained from the process of care delivery. Monetary values of the benefits of interventions are often measured directly by asking respondents how much they would be willing to pay for one intervention over another (e.g., [
] by including cost as an additional attribute in the DCE. As mentioned previously, this approach has been applied recently in diabetes (e.g., in the study by Feher et al. [
]). Nevertheless, to our knowledge, to date this methodology has not been used to value a pre-existing classification system. Application of a DCE approach in this context provides an analogous way of valuing health to DCETTO used with the EQ-5D and the SF-6D, but provides monetary estimates of the WTP to avoid a health state that can potentially be used to inform CBA.
Diabetes costs across the globe are rising because of the increased prevalence of the disease and the increased complexity of its treatment. For example, in the United Kingdom, £936.7 million was spent on prescriptions for diabetes in 2015 [
]. It benefits patients by giving them the confidence and skills to self-manage their condition, but of those newly diagnosed, less than 6% have been recorded as attending such a course. Evaluating the true monetary value of interventions designed to improve self-management is urgently needed. Self-management of diabetes varies from one individual to another, and similarly the impact of self-management upon an individual is also very personal.
Currently, quality of life is linked to QALYs on the basis of questionnaires that are not diabetes-specific (e.g., the EQ-5D or the SF-12), and so the intended improvement in self-management skills of an intervention cannot be evaluated in economic terms. Likewise, measures that are diabetes-specific (e.g., Problem Areas in Diabetes [
]) are not linked to QALYs. The classification system used in this study was developed to provide a formal and consistent way to take account of self-management across different interventions, because existing measures do not consider the direct impact of different self-management regimes on patients’ quality of life from their own perspective [
]. Without the use of a single widely applicable classification system, the change in processes is often measured using study-specific descriptions or vignettes, rather than assessing the impact on quality of life through the use of patient-completed questionnaires in clinical studies.
This article describes the use of DCE including a cost attribute to value a classification system measuring the quality-of-life impact of self-management for diabetes. The article presents a DCE survey with a cost attribute conducted in general population and diabetic patient samples as well as the results of regression analyses to model the DCE data to provide monetary values of the WTP to avoid each state defined by the classification system for both general population and diabetic patient samples. We then discuss the results in terms of the implications for valuing this and other classification systems using this method.
Methods
Classification System
The Health and Self-Management in Diabetes classification system was developed to capture the impact of self-management on quality of life in diabetes (see Fig. 1). Four of the dimensions (mood, hypoglycemic attacks, vitality, and social limitations) represent HRQOL and the remaining four dimensions (control, hassle, stress, and support) represent self-management. The dimensions of HRQOL are taken from the Diabetes Health Profile-Five Dimension [
Mulhern B, Labeit A, Rowen D, et al. Developing preference based measures for diabetes: DHP-3D and DHP-5D. Diabet Med [published online ahead of print May 6, 2017]. doi:10.1111/dme.13377.
The Diabetes Health Profile (DHP): a new instrument for assessing the psychosocial profile of insulin requiring patients—development and psychometric evaluation.
Adaptation of the Diabetes Health Profile (DHP-1) for use with patients with type 2 diabetes mellitus: psychometric evaluation and cross-cultural comparison.
DCE tasks present two or more profiles, in which each profile consists of attribute levels selected from a classification system and respondents are asked to indicate their preferred profile. DCE was selected in this study because it enables WTP values to be generated for every state defined by the classification system through the inclusion of a cost attribute, and the technique is amenable to online data collection [
Limited guidance is provided in the DCE literature about how to choose levels for a cost attribute, and many published studies are either extremely brief in their details of how they determined the levels for the cost attribute or do not report details at all. Nevertheless, the levels should accurately capture the range of preferences for most of the respondents; otherwise, their inclusion will not add any useful information. It is important to ensure the levels are not too high or too low for the treatment or condition being valued, because otherwise cost would be either prohibitive or irrelevant [
]. It has been argued that the range for the cost attribute levels should incorporate values that are higher than the market price, because this may not be the maximum amount that people are willing to pay [
]. Typically, cost levels used in the literature reflect a range around mean cost that includes either a low cost or zero cost. In terms of wording the cost attribute, previous DCE experiments with a cost attribute in diabetes have used “personal cost to you each month” [
To empirically inform the selection of the levels of the cost attribute, an online binary choice survey of 400 members of the general population was conducted to assess people’s WTP for hypothetical self-management and HRQOL states (recruitment followed the same process for the main general population survey reported later). Respondents completed experimental binary choice questions, where each question was a choice between a poor state with zero cost or a good state with nonzero cost that was randomly varied across different questions using various levels (across different survey versions the levels used were £10, £25, £50, £75, £100, £150, £200, £300, £400, and £600). The proportions of respondents choosing the better state with nonzero cost were compared to determine how the different costs impacted on choice.
Four levels of the cost attribute were selected, in common with the four severity levels of all other attributes. In the literature of DCE involving a cost attribute to determine WTP, the severity levels are not usually equal but increase exponentially (e.g., 2, 4, 8, 16), and this approach was used here to inform the selection of levels. The lowest level of £10 was selected because about a quarter of the respondents being asked the question were not willing to pay £10 to improve their health. The highest level of £600 was selected as the upper end because about one-third of the respondents were willing to pay £600 to improve their health. Levels of £75 and £200 were selected to represent the intermediate cost levels to ensure good coverage.
Selecting Profiles
Eight attributes from the classification system plus four cost levels resulted in 262,144 profiles, and many millions of possible pairs. Therefore, a subset of profiles was selected using D-optimal methods in Ngene software, distributed by ChoiceMetrics [
] to produce a design that enables estimation of the parameters in a prespecified regression model with precision. The design selected 120 choice sets across 10 survey versions (12 per survey) and was piloted (50 respondents) to generate priors for each attribute level of each dimension. These priors were then used to generate the design (i.e., select 120 choice sets) for the main study. The rationale for this was to improve the efficiency of the design with information regarding the magnitude of the parameter values.
Respondents
Presently, there is no consensus in the literature about whether values should be obtained from the general population or from a patient sample [
]. This article does not propose to review this debate, but given the diversity of viewpoints, values have been collected from both general population and individuals with diabetes.
General population sample
Respondents were recruited by a market research agency using an existing online panel. Respondents were selected to obtain a representative sample of the UK population in terms of age and sex. They were rewarded for their participation with a nominal amount of vouchers that could be accumulated and exchanged for goods.
Patient sample
Diabetic patients were recruited in various ways to ensure a good coverage across different groups in terms of severity, background, and setting:
1.
Posters in clinics and invitation letters posted with the usual clinic appointment letter to patients at Sheffield Teaching Hospitals;
2.
Invitation letters posted to members of the Sheffield Diabetes Research Database who have previously provided consented to receive information about diabetes research projects;
3.
Diabetes UK and DAFNE Online advertised a link to the online survey;
4.
Twitter and Facebook posts to diabetes charities, organizations, and the University of Sheffield staff and students and using a social media marketing company.
All patient respondents were offered optional entry into a prize draw for £50 shopping vouchers per 50 respondents.
The DCE Survey
Respondents were presented with the project information sheet and consented. They then completed sociodemographic and health questions asking about sex, age, education level, annual household income, and whether they have diabetes. After this, respondents were provided with information about what it is like to live with diabetes and were asked to complete the Health and Self-Management in Diabetes classification system for themselves if they had diabetes or imagine someone with diabetes to familiarize themselves with the classification system. Respondents were then given an introduction to the DCE questions (shown in Supplemental Materials found at doi:10.1016/j.jval.2017.06.016). Respondents then answered 1 practice DCE question plus 12 DCE questions, where respondents chose between health description A and health description B, an example question of which is shown in Figure 2. Respondents were told to imagine that these health descriptions would last for 10 years and then they would die. Attributes that differed across the two health descriptions were highlighted in yellow. Finally, patients were also asked about their self-management of diabetes.
The draft survey instrument was shown to a general patient and public involvement panel for comments and piloted with 50 members of the general population to inform the final survey design.
Analysis
Sociodemographic and self-reported health characteristics were analyzed and categorized.
The DCE data were modeled using the following model specification:
(1)
where represents cost, is the coefficient for costs, and represents the coefficients for the 24 nonreference attribute levels of the classification system. This model produces unanchored values for scenarios or combinations of attribute levels. The monetary value (WTP) of each level of each attribute is estimated by dividing the coefficient attached to the relevant attribute level by the coefficient attached to the cost attribute (after previous checking for the linearity of the cost attribute) to generate the marginal rate of substitution,. This enables the monetary value of self-management alone, or the combined monetary value of HRQOL and self-management, to be estimated. Models were estimated separately for the general population and the diabetic patient samples. Models were estimated using the conditional logit model with robust standard errors, and confidence intervals and standard errors of the WTP estimates were determined using the delta method. Model performance was examined using sign, significance, and logical consistency of coefficients, log likelihood, and pseudo-R2. The models assumed that cost was linear and a continuous variable. This was examined by modeling cost as a categorical variable and plotting the cost coefficients [
Regressions were re-estimated excluding all responses to the DCE questions that were achieved within an implausibly short time period because it is likely that such quick responses indicate that these respondents are less likely to have read, understood, and considered the profiles. Regression models were also estimated to determine whether sociodemographic characteristics impacted on the results, including main effects plus interaction effects (one at a time) for the sociodemographic characteristics of age, sex, low income, high income, EQ-5D score, and whether the respondent has diabetes.
Results
The Sample
The characteristics of the general population and diabetic patient samples compared with those of the UK general population are presented in Table 1. The general population sample has a similar proportion of males and similar employment status to the UK general population, but has a lower proportion of respondents older than 65 years. The diabetic patient samples have a lower proportion of males, respondents with a degree qualification, employed respondents, and respondents in the higher income category; a larger proportion of respondents older than 65 years; and a lower EQ-5D-5L score (scored using the cross-walk from EQ-5D-5L to EQ-5D [
]). For a single DCE task level, 37.1% of responses were completed within 10 seconds, 60.2% of responses were completed within 20 seconds, 94.1% of responses were completed within 1 minute, and 99.1% of responses were completed within 3 minutes.
Table 1Samples of respondents to the DCE survey, by population, in comparison with the UK general population
Statistics for England in the Census 2011. The census includes persons aged 16 y and older, whereas this study surveys only persons aged 18 y and older.
Age distribution is here reported as the percentage of all adults aged 18 y and older.
† Statistics for England in the Census 2011. The census includes persons aged 16 y and older, whereas this study surveys only persons aged 18 y and older.
Regression results for the conditional logit model with robust standard errors and the WTP values generated using the marginal rate of substitution are presented in Table 2. The specified model is acceptable for both the general population and the patient population using log likelihood and pseudo-R2. The WTP values reflect the amount that respondents would be willing to pay each month in pound sterling (£) to avoid decrement in the HRQOL or the self-management attributes.
Table 2Regression analysis of DCE survey responses, by population, reporting unanchored and WTP estimates
Variable
General population unanchored estimates (n = 1497)
Diabetic population unanchored estimates (n = 405)
Note. Pseudo-R2 = 0.160 for general population model; pseudo-R2 = 0.101 for diabetic population model. Log likelihood = −10458.4 for general population model; log likelihood = −3027.3 for diabetic population model.
DCE, discrete-choice experiment; WTP, willingness to pay.
The cost coefficient has the expected negative coefficient, showing that individuals are willing to pay to avoid decrements in health or self-management outcomes. Previous analyses (not reported) indicated that it was appropriate to assume cost was linear and continuous, using a plot for each sample of the cost levels and coefficients estimated using a model in which cost was included as a categorical variable.
The estimated coefficients of the HRQOL and the self-management attributes are all negative and logically consistent, with the exception of level 2 coefficients for mood, vitality, and support, although only the vitality level 2 coefficient is approaching significance (at the 10% level). Out of a possible 25 coefficients, 21 and 17 are statistically significant in the models in the general population and the diabetic patient population, respectively. All level 3 and level 4 coefficients are negative and significant with the exception of the level 3 coefficient for vitality in the model estimated for the diabetic patient population.
Table 3 presents the rank ordering of the level 4 coefficients for the general population and the diabetic patient samples, which indicate the ordering of which attributes have the largest impact on WTP at the most severe level. There are noticeable differences, particularly for vitality and mood.
Table 3Absolute size ranking of level 4 WTP estimates, by population
Figure 3 shows the relative size of the WTP values for the general population and diabetic patient samples and shows the relative importance of each of the attributes. The size of the WTP values is similar to those of the HRQOL and the self-management attributes, implying that self-management and HRQOL are equally important. In general, the coefficients of the patient sample tend to be larger than those of the general population sample for more severe levels of each attribute, but smaller for the least severe level of each attribute. This suggests that patients are willing to pay less to avoid mild health states but are willing to pay more to avoid severe health states in comparison with the general population. For example, the WTP to avoid the mild state of state 22222222 is £121.99 per month for the general population and £64.02 per month for the diabetic patient population. In contrast, the WTP to avoid the most severe state of 44444444 is £1118.53 per month for the general population and £2356.02 per month for the diabetic patient population.
Fig. 3Anchored coefficients for the general population and diabetic patients.
Robustness analyses excluded all responses for a given DCE task that were answered in less than 5 seconds (19% and 5% of responses for general population and patients, respectively) and in less than 10 seconds, respectively (40% and 16% of responses for general population and patients, respectively), resulting in larger WTP values at levels 3 and 4. This suggests that the exclusion of responses from respondents who may not have understood or engaged with the task will increase the size of the WTP estimates. Regression models including main effects plus interaction effects (one at a time) for the sociodemographic characteristics of age, sex, low income, high income, EQ-5D score, and whether the respondent has diabetes found no distinguishable pattern in terms of which coefficients were significant. This was surprising for the income variables, given that it was anticipated that WTP may increase as annual household income increases.
Discussion
This article describes the use of DCE including a cost attribute to value a classification system describing the impact of health and self-management of diabetes on quality of life. The classification system is distinct in that it provides a formal and consistent way to take account of self-management across different interventions and one that is not intervention-specific. Nevertheless, to use this information to inform health technology assessment, a means of weighting the severity levels of each attribute to produce a single score is required. This article describes a novel application of the technique of DCE involving a cost attribute to enable the valuation of a standardized classification system to provide WTP values for every state defined by the classification system. The article also provides a comparison of general population and diabetic population WTP values.
In the regression analysis, the WTP values for the HRQOL and the self-management attributes are of similar magnitude, suggesting that HRQOL and self-management have a similar relative impact in terms of their utility decrement or amount respondents are willing to pay to avoid more severe levels of the attribute. This has important implications for policy, because it suggests that the impact of changes in self-management on the lives of people with diabetes can be as important as any HRQOL improvement resulting from the change.
The largest difference in the rank ordering of the level 4 coefficients between the general population and the patient samples is the ranking of the vitality attribute, which is ranked the most important for the general population, whereas it is ranked the second least important for the patient population. This is interesting given that it is the attribute in the classification system derived from the generic short form 36 health survey, whereas the other items were all derived either from a diabetes-specific measure or from interviews with people with diabetes. Although this suggests that the general population may place a higher relative value on reductions in vitality than the diabetic patient population, the absolute value is similar for both samples. In addition, the mood attribute is ranked the second most important for the diabetic population but the third least important for the general population. Nevertheless, both the hypoglycemic attack and the social limitations attributes are ranked among the least important for both populations.
The WTP per month to avoid state 44444444 is £1118.53 for the general population and £2356.02 for the diabetic patient population. This is significantly larger than the largest cost level included in the survey of £600 per month. An indication of how large these values are relative to annual household income can be calculated. Approximately 23.9% of the general population sample has an annual household income less than £20,800, and for these respondents this would represent at least 64.5% of their income (£13,422.36). Obviously, this is a somewhat unrealistic example because it includes the most severe state experienced by respondents with the lowest income, but indicates that the results should be interpreted with caution, because they may indicate only relative values rather than absolute values. It is possible that the values suffer from “hypothetical bias,” when people state that they are willing to pay higher values than they would actually be willing to pay when using their own money in a laboratory or field experiment [
]. This is an important limitation of this type of study that is purely hypothetical.
One limitation of the study is that the patient sample is relatively small given that the survey design included 12 blocks of 10 DCE questions, meaning approximately 29 to 34 respondents answered each block of 10 questions. This may have impacted on the significance of coefficients for the regression models estimated on the patient samples.
The regression models had some insignificant and some inconsistent results. It is, however, important to note that it is not uncommon for valuation surveys of classification systems to find this; for example, the UK valuation of the SF-6D using standard gamble had both inconsistent and insignificant coefficients [
The design of the survey may have impacted on the results. The DCE tasks ask respondents to consider two health descriptions that contained 18 pieces of information (9 pieces per description). This is a large amount of information to simultaneously consider and could mean that respondents did not fully consider all information or all attributes. Other valuation surveys of classification systems have also included a large number of attributes (e.g., the survey by Norman et al. [
] included 11 attributes), and yet in the design these have fixed some attributes at the same level across both profiles in the DCE, and this is an option recommended for use in future surveys. It is possible that there may be interaction in preferences for some of the attributes, for example, mood and stress. Nevertheless, the study design selected choice sets assuming an additive model, meaning that interaction effects cannot be accurately estimated.
The use of existing general population online panels can be criticized for not being representative of the UK population. Members of online panels may differ from the general population in that they exclude the computer illiterate and those without access to the Internet. In addition, respondents have signed up to be members of a panel with a market research agency and stated that they are willing to answer surveys in return for points that can be exchanged for goods, and therefore their motivation for undertaking the survey may have an impact on their responses, potentially making their responses unrepresentative. Nevertheless, in terms of sociodemographic characteristics measured, they were found to be similar to the UK general population except that there being fewer who were older than 65 years and in contrast a larger proportion of retired respondents, although they may differ in terms of other unmeasured characteristics. The patient sample was different in ways that would be expected for those with diabetes.
The major limitation of this study is concerned with the accuracy of the elicitation of preferences for the cost attribute. The UK health care system is publicly funded, meaning that the general population receives free health care at the point of use with the exception of small charges such as prescription charges. This means that respondents are not used to paying for health care, or paying to improve their health state. This may have affected the results because it may mean that responses are not fully considered or fully informed because of inexperience in paying for health care or health improvement. It is possible that the selection of four levels rather than a higher number for the cost attribute may have also affected the results. In addition, respondents were forced to make a choice, meaning that they could not state indifference or have an “opt out” response indicating that they were not prepared to pay any cost for their health. This may have affected the results, because this does not reflect real life in which people can choose to not pay to improve their health. From a modeling perspective, however, an indifference or opt-out option is not beneficial, although it would indicate these respondents who may have chosen at random. One possibility is to include a zero cost level, and further research is encouraged to determine how this impacts on responses.
The DCE in this study was purposively designed to maximize statistical efficiency and minimize any dominance, for example, when one profile is clearly dominant or dominant with the exception of one attribute such as cost. Therefore, although we can observe within choice sets whether the low or high cost option was selected (where cost differs), we are unable to infer whether this selection was only due to cost, mainly due to cost, or despite cost. A better way to examine this question would be a binary choice survey with specifically selected scenarios to examine dominance of the cost attribute, a small think-aloud survey asking respondents the reasons for their choices directly, and/or research using eye tracking devices to determine where respondents focus their gaze when making their choices. This is an important issue for further research.
The wording of the cost attribute may have impacted on the results. In the profiles the wording was “The monthly cost of treatment to you.” It is possible that respondents may not have correctly interpreted the cost as being applicable every month and may have instead interpreted it as a one-off payment. The frequency of payment as monthly rather than annually may have also had an effect as, for example, altering the wording to annual rather than monthly may not have multiplied the size of the WTP estimates by 12 times.
Conclusions
This article contributes to the methodological literature on the valuation of standardized patient-reported outcome measures using WTP for potential application in CBA. This approach could potentially add to the use of WTP in economic evaluation. The levels of the cost attribute used in the DCE survey were derived empirically, adding strength to the approach. The project also compared general population and diabetic patient values. Overall this approach shows promise, but there are some concerns with some inconsistencies in results, insignificant coefficients, and large WTP estimates observed that extend beyond the cost levels used in the survey. The novel application of DCE with a cost attribute to value a classification system provides an alternative approach to DCE with duration, an approach that has been previously used to value classification systems. Further research applying DCE with a cost attribute to value a classification system and to compare this method with DCE with duration is ongoing.
Acknowledgments
We acknowledge all the participants in the surveys, both patients and from the general population. We also acknowledge those who took part in our patient and public involvement groups and gave their time to the project. We are very grateful for the advice provided by Verity Watson on survey 1 used to determine the cost levels. Any remaining errors are those of the authors alone.
Source of financial support: This project was funded by The Health Foundation (London). The publication of the study results was not contingent on the sponsor’s approval or censorship of the manuscript.
Mulhern B, Labeit A, Rowen D, et al. Developing preference based measures for diabetes: DHP-3D and DHP-5D. Diabet Med [published online ahead of print May 6, 2017]. doi:10.1111/dme.13377.
The Diabetes Health Profile (DHP): a new instrument for assessing the psychosocial profile of insulin requiring patients—development and psychometric evaluation.
Adaptation of the Diabetes Health Profile (DHP-1) for use with patients with type 2 diabetes mellitus: psychometric evaluation and cross-cultural comparison.
Age distribution is here reported as the percentage of all adults aged 18 y and older.
30319-4&id=gr1.jpg){kind=link}
30319-4&id=gr2.jpg){kind=link}
30319-4&id=gr3.jpg){kind=link}