Advertisement

Societal Effects Are a Major Factor for the Uptake of the Coronavirus Disease 2019 (COVID-19) Digital Contact Tracing App in The Netherlands

Open AccessPublished:March 09, 2021DOI:https://doi.org/10.1016/j.jval.2021.01.001

      Highlights

      • Digital contact-tracing (DCT) apps can contribute to significantly reducing coronavirus disease 2019 (COVID-19) transmission rates. The main determinant of the effectiveness of a DCT app is the level of adoption among potential users.
      • The prevention of deaths and financial problems of households had a very strong influence on the uptake of the app. Predicted app uptake rates ranged from 24% to 78% for the worst and best possible app for these societal effects. The uptake levels in our study are much more volatile than the uptake levels predicted in comparable studies that did not include societal effects in their discrete choice experiments.
      • Our study provides decision makers with information concerning the extent to which the uptake of the COVID-19 DCT app can be affected by its configurations and government policies toward such an app.

      Abstract

      Objectives

      Our study investigates the extent to which uptake of a COVID-19 digital contact-tracing (DCT) app among the Dutch population is affected by its configurations, its societal effects, and government policies toward such an app.

      Methods

      We performed a discrete choice experiment among Dutch adults including 7 attributes, that is, who gets a notification, waiting time for testing, possibility for shops to refuse customers who have not installed the app, stopping condition for contact tracing, number of people unjustifiably quarantined, number of deaths prevented, and number of households with financial problems prevented. The data were analyzed by means of panel mixed logit models.

      Results

      The prevention of deaths and financial problems of households had a very strong influence on the uptake of the app. Predicted app uptake rates ranged from 24% to 78% for the worst and best possible app for these societal effects. We found a strong positive relationship between people’s trust in government and people’s propensity to install the DCT app.

      Conclusions

      The uptake levels we find are much more volatile than the uptake levels predicted in comparable studies that did not include societal effects in their discrete choice experiments. Our finding that the societal effects are a major factor in the uptake of the DCT app results in a chicken-or-the-egg causality dilemma. That is, the societal effects of the app are severely influenced by the uptake of the app, but the uptake of the app is severely influenced by its societal effects.

      Keywords

      Introduction

      The coronavirus disease 2019 (COVID-19) pandemic forms an unprecedented public health and economic crisis. In the absence of a vaccine or an effective treatment, societies are seeking approaches to effectively control the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes COVID-19. Contact tracing is a key approach deployed by countries worldwide to stop the spread of the virus. Contact tracing means that individuals who have been in close contact with a person infected with SARS-CoV-2 are notified and advised (and in some countries obliged) to self-quarantine, in an effort to break chains of transmission.
      Manual approaches to contact tracing are, however, labor intensive and time-consuming, and such traditional practices can be rapidly overwhelmed by the magnitude of the pandemic.
      • Ferretti L.
      • Wymant C.
      • Kendall M.
      • et al.
      Quantifying dynamics of SARS-CoV-2 transmission suggests that epidemic control is feasible through instantaneous digital contact tracing.
      Digital contact-tracing (DCT) apps have been developed to assist health departments in notifying individuals of potential exposure to SARS-CoV-2. DCT apps often use Bluetooth technology to (temporarily) record proximity events between 2 phones running the app.
      • Ferretti L.
      • Wymant C.
      • Kendall M.
      • et al.
      Quantifying dynamics of SARS-CoV-2 transmission suggests that epidemic control is feasible through instantaneous digital contact tracing.

      ECDPC (European Centre for Disease Prevention and Control). Contact tracing: public health management of persons, including healthcare workers, having had contact with COVID-19 cases in the European Union – second update, 8 April 2020. Technical report.

      • Sattler F.
      • Ma J.
      • Wagner P.
      • et al.
      Risk estimation of SARS-CoV-2 transmission from Bluetooth low energy measurements.
      If users are diagnosed with COVID-19, they can use the app to declare the diagnosis and recent contacts are instantly, automatically, and anonymously notified of their risk and asked to self-quarantine. Various countries already use DCT apps (eg, Germany and Singapore), but in other countries such an app was not (yet) introduced at the time that we conducted our study (eg, The Netherlands and Sweden).
      The main determinant of the effectiveness of a DCT app is the level of adoption among potential users.
      • Ferretti L.
      • Wymant C.
      • Kendall M.
      • et al.
      Quantifying dynamics of SARS-CoV-2 transmission suggests that epidemic control is feasible through instantaneous digital contact tracing.
      Model simulations reveal that the pandemic can be controlled when approximately 60% of the population uses the DCT app and the app is combined with effective testing practices, other measures such as physical distancing, and a sufficient number of people complying with quarantine rules/recommendations.
      • Ferretti L.
      • Wymant C.
      • Kendall M.
      • et al.
      Quantifying dynamics of SARS-CoV-2 transmission suggests that epidemic control is feasible through instantaneous digital contact tracing.
      ,
      • Albert H.
      How to use Covid-19 tracking apps and still protect your privacy. Forbes. June 19, 2020.
      Nevertheless, the app only has a very small impact on breaking chains of transmission when around 15% of the population takes part.
      • Ferretti L.
      • Wymant C.
      • Kendall M.
      • et al.
      Quantifying dynamics of SARS-CoV-2 transmission suggests that epidemic control is feasible through instantaneous digital contact tracing.
      ,
      • Albert H.
      How to use Covid-19 tracking apps and still protect your privacy. Forbes. June 19, 2020.
      Some studies argue that not only the aggregate adoption rates but also the distribution of uptake in the population is important because the effect of a DCT app will be marginal when only the people who are more cautious (and therefore less likely to be infected with the virus anyway) will use the app.
      • Dignum F.
      Simulaties tonen aan dat corona apps werken niet.
      In sum, there is an urgent need to investigate how the uptake of DCT apps in different segments of the population can be improved.
      People’s preferences for a DCT app have been examined through 3 discrete choice experiments (DCEs).
      • Jonker M.F.
      • de Bekker-Grob E.
      • Veldwijk J.
      • Goossens L.
      • Bour S.
      • Ruten van Mölken M.
      COVID-19 contact-tracing apps: predicted uptake in the Netherlands based on a discrete choice experiment.
      • Frimpong J.
      • Helleringer S.
      Financial incentives for downloading COVID–19 digital contact tracing apps.
      • Wiertz C.
      • Banerjee A.
      • Acar O.A.
      • Ghosh A.
      Predicted adoption rates of contact tracing app configurations–insights from a choice-based conjoint study with a representative sample of the UK population.
      Nevertheless, these studies all adopted an individualistic approach toward investigating preferences of potential app users in the sense that they focused on the positive and negative impacts that potential app users would experience themselves
      • Jonker M.F.
      • de Bekker-Grob E.
      • Veldwijk J.
      • Goossens L.
      • Bour S.
      • Ruten van Mölken M.
      COVID-19 contact-tracing apps: predicted uptake in the Netherlands based on a discrete choice experiment.
      • Frimpong J.
      • Helleringer S.
      Financial incentives for downloading COVID–19 digital contact tracing apps.
      • Wiertz C.
      • Banerjee A.
      • Acar O.A.
      • Ghosh A.
      Predicted adoption rates of contact tracing app configurations–insights from a choice-based conjoint study with a representative sample of the UK population.
      (eg, What are the personal benefits that I gain from installing the app?
      • Jonker M.F.
      • de Bekker-Grob E.
      • Veldwijk J.
      • Goossens L.
      • Bour S.
      • Ruten van Mölken M.
      COVID-19 contact-tracing apps: predicted uptake in the Netherlands based on a discrete choice experiment.
      ). Because a person’s choice to install a DCT app is not only influenced by impacts they experience themselves, but also by effects on public health as well as the greater good,
      • Williams S.
      • Armitage C.J.
      • Tampe T.
      • Dienes K.
      Public attitudes towards COVID-19 contact tracing apps: a UK-based focus group study.
      we also included 3 societal effects in our experiment that might affect uptake, according to the literature: (1) decrease in the number of deaths,
      • Williams S.
      • Armitage C.J.
      • Tampe T.
      • Dienes K.
      Public attitudes towards COVID-19 contact tracing apps: a UK-based focus group study.
      ,
      • Altmann S.
      • Milsom L.
      • Zillessen H.
      • et al.
      Support for app-based contact tracing of Covid-19: cross-country evidence.
      (2) decrease in the number of households facing long-term financial problems,
      • Williams S.
      • Armitage C.J.
      • Tampe T.
      • Dienes K.
      Public attitudes towards COVID-19 contact tracing apps: a UK-based focus group study.
      and (3) the number of people quarantined at home as a result of an incorrect notification by the app.
      • Sattler F.
      • Ma J.
      • Wagner P.
      • et al.
      Risk estimation of SARS-CoV-2 transmission from Bluetooth low energy measurements.
      ,
      • Dignum F.
      Simulaties tonen aan dat corona apps werken niet.
      Therefore, the key objective of our study is to investigate the extent to which uptake of a DCT app among the Dutch population is affected by its configurations, its societal effects, as well as by government policies toward such an app, and whether preferences differ between subgroups in the population. We have addressed these questions through a DCE.

      Methodology

      The core idea behind using DCEs is that individuals’ preferences for a product are driven by preferences for the characteristics (so-called attributes) of a product.
      • Lancaster K.J.
      A new approach to consumer theory.
      The relative importance of attributes can be assessed by presenting respondents a series of choice tasks in which they are asked to choose a preferred alternative (in this case a DCT app) from a set of 2 or more alternatives with varying combinations of attribute levels.
      • Hensher D.A.
      • Rose J.M.
      • Greene W.H.
      Applied Choice Analysis.

       Attributes and Levels

      The selection of the attributes was based on dimensions of the DCT app that played a role in the public debate in The Netherlands, comparable studies on consumer preferences of a COVID-19 DCT app,
      • Jonker M.F.
      • de Bekker-Grob E.
      • Veldwijk J.
      • Goossens L.
      • Bour S.
      • Ruten van Mölken M.
      COVID-19 contact-tracing apps: predicted uptake in the Netherlands based on a discrete choice experiment.
      • Frimpong J.
      • Helleringer S.
      Financial incentives for downloading COVID–19 digital contact tracing apps.
      • Wiertz C.
      • Banerjee A.
      • Acar O.A.
      • Ghosh A.
      Predicted adoption rates of contact tracing app configurations–insights from a choice-based conjoint study with a representative sample of the UK population.
      and insights from the unified theory of acceptance and use of technology (UTAUT)
      • Viswanath V.
      • Morris M.G.
      • Davis G.B.
      • Davis F.D.
      User acceptance of information technology: toward a unified view.
      and the technology acceptance model (TAM).
      • Venkatesh V.
      • Davis F.D.
      A theoretical extension of the technology acceptance model: four longitudinal field studies.
      We adjusted the list of attributes based on feedback from 6 experts (information communication technology specialists, choice modelers, medical ethicists, and epidemiologists). The draft version of the DCE consisted of 7 attributes and was pretested in a convenience sample of 80 respondents. Based on insights from the pretest, we made changes in the descriptions of some of the attributes. The attributes can be distinguished in 3 categories: (1) individual effects: waiting time for testing and possibility for shops to refuse customers who have not installed the app; (2) privacy aspects: who gets a notification and stopping condition for contact tracing; (3) societal effects: number of people unjustifiably quarantined at home, number of deaths prevented, and number of households with financial problems prevented. For the 3 societal attributes, we introduced the qualifier “if a majority of the Dutch population installs the app” to avoid giving the impression that respondents could save thousands of lives by their individual decision to install the app. Table 1 provides an overview of the attributes and their levels. Appendix A (in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.01.001) describes how we selected the attributes and levels.
      Table 1An overview of the attributes and their levels.
      AttributeLevel 1Level 2Level 3Level 4
      1. Who gets a notification in case of contact with an infected person with the advice to stay in quarantine for 14 days?Just youYou and the Municipal Health Service (GGD)
      2. After how many days can you be tested for coronavirus after contact with an infected person?3 days6 days9 days
      3. May shops (and later the hospitality industry, cinemas, and cultural institutions) refuse customers who have not installed the app?YesNo
      4. How many people are unjustifiably quarantined at home by the app each day, if a majority of the Dutch population installs the app?500010 00015 000
      5. Decrease in the number of deaths if a majority of the Dutch population installs the app between June 1, 2020 and January 1, 2021.10004000700010 000
      6. Decrease in the number of households facing long-term financial problems if a majority of the Dutch population installs the app between June 1, 2020 and January 1, 2021.100 000200 000300 000400 000
      7. When does keeping track of contacts via the app stop?The government reviews this over time.Criteria are set in advance.This stops automatically on a predetermined date.
      Once we defined the attributes and the initial set of attribute levels, we constructed a Bayesian D-efficient design for our DCE.
      • Kessels R.
      • Jones B.
      • Goos P.
      • Vandebroek M.
      The usefulness of Bayesian optimal designs for discrete choice experiments.
      We incorporated prior knowledge in the design that acknowledges that, for the 3 societal effects included, a large decrease in the number of deaths and the number of households facing long-term financial problems is generally preferred over a smaller one, whereas a lower number of quarantined people is preferred over a higher number. Also, the preferred waiting time for testing is expected to be as short as possible. Furthermore, we expressed uncertainty around our beliefs in a multivariate prior parameter distribution.
      After the pretest, we enhanced the realism of the design by excluding unrealistic combinations of attribute levels. In accordance with these constraints, we further updated our Bayesian D-efficient design. The final design consisted of 40 choice tasks, which were grouped into 5 blocks of 8 choice tasks. Appendix B (in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.01.001) shows the design together with the constraints we imposed as well as a detailed specification of the design efficiency.
      Table 2 provides an example of a choice task. The 2 profiles in each choice set are partial profiles because they differed only in 4 of the 7 relevant attributes to reduce the cognitive burden of the choice sets and improve attribute attendance.
      • Kessels R.
      • Jones B.
      • Goos P.
      Bayesian optimal designs for discrete choice experiments with partial profiles.
      • Kessels R.
      • Jones B.
      • Goos P.
      An improved two-stage variance balance approach for constructing partial profile designs for discrete choice experiments.
      • Jonker M.F.
      • Donkers B.
      • de Bekker-Grob E.
      • Stolk E.A.
      • et al.
      Attribute level overlap (and color coding) can reduce task complexity, improve choice consistency, and decrease the dropout rate in discrete choice experiments.
      These 4 attributes were highlighted in yellow. In each choice task, respondents were asked which of the 2 apps they would prefer to install on their smartphone (ie, forced choice). Subsequently, they were asked if they would like to opt out if given the choice (dual response design).
      • Brazell J.D.
      • Diener C.G.
      • Karniouchina E.
      • Moore W.L.
      • Severin V.
      • Uldry P.-F.l.
      The no-choice option and dual response choice designs.
      • Diener C.
      • Orme B.K.
      • Yardley D.
      Dual Response “None” Approaches: Theory and Practice.
      • Veldwijk J.
      • Lambooij M.S.
      • de Bekker-Grob E.W.
      • de Smit H.A.
      • de Wit G.A.
      The effect of including an opt-out option in discrete choice experiments.
      Respondents were told that if they did not have a smartphone, they could expect to be provided with a small device (token) free of charge by the government with the same features as the app, to make sure that the DCE was relevant for everyone, irrespective of possessing a smartphone.
      Table 2Example of a choice screen as presented to respondents.
      App AApp B
      Who gets a notification if you have had contact with an infected person? (After such a contact, the advice is to stay in home quarantine for 14 days.)You and the Municipal Health Service (GGD)You and the Municipal Health Service (GGD)
      After how many days can you be tested for coronavirus after contact with an infected person?9 days3 days
      May shops (and later the hospitality industry, cinemas, and cultural institutions) refuse customers who have not installed the app?NoNo
      How many people are unjustifiably quarantined at home by the app each day, if a majority of Dutch people install the app?15 0005000
      Decrease in the number of deaths if a majority of the Dutch people install the app10 0001000
      Decrease in the number of households facing long-term financial problems if a majority of Dutch people install the app200 000200 000
      When does keeping track of contacts via the app stop?The government reviews this over time.Criteria are set in advance.
      GGD indicates Municipal Health Service.

       Follow-up Questions

      We asked participants a number of follow-up questions about how easy they found the choice task, how convinced they were of their choices, and how likely they thought it was that a majority of the Dutch population would install the app. We then asked a series of questions about their trust in the government’s ability to take the right measures concerning the app and to carefully handle the collected data. We also asked respondents how well they had managed to keep a distance of 1.5 m from others, how likely they deemed it for themselves to become infected with COVID-19 or for them to infect others, and how severe it would be for them to be infected or to infect others. We further collected information about sociodemographic characteristics (age, sex, education, rurality), and about smartphone ownership and mobile skills.
      • Van Deursen A.J.A.M.
      • Helsper E.J.
      • Eynon R.
      Measuring digital skills. From Digital Skills to Tangible Outcomes project report.

       Data Collection and Sample Characteristics

      We conducted a survey among Dutch adult inhabitants (≥18 years) between May 21 and May 28, 2020. We recruited respondents from an internet panel of Kantar Public in such a way that they were representative of the Dutch adult population regarding age and sex. Participation was incentivized through credit rewards transferable into coupons. The Ethical Review Committee Inner City faculties of Maastricht University approved our study protocol (ERCIC_191_07_05_2020).

       Analysis

      To derive the marginal utility that respondents obtain from the attributes of the DCT app, we estimated a panel mixed logit (PML) model with a linear-additive utility function using the Hierarchical Bayes technique in the JMP Pro 15 Choice platform (based on 10 000 iterations, of which the last 5000 were used for the actual estimation; SAS Institute Inc, Cary, NC, USA). Appendix B provides more detail (see Appendix B in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.01.001). A PML model is a logit model where it is assumed that the preference or utility parameters differ randomly across persons. Hence, the model takes into account the heterogeneity between respondents in their preferences for (the attributes of) the DCT app.
      • Train K.
      Discrete Choice Methods with Simulation.
      First, we analyzed the forced choice data and used Ward’s hierarchical cluster analysis of the individual preference estimates to establish distances between subsets of respondents in cluster formation. Second, we analyzed the opt-out data from the follow-up question of the choice situations where respondents were asked whether or not they would install the app they had primarily selected in the forced choice task. Using the estimated opt-out PML model, we were able to predict uptake percentages for various app profiles given a choice set with only the app and the opt-out option. Confidence intervals for these uptake rates were computed by means of the method of endpoint transformations.
      • Xu J.
      • Long J.S.
      Confidence intervals for predicted outcomes in regression models for categorical outcomes.
      We performed all analyses using the JMP Pro 15 software.

      Results

      A total of 1220 members of the panel were invited to fill out the survey. Of these, 1100 respondents started the survey, and 990 respondents fully completed it, resulting in 110 dropouts (10%). Furthermore, we excluded 64 respondents from the final dataset because they filled out the survey too quickly, that is, in less than a third of the median time to complete the survey for the entire sample, or provided the same answer to each choice question. Hence, we based our analyses on survey results from 926 respondents (Appendix C reports the sociodemographic characteristics; see Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.01.001).

       Outcomes of the Forced Choices in the DCE

      The analysis of the forced choices between 2 profiles of the DCT app revealed that respondents can be divided into 2 distinguishing clusters in their preferences for the attributes of the DCT app. Only main effects turned out to be significant in the models for the clusters. Table 3 shows the marginal utility estimates that the 326 respondents belonging to cluster 1 derive from the levels of the attributes of the DCT app. Table 4 provides the same information for the 600 respondents belonging to cluster 2. To enhance interpretation of the tables, the utility estimates are visualized in Figure 1A,B. Table 3 shows that cluster 1 is more likely to install an app that effectively reduces the negative societal impacts of COVID-19 and an app that sends a notification to the Municipal Health Service in case of contact with an infected person. Table 4 shows that the 600 respondents belonging to cluster 2 put a strong emphasis on privacy and freedom. They are less likely to install an app which sends a notification to the Municipal Health Service. Moreover, for cluster 2 an app becomes less attractive when shops and other businesses have the possibility to refuse customers who have not installed the DCT app. The choices between possible DCT apps of respondents belonging to cluster 2 are not affected by the extent to which the DCT app effectively reduces the number of COVID-19–related deaths or households with financial problems.
      Table 3PML model estimates for cluster 1 (326 respondents): mean and standard deviation (std dev) and significance of the attribute effects obtained from likelihood ratio (LR) tests with specified number of degrees of freedom (DF).
      Model termMean estimate (std dev; subject std dev)95% credible intervalLR chi-squareDFP value
      Notification
       Just you−3.411 (0.453; -0.903)[-4.599; -2.764]132.9221<.0001
       You and the GGD3.411 (0.453; -0.903)[2.764; 4.599]
      Testing
       After 3 days2.957 (0.450; -1.030)[1.956; 3.695]72.4862<.0001
       After 6 days−0.752 (0.387; -0.681)[-1.530; -0.065]
       After 9 days−2.204 (0.432; -1.300)[-2.917; -1.232]
      Refusal by shops
       Yes0.757 (0.206; -1.015)[0.343; 1.176]5.4991.0190
       No−0.757 (0.206; -1.015)[-1.176; -0.343]
      People unjustifiably self-quarantined per day
       50001.773 (0.455; -2.393)[0.830; 2.587]38.1942<.0001
       10 0001.165 (0.404; -1.057)[0.330; 1.930]
       15 000−2.938 (0.489; -2.710)[-3.818; -1.931]
      Decrease in deaths
       1000−7.061 (1.205; -1.674)[-9.282; -4.504]159.2663<.0001
       4000−2.182 (0.514; -3.323)[-3.236; -1.202]
       70000.270 (0.465; -1.871)[-0.649; 1.216]
       10 0008.973 (1.281; -4.880)[6.539; 11.394]
      Decrease in households with financial problems
       100 000−6.989 (0.990; -2.117)[-8.739; -5.023]148.4283<.0001
       200 000−1.101 (0.407; -1.685)[-1.934; -0.356]
       300 0001.684 (0.480; -1.036)[0.814; 2.653]
       400 0006.406 (0.935; -3.553)[4.564; 8.089]
      Stopping condition
       Review over time−0.166 (0.296; -1.697)[-0.775; 0.395]1.8922.3882
       Criteria set in advance0.649 (0.311; -1.215)[0.050; 1.234]
       Predetermined date−0.483 (0.302; -2.290)[-1.096; 0.121]
      GGD indicates Municipal Health Service; PML, panel mixed logit; std dev, standard deviation.
      Table 4PML model estimates for cluster 2 (600 respondents): mean and standard deviation (std dev) and significance of the attribute effects obtained from likelihood ratio (LR) tests with specified number of degrees of freedom (DF).
      Model termMean estimate (std dev; subject std dev)95% credible intervalLR chi-squareDFP value
      Notification
       Just you0.517 (0.073; 1.041)[0.380; 0.662]41.5601<.0001
       You and the GGD−0.517 (0.073; 1.041)[-0.662; -0.380]
      Testing
       After 3 days0.847 (0.106; 0.893)[0.658; 1.063]80.2752<.0001
       After 6 days0.081 (0.079; 0.395)[-0.067; 0.238]
       After 9 days−0.927 (0.112; 1.018)[-1.156; -0.726]
      Refusal by shops
       Yes−1.242 (0.097; 1.224)[-1.437; -1.073]339.9161<.0001
       No1.242 (0.097; 1.224)[1.073; 1.437]
      People unjustifiably self-quarantined per day
       50000.380 (0.103; 0.429)[0.174; 0.578]27.2652<.0001
       10 0000.050 (0.095; 0.152)[-0.166; 0.220]
       15 000−0.430 (0.115; 0.500)[-0.652; -0.201]
      Decrease in deaths
       10000.274 (0.171; 0.438)[-0.074; 0.601]3.8123.2825
       4000−0.353 (0.138; 0.418)[-0.589; -0.043]
       70000.093 (0.096; 0.226)[-0.095; 0.279]
       10 000−0.014 (0.101; 0.848)[-0.207; 0.191]
      Decrease in households with financial problems
       100 000−0.204 (0.141; 0.365)[-0.483; 0.062]5.7863.1225
       200 0000.335 (0.120; 0.286)[0.092; 0.558]
       300 000−0.297 (0.116; 0.499)[-0.524; -0.063]
       400 0000.166 (0.134; 0.794)[-0.100; 0.438]
      Stopping condition
       Review over time−0.469 (0.111; 0.629)[-0.705; -0.268]16.1502.0003
       Criteria set in advance0.134 (0.088; 0.266)[-0.032; 0.308]
       Predetermined date0.335 (0.104; 0.754)[0.137; 0.541]
      GGD indicates Municipal Health Service; PML, panel mixed logit; std dev, standard deviation.
      Figure thumbnail gr1
      Figure 1Mean utility estimates from the PML models for (A-B) the forced choice data of cluster 1 (326 respondents) in and cluster 2 (600 respondents) in , and (C) the opt-out data of all 926 respondents in .
      PML indicates panel mixed logit.

       Findings Regarding Uptake of the DCT App

      After respondents made a forced choice between 2 profiles of the DCT app, they were asked whether or not they would install the app they selected. An analysis of these choices revealed that 33.7% of the respondents chose to install the app they preferred in all 8 choice situations, which implies that they preferred a DCT app with the least preferred specifications presented to them in the DCE over not installing a DCT app at all. On the other hand, 29.7% of the respondents stated that they would install neither of the 2 apps they could choose from in all 8 choice tasks. Finally, 36.7% of the respondents indicated to install the app in some choice tasks, while in other choice tasks they did not install the app. This undecided group of respondents bases its decision to (not) install the app on the configurations of the app, its societal effects, as well as related government policies. Appendix C (in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.01.001) provides more detail regarding the (sociodemographic) characteristics of these 3 groups of respondents.
      To determine the most important characteristics and opinions for discriminating among the 3 groups, we conducted a logistic regression of the group variable on all descriptive variables jointly (see Appendix C in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.01.001). This logistic regression reveals that there is no significant relation among sex, education, and geographical population density and respondents’ likeliness to install the DCT app. We find a small effect for age in the sense that the number of young people in the undecided group is relatively large, while older people are overrepresented in the 2 extreme groups (P = .08). People who revealed that they always succeed in keeping distance from fellow citizens who are not part of their household are not more prone to install the DCT app than those who do not succeed in keeping distance. On the other hand, respondents who believe that it is likely that a majority of Dutch citizens will install a DCT app are expected to install the app more often than respondents who believe that this is unlikely (P < .0001). Moreover, respondents who install the app in all choice tasks have a relatively high trust that the government will handle the collected data in the app carefully (P = .0004) and that the government will make the right decision concerning the DCT app (P = .0494). These respondents also believe that it is more likely that they will become infected with COVID-19 than respondents who are not prone to install the app (P = .0031).
      To determine the percentage of the Dutch population that will install the app, we estimated an opt-out PML model for the entire sample for which only main effects proved to be significant (see Table 5 and Fig. 1C).
      Table 5Opt-out PML model estimates for all 926 respondents: mean and standard deviation (std dev) and significance of the attribute effects obtained from likelihood ratio (LR) tests with specified number of degrees of freedom (DF).
      Model termMean estimate (std dev; subject std dev)95% credible intervalLR chi-squareDFP value
      Opt-out0.851 (0.132; 6.663)[0.604; 1.132]485.6281<.0001
      Notification
       Just you−0.304 (0.045; 0.652)[-0.393; -0.220]48.3711<.0001
       You and the GGD0.304 (0.045; 0.652)[0.220;  0.393]
      Testing
       After 3 days0.445 (0.053; 0.380)[0.342; 0.547]48.4092<.0001
       After 6 days0.084 (0.053; 0.167)[-0.026; 0.184]
       After 9 days−0.529 (0.054; 0.449)[-0.634; -0.424]
      Refusal by shops
       Yes−0.208 (0.038; 0.590)[-0.285; -0.136]10.1951.0014
       No0.208 (0.038; 0.590)[0.136; 0.285]
      People unjustifiably self-quarantined per day
       50000.125 (0.052; 0.264)[0.027; 0.235]5.3392.0693
       10 0000.049 (0.049; 0.175)[-0.050; 0.144]
       15 000−0.174 (0.060; 0.366)[-0.300; -0.061]
      Decrease in deaths
       1000−0.631 (0.098; 0.436)[-0.831; -0.442]41.4983<.0001
       4000−0.106 (0.070; 0.219)[-0.247; 0.033]
       70000.244 (0.071; 0.146)[0.115; 0.381]
       10 0000.492 (0.076; 0.649)[0.350; 0.647]
      Decrease in households with financial problems
       100 000−0.620 (0.088; 0.314)[-0.791; -0.450]30.6293<.0001
       200 0000.084 (0.055; 0.133)[-0.017; 0.189]
       300 0000.106 (0.065; 0.143)[-0.034; 0.230]
       400 0000.430 (0.071; 0.451)[0.353; 0.635]
      Stopping condition
       Review over time−0.166 (0.052; 0.197)[-0.272; -0.060]5.2922.0709
       Criteria set in advance0.115 (0.054; 0.142)[0.011; 0.223]
       Predetermined date0.051 (0.054; 0.283)[0.322; 0.534]
      GGD indicates Municipal Health Service; PML, panel mixed logit; std dev, standard deviation.
      The information regarding people’s preferences for (the attributes of) the DCT app can be used for predicting uptake levels for apps within the population. The scenarios presented in Table 6 contain the apps that best match the preferences of respondents belonging to cluster 1 and cluster 2 that maximize the probability of being chosen by each cluster based on the estimated PML models for the forced choice data of cluster 1 and cluster 2. We computed the uptake percentages for these best apps using the estimated opt-out PML model on a choice set with only the app and the opt-out option. The app with the largest uptake rate in the population corresponds to the app that best matches the preferences of the full sample. Table 6 shows that the uptake of the app with the most preferred combination of attribute levels is 78%. The maximum uptake of 78% can be conceived as counterintuitive because one would expect that the uptake of the best possible app could not be higher than 70.3%, as 29.7% of the respondents stated that they would install neither of the 2 apps they could choose from in all 8 choice tasks. The uptake of the best possible app being higher than 70.3% can be explained by the fact that an app with this optimal combination of attribute levels was not included in the DCE. That is, all the apps that respondents were asked to evaluate in the DCE had less attractive combinations of attribute levels. To demonstrate the major impact of the 3 societal effects, we varied them gradually in 3 specified sensitivity tests and computed once more the uptake percentages for the apps using the estimated opt-out PML model on a choice set with only the app and the opt-out option. These sensitivity tests reveal that lower effectiveness of the DCT app for these societal effects would have a substantial negative impact on its uptake. Table 6 illustrates that the uptake of the DCT app with the least attractive combination of societal impacts in our DCE is 24%, which suggests that societal effects are a major factor in the uptake of the DCT app in The Netherlands.
      Table 6Uptake levels for apps that best match the preferences of respondents and sensitivity tests.
      App that best matches the preferences of the respondents of this studyApp that best matches the preferences of respondents belonging to cluster 1App that best matches the preferences of respondents belonging to cluster 2Sensitivity test for substantial societal effectsSensitivity test for moderate societal effectsSensitivity test for smallest societal effects
      You and the Municipal Health Service (GGD) are notified in case of contact with an infected person with the advice to self-quarantine for 14 days.You and the Municipal Health Service (GGD) are notified in case of contact with an infected person with the advice to self-quarantine for 14 days.Only you are notified in case of contact with an infected person with the advice to self-quarantine for 14 days.You and the Municipal Health Service (GGD) are notified in case of contact with an infected person with the advice to self-quarantine for 14 days.You and the Municipal Health Service (GGD) are notified in case of contact with an infected person with the advice to self-quarantine for 14 days.You and the Municipal Health Service (GGD) are notified in case of contact with an infected person with the advice to self-quarantine for 14 days.
      You can be tested after 3 days after reporting contact with an infected person.You can be tested after 3 days after reporting contact with an infected person.You can be tested after 3 days after reporting contact with an infected person.You can be tested after 3 days after reporting contact with an infected person.You can be tested after 3 days after reporting contact with an infected person.You can be tested after 3 days after reporting contact with an infected person.
      Shops (and later the hospitality industry, cinemas, and cultural institutions) may not refuse customers who have not installed the app.Shops (and later the hospitality industry, cinemas, and cultural institutions) may refuse customers who have not installed the app.Shops (and later the hospitality industry, cinemas, and cultural institutions) may not refuse customers who have not installed the app.Shops (and later the hospitality industry, cinemas, and cultural institutions) may not refuse customers who have not installed the app.Shops (and later the hospitality industry, cinemas, and cultural institutions) may not refuse customers who have not installed the app.Shops (and later the hospitality industry, cinemas, and cultural institutions) may not refuse customers who have not installed the app.
      5000 people are unjustifiably self-quarantined by the app each day, if a majority of Dutch people install the app5000 people are unjustifiably self-quarantined by the app each day, if a majority of Dutch people install the app5000 people are unjustifiably self-quarantined by the app each day, if a majority of Dutch people install the app15 000 people are unjustifiably self-quarantined by the app each day, if a majority of Dutch people install the app10 000 people are unjustifiably self-quarantined by the app each day, if a majority of Dutch people install the app5000 people are unjustifiably self-quarantined by the app each day, if a majority of Dutch people install the app
      10 000 fewer deaths if a majority of Dutch people install the app10 000 fewer deaths if a majority of Dutch people install the app10 000 fewer deaths if a majority of Dutch people install the app (not significant)7000 fewer deaths if a majority of Dutch people install the app4000 fewer deaths if a majority of Dutch people install the app1000 fewer deaths if a majority of Dutch people install the app
      400 000 fewer households facing long-term financial problems, if a majority of Dutch people install the app400 000 fewer households facing long-term financial problems, if a majority of Dutch people install the app400 000 fewer households facing long-term financial problems, if a majority of Dutch people install the app (not significant)300 000 fewer households facing long-term financial problems, if a majority of Dutch people install the app200 000 fewer households facing long-term financial problems, if a majority of Dutch people install the app100 000 fewer households facing long-term financial problems, if a majority of Dutch people install the app
      Criteria are set in advance to stop tracking contacts via the appCriteria are set in advance to stop tracking contacts via the app (not significant)Tracking contacts via the app stops automatically on a predetermined dateCriteria are set in advance to stop tracking contacts via the appCriteria are set in advance to stop tracking contacts via the appCriteria are set in advance to stop tracking contacts via the app
      78% [72%; 83%] adoption by full sample70% [63%; 76%] adoption by full sample65% [56%; 72%] adoption by full sample60% [52%; 67%] adoption by full sample56% [50%; 62%] adoption by full sample24% [22%; 36%] adoption by full sample
      GGD indicates Municipal Health Service.

      Conclusions and Discussion

       Main Findings

      Our study shows that effectiveness of the DCT app in terms of its societal effects has a very strong influence on its uptake. We found that 78% of the respondents would install the best possible app with very high societal effects, whereas only 24% would install the app when we would assume low societal effects. We found a strong positive relationship between people’s trust in government and people’s propensity to install the DCT app.
      We established that respondents can be divided into 2 clusters in their preferences for a DCT app. The first cluster was more likely to install an app that effectively reduces the negative societal effects of COVID-19. The second cluster puts a relatively strong emphasis on privacy and freedom when choosing between apps. The choices among possible DCT apps of respondents belonging to this cluster are not affected by the extent to which the DCT app effectively reduces the number of COVID-19 deaths or households with financial problems. Hence, although the societal effects of the DCT app are a major factor in the uptake of the app in The Netherlands, these societal effects do not seem to be decisive for the choices among alternative profiles of the app for this cluster of respondents.

       Comparison to Other Studies

      The uptake levels in our study are much more volatile than the uptake levels predicted in comparable DCEs.
      • Jonker M.F.
      • de Bekker-Grob E.
      • Veldwijk J.
      • Goossens L.
      • Bour S.
      • Ruten van Mölken M.
      COVID-19 contact-tracing apps: predicted uptake in the Netherlands based on a discrete choice experiment.
      • Frimpong J.
      • Helleringer S.
      Financial incentives for downloading COVID–19 digital contact tracing apps.
      • Wiertz C.
      • Banerjee A.
      • Acar O.A.
      • Ghosh A.
      Predicted adoption rates of contact tracing app configurations–insights from a choice-based conjoint study with a representative sample of the UK population.
      For instance, Jonker et al.
      • Jonker M.F.
      • de Bekker-Grob E.
      • Veldwijk J.
      • Goossens L.
      • Bour S.
      • Ruten van Mölken M.
      COVID-19 contact-tracing apps: predicted uptake in the Netherlands based on a discrete choice experiment.
      predicted uptake levels of 59.3% to 65.7% for the worst and best possible DCT app in their study. We think that the relatively volatile uptake levels predicted in our study may result from the integration of societal effects in our DCE, whereas other DCEs did not include such effects. Our study shows that respondents who believe that it is likely that a majority of Dutch citizens will install a DCT app are expected to install the app more often than respondents who believe that this is unlikely. Hence, respondents who have a high propensity to install the app do not seem to have a strong intention to free-ride on the willingness of fellow citizens who consider installing the app. The absence of free-rider behavior was also found in a recent study regarding people’s decision making on vaccination.
      • Hoogink J.
      • Verelst F.
      • Kessels R.
      • et al.
      Preferential differences in vaccination decision-making for oneself or one’s child in The Netherlands: a discrete choice experiment.
      We investigated respondents’ propensity to install the DCT app for different segments of the population, because Dignum
      • Dignum F.
      Simulaties tonen aan dat corona apps werken niet.
      emphasized that the effect of a DCT app will be marginal when only the people who are more cautious (and therefore less likely to be infected with the virus anyway) will use the app. Nevertheless, we did not find a difference between people’s propensity to keep distance from fellow citizens who do not live in the same household and their likeliness to install the DCT app, and we even found that respondents who believe that it is likely that they will be infected with COVID-19 have a higher propensity to install the app.

       Limitations

      A first limitation is that our study was conducted in a period with a low number of infections in The Netherlands and much uncertainty regarding the societal effects of the DCT app, and it is questionable to which extent the predicted uptake levels are generalizable to contexts with substantially higher infection rates and a lower level of uncertainty regarding the societal effects of the app. We are confident that our study shows that societal effects are a major factor in the uptake of a COVID-19 DCT app in The Netherlands, and we think that our study provides accurate estimations regarding people’s preferences toward a COVID-19 DCT app in a context with low infection rates and high uncertainty regarding societal effects of a DCT app. Nevertheless, we believe that we cannot conclude that the societal effects are the decisive factor for the uptake of the COVID-19 DCT app. We can only draw such a conclusion in a reliable way through repeating the experiment in contexts with higher infection rates and/or contexts with less uncertainty regarding the societal effects of a DCT app (which allows using smaller distances between the levels of the societal attributes). A second limitation of our study is that the results are not directly generalizable to other countries. We expect that the uptake levels of the DCT app will be higher in The Netherlands than in many other countries, because we observe that trust in government is an important driver of people’s propensity to install the DCT app and trust in government is relatively high in The Netherlands.
      Organisation for Economic Co-operation and Development
      Government at a glance 2019.
      Moreover, research on people’s preferences for coronavirus measures in 7 European countries found that trust in information from the national government ranked highest in The Netherlands (more than 70% of respondents trusted this information “much” or “very much”) whereas it was lowest in France (27% of respondents had a high level of trust).
      • Sabat I.
      • Neuman-Böhme S.
      • Varghese N.E.
      • et al.
      United but divided: policy responses and people’s perceptions in the EU during the COVID-19 outbreak.
      When this study is repeated in other countries, we also recommend including other privacy aspects than the ones we included in our study, because this would produce insights into the extent to which people’s preferences are affected by these privacy aspects. In the context of the public debate of The Netherlands, it was not opportune to include other privacy aspects—such as security of the app and communication of place and time where the infection has taken place—as attributes in the DCE, because the Minister of Health already decided that the DCT app should respect these privacy issues. Finally, predicted uptake levels in our study are based on stated preferences that might differ from people’s real-world decisions to (not) install a DCT app. For instance, individuals are incentivized to overstate their purchase proclivities for a new private good in stated preference studies, because this will encourage the production of the good and the individual can always decide later whether or not to purchase the good in question without experiencing any negative consequences when defecting.
      • Carson R.T.
      • Groves T.
      Incentive and informational properties of preference questions.
      Hence, the predicted uptake of the DCT app in our study might be an overestimation of real-world uptake rates. On the other hand, recent studies in health economics show that the external validity of DCEs is high, as 90% of individuals’ real-world choices to opt for influenza vaccination and colorectal cancer screening were correctly predicted at the individual level.
      • de Bekker-Grob E.W.
      • Swait J.D.
      • Kassahun H.T.
      • et al.
      Are healthcare choices predictable? The impact of discrete choice experiment designs and models.
      ,
      • de Bekker-Grob E.W.
      • Donkers B.
      • Bliemer M.C.J.
      • Veldwijk J.
      • Swait J.D.
      Can healthcare choice be predicted using stated preference data?.

       Policy Implications

      Our finding that the societal effects play an important role in the uptake of the DCT app results in a chicken-or-the-egg causality dilemma. That is, the societal effects of the DCT app are severely influenced by the uptake of the app, but the uptake of the DCT app is severely influenced by its societal effects. The causality we observe in our study lines up with evidence that vaccination uptake substantially increases when the severity of a pandemic increases.
      • Determann D.
      • Korfage I.J.
      • Lambooij M.S.
      • et al.
      Acceptance of vaccinations in pandemic outbreaks: a discrete choice experiment.
      Finally, our finding that young people are overrepresented in the group of respondents that is undecided about installation of the DCT app might have policy implications, because this suggests that the Dutch government can particularly improve uptake of the DCT app by tailoring policies and specifications of the app toward the preferences of this subgroup.
      Author Contributions: Concept and design: Mouter, Kessels, de Wit, Rotteveel, Lambooij, Collewet
      Acquisition of data: Mouter, Kessels, Collewet
      Analysis and interpretation of data: Mouter, Kessels, de Wit, Rotteveel, Lambooij, Collewet
      Drafting of the manuscript: Mouter, Kessels, de Wit, Rotteveel, Lambooij, Collewet
      Critical revision of the manuscript for important intellectual content: Mouter, Kessels, de Wit, Lambooij, Collewet
      Statistical analysis: Kessels
      Obtaining funding: Kessels
      Conflict of Interest Disclosures: The authors reported no conflicts of interest.
      Funding/Support: Dr Kessels acknowledges her Elinor Ostrom research grant from Maastricht University which was used to collect the data. Also, she thanks the JMP Division of SAS Institute for further financial support.
      Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

      Supplemental Material

      References

        • Ferretti L.
        • Wymant C.
        • Kendall M.
        • et al.
        Quantifying dynamics of SARS-CoV-2 transmission suggests that epidemic control is feasible through instantaneous digital contact tracing.
        Science. 2020; 368
      1. ECDPC (European Centre for Disease Prevention and Control). Contact tracing: public health management of persons, including healthcare workers, having had contact with COVID-19 cases in the European Union – second update, 8 April 2020. Technical report.

        • Sattler F.
        • Ma J.
        • Wagner P.
        • et al.
        Risk estimation of SARS-CoV-2 transmission from Bluetooth low energy measurements.
        NPJ Digit Med. 2020; 3: 129
        • Albert H.
        How to use Covid-19 tracking apps and still protect your privacy. Forbes. June 19, 2020.
        • Dignum F.
        Simulaties tonen aan dat corona apps werken niet.
        • Jonker M.F.
        • de Bekker-Grob E.
        • Veldwijk J.
        • Goossens L.
        • Bour S.
        • Ruten van Mölken M.
        COVID-19 contact-tracing apps: predicted uptake in the Netherlands based on a discrete choice experiment.
        J Med Internet Res. 2020; 8e20741
        • Frimpong J.
        • Helleringer S.
        Financial incentives for downloading COVID–19 digital contact tracing apps.
        • Wiertz C.
        • Banerjee A.
        • Acar O.A.
        • Ghosh A.
        Predicted adoption rates of contact tracing app configurations–insights from a choice-based conjoint study with a representative sample of the UK population.
        • Williams S.
        • Armitage C.J.
        • Tampe T.
        • Dienes K.
        Public attitudes towards COVID-19 contact tracing apps: a UK-based focus group study.
        • Altmann S.
        • Milsom L.
        • Zillessen H.
        • et al.
        Support for app-based contact tracing of Covid-19: cross-country evidence.
        • Lancaster K.J.
        A new approach to consumer theory.
        J Polit Econ. 1966; 74: 132-157
        • Hensher D.A.
        • Rose J.M.
        • Greene W.H.
        Applied Choice Analysis.
        2nd ed. Cambridge University Press, Cambridge, UK2015
        • Viswanath V.
        • Morris M.G.
        • Davis G.B.
        • Davis F.D.
        User acceptance of information technology: toward a unified view.
        MIS Quarterly. 2003; 27: 425-478
        • Venkatesh V.
        • Davis F.D.
        A theoretical extension of the technology acceptance model: four longitudinal field studies.
        Manage Sci. 2000; 46: 186-204
        • Kessels R.
        • Jones B.
        • Goos P.
        • Vandebroek M.
        The usefulness of Bayesian optimal designs for discrete choice experiments.
        Appl Stoch Models Bus Ind. 2011; 27: 173-188
        • Kessels R.
        • Jones B.
        • Goos P.
        Bayesian optimal designs for discrete choice experiments with partial profiles.
        J Choice Model. 2011; 4: 52-74
        • Kessels R.
        • Jones B.
        • Goos P.
        An improved two-stage variance balance approach for constructing partial profile designs for discrete choice experiments.
        Appl Stoch Models Bus Ind. 2015; 31: 626-648
        • Jonker M.F.
        • Donkers B.
        • de Bekker-Grob E.
        • Stolk E.A.
        • et al.
        Attribute level overlap (and color coding) can reduce task complexity, improve choice consistency, and decrease the dropout rate in discrete choice experiments.
        Health Econ. 2019; 28: 350-363
        • Brazell J.D.
        • Diener C.G.
        • Karniouchina E.
        • Moore W.L.
        • Severin V.
        • Uldry P.-F.l.
        The no-choice option and dual response choice designs.
        Marketing Lett. 2006; 17: 255-268
        • Diener C.
        • Orme B.K.
        • Yardley D.
        Dual Response “None” Approaches: Theory and Practice.
        Sawtooth Software, Sequim, WA2006
        • Veldwijk J.
        • Lambooij M.S.
        • de Bekker-Grob E.W.
        • de Smit H.A.
        • de Wit G.A.
        The effect of including an opt-out option in discrete choice experiments.
        PLoS One. 2013; 9e111805
        • Van Deursen A.J.A.M.
        • Helsper E.J.
        • Eynon R.
        Measuring digital skills. From Digital Skills to Tangible Outcomes project report.
        • Train K.
        Discrete Choice Methods with Simulation.
        2nd ed. Cambridge University Press, Cambridge, UK2009
        • Xu J.
        • Long J.S.
        Confidence intervals for predicted outcomes in regression models for categorical outcomes.
        Stata J. 2005; 5: 537-559
        • Hoogink J.
        • Verelst F.
        • Kessels R.
        • et al.
        Preferential differences in vaccination decision-making for oneself or one’s child in The Netherlands: a discrete choice experiment.
        BMC Public Health. 2020; 20: 828
        • Organisation for Economic Co-operation and Development
        Government at a glance 2019.
        • Sabat I.
        • Neuman-Böhme S.
        • Varghese N.E.
        • et al.
        United but divided: policy responses and people’s perceptions in the EU during the COVID-19 outbreak.
        Health Policy. 2020; 124: 909-918
        • Carson R.T.
        • Groves T.
        Incentive and informational properties of preference questions.
        Environ Resour Econ. 2007; 37: 181-210
        • de Bekker-Grob E.W.
        • Swait J.D.
        • Kassahun H.T.
        • et al.
        Are healthcare choices predictable? The impact of discrete choice experiment designs and models.
        Value Health. 2019; 22: 1050-1062
        • de Bekker-Grob E.W.
        • Donkers B.
        • Bliemer M.C.J.
        • Veldwijk J.
        • Swait J.D.
        Can healthcare choice be predicted using stated preference data?.
        Soc Sci Med. 2020; 246112736
        • Determann D.
        • Korfage I.J.
        • Lambooij M.S.
        • et al.
        Acceptance of vaccinations in pandemic outbreaks: a discrete choice experiment.
        PLoS One. 2014; 9e102505