Advertisement

Measuring Productivity Costs in Patients With Musculoskeletal Disorders: Measurement Properties of the Institute for Medical Technology Assessment Productivity Cost Questionnaire

Open ArchivePublished:September 12, 2019DOI:https://doi.org/10.1016/j.jval.2019.07.011

      Highlights

      • Although productivity costs often represent a large proportion of the total cost related to health and healthcare interventions, they are frequently ignored in health economic evaluations. The Institute for Medical Technology Assessment Productivity Cost Questionnaire (iPCQ) was recently designed to optimize the features of existing instruments and to be a short generic outcome measure allowing for quantification and valuation of productivity costs. Feasibility, face validity, and partial reliability of the iPCQ have previously been confirmed.
      • This article is the first to investigate and demonstrate overall good content and construct validity and good reliability of the entire iPCQ. Hence, the iPCQ is recommended as a useful tool for measuring productivity cost among patients with musculoskeletal disorders, both for clinical and research purposes.
      • We believe that our article improves the toolset needed to conduct a comprehensive health economic evaluation and will assist in decision making concerning how to best allocate healthcare resources.

      Abstract

      Background

      The Institute for Medical Technology Assessment Productivity Cost Questionnaire (iPCQ) was recently developed to cover all domains of productivity costs; absenteeism, presenteeism and productivity costs related to unpaid work. The original iPCQ has not been tested with respect to neither content or construct validity, nor reliability, and there is no Norwegian version of the questionnaire.

      Objectives

      To translate and cross-culturally adapt the iPCQ into Norwegian and to test its measurement properties among patients with musculoskeletal disorders.

      Methods

      Translation and cross-cultural adaptation was conducted according to guidelines, and measurement properties were investigated using a cross-sectional design including a test–retest assessment. Patients with musculoskeletal disorders were recruited from secondary care. Data quality, content validity (10 patients evaluated comprehensibility, 2 researchers and 1 clinician evaluated relevance and comprehensiveness), construct validity (factor analysis, internal consistency, divergent hypothesis testing), and test–retest reliability (intraclass correlation coefficient two-way random average agreement, Cohen’s unweighted kappa) were assessed.

      Results

      In total, 115 patients with a mean age (SD) of 46 (9) years were included, and 62 responded to the retest. The questionnaire was feasible, with little missing data and no floor or ceiling effects. Content validity displayed good comprehensibility and relevance and sufficient comprehensiveness. Factor analysis revealed a 3-component solution accounting for 82% of the total variance; items loaded as expected and supported the original structure of the iPCQ. Internal consistency was acceptable for the 3 components of productivity cost, with an inter-item correlation ranging from 0.42 to 0.62. Further, a total of 91% of our hypotheses were verified. The intraclass correlation coefficient values ranged from 0.88 to 0.99 for all items except one; kappa ranged from 0.61 to 0.92, indicating overall good reliability of the questionnaire.

      Conclusions

      The Norwegian iPCQ showed good measurement properties among patients with musculoskeletal disorders from secondary care in Norway. We therefore recommend the iPCQ as a useful tool for measuring productivity costs in patients with musculoskeletal disorders.

      Keywords

      Introduction

      Musculoskeletal disorder is one of the leading causes of disability worldwide,
      Global Burden of Disease Study Collaborators
      Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013.
      accounting for a huge amount of productivity loss.
      • Bevan S.
      Economic impact of musculoskeletal disorders (MSDs) on work in Europe.
      The impact of disease and disorders on productivity is an important part of health economic evaluations. When a societal perspective is included in research, it can provide information on the relative cost of different disorders and on the relative cost-effectiveness of healthcare interventions, and it is therefore an important tool in decision making for how to best allocate resources.
      • Burdorf A.
      Economic evaluation in occupational health--its goals, challenges, and opportunities.
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      Currently, there is no gold standard for measuring productivity costs.
      • Krol M.
      • Brouwer W.
      • Rutten F.
      Productivity costs in economic evaluations: past, present, future.
      • Tang K.
      Estimating productivity costs in health economic evaluations: a review of instruments and psychometric evidence.
      • Zhang W.
      • Bansback N.
      • Anis A.H.
      Measuring and valuing productivity loss due to poor health: a critical review.
      Nevertheless, there is a general agreement that one should measure not only the productivity costs related to absence from paid work (absenteeism) and reduced productivity while at paid work (presenteeism) but also costs related to unpaid work such as household work, care work, and volunteer work.
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      The Institute for Medical Technology Assessment (iMTA) Productivity Cost Questionnaire (iPCQ) was recently developed to cover these 3 domains of productivity costs. It was designed to capture core parts of existing questionnaires and to be a short, generic, patient-reported outcome measure, allowing for quantification and valuation of all productivity costs.
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      Two studies have tested some of the measurement properties of the iPCQ. Bouwmans et al
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      confirmed its feasibility and face validity. In a modified version (iPCQ-VR), Beemster et al
      • Beemster T.T.
      • van Velzen J.M.
      • van Bennekom C.A.M.
      • Reneman M.F.
      • Frings-Dresen M.H.W.
      Test-retest reliability, agreement and responsiveness of productivity loss (iPCQ-VR) and healthcare utilization (TiCP-VR) questionnaires for sick workers with chronic musculoskeletal pain.
      tested reliability, agreement, and responsiveness of the core parts of absenteeism and presenteeism; they found good measurement properties on long-term sick leave and poor measurement properties on short-term sick leave and presenteeism. To the best of our knowledge, the original iPCQ version has not been tested with respect either to content or construct validity or to the reliability of the entire questionnaire. Furthermore, there is no Norwegian version of this instrument. Therefore, the purpose of this study was to translate and cross-culturally adapt the original iPCQ into Norwegian and to test its measurement properties among patients with musculoskeletal disorders.

      Methods

       Design

      This study was carried out in 2 stages. First, the original version of the iPCQ was translated and cross-culturally adapted into Norwegian. The Norwegian iPCQ was then tested for its measurement properties using a cross-sectional design. In addition, a test–retest assessment was conducted after 2 to 3 days.

       Translation and Cross-Cultural Adaptation

      The translation and cross-cultural adaptation was carried out according to international guidelines.
      • Guillemin F.
      • Bombardier C.
      • Beaton D.
      Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines.
      • Beaton D.E.
      • Bombardier C.
      • Guillemin F.
      • Ferraz M.B.
      Guidelines for the process of cross-cultural adaptation of self-report measures.
      Two persons (1 philologist and 1 clinician), whose mother tongue was Norwegian, independently translated the original iPCQ from English into Norwegian. The 2 Norwegian versions were then synthesized into 1 version before being translated back into English. Two native English speakers (1 philologist and 1 clinician), both blinded to the original iPCQ, independently performed the back-translation and synthesized the 2 English versions into one. An expert committee consisting of the translators and 2 researchers in our research group reviewed all translations. In a formal meeting, the committee discussed deviations until consensus on a prefinal version was reached. The goal was for the prefinal Norwegian iPCQ to be as concise and easy to understand as possible. The prefinal version was tested on 10 patients with musculoskeletal disorders. None of the patients had difficulty understanding the meaning of items or responses, and they found it easy to comprehend. No changes had to be made, so the final version of the Norwegian iPCQ evaluated in this study is the same as the prefinal version.

       Participants

      We planned to recruit 100 patients based on quality criteria recommended by Terwee et al
      • Terwee C.B.
      • Bot S.D.
      • de Boer M.R.
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.
      and Nunnally.
      • Nunnally J.O.
      Psychometric Theory.
      These criteria suggest a minimum of 100 participants for assessing internal consistency, at least 50 participants for assessing reproducibility and floor or ceiling effects,
      • Terwee C.B.
      • Bot S.D.
      • de Boer M.R.
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.
      and at least 10 participants for each item being included in the factor analysis.
      • Nunnally J.O.
      Psychometric Theory.
      Participants were recruited from secondary care at an outpatient rehabilitation clinic in Akershus, Norway, between November 2015 and January 2018. Eligible participants were patients with musculoskeletal disorders, aged 18 years or older, who were working or on sick leave. The exclusion criterion was the inability to speak, read, or write in Norwegian. Inclusion was performed by a clinician, primarily a physiotherapist, who met the patients at the clinic. At baseline, all patients received written and oral information about the study, and signed informed consent was obtained from all patients. The study was classified as a quality assessment study by the Norwegian Regional Committee for Medical Research Ethics (reference No. 2014/1634/REK vest) and was approved by the Norwegian Social Science Data Service (reference No. 45367) in 2015.

       Procedures and Measurements

      At baseline, the included patients completed the iPCQ as part of a comprehensive questionnaire, which also included sociodemographic variables, pain localization, pain intensity and history, health-related quality of life, physical workload at work, and psychosocial work environment. The McGill pain drawing and the Numeric Rating Scale were used to measure pain localization and intensity.
      • Escalante A.
      • Lichtenstein M.J.
      • White K.
      • Rios N.
      • Hazuda H.P.
      A method for scoring the pain map of the McGill Pain Questionnaire for use in epidemiologic studies.
      • Von Korff M.
      • Jensen M.P.
      • Karoly P.
      Assessing global pain severity by self-report in clinical and health services research.
      The Short Form–36 Health Status Questionnaire was used to measure health-related quality of life,
      • Ware Jr., J.E.
      • Sherbourne C.D.
      The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection.
      the Physical Workload Questionnaire was used to measure physical workload at work,
      • Bot S.D.
      • Terwee C.B.
      • van der Windt D.A.
      • et al.
      Internal consistency and validity of a new physical workload questionnaire.
      and questions from the QPS Nordic questionnaire were used to measure characteristics of the psychosocial work environment.
      • Dallner M.
      • Elo A.-L.
      • Gamberale F.
      • et al.
      Validation of the general Nordic questionnaire (QPSNordic) for psychological and social factors at work.
      Patients consenting to participate in the reproducibility part of the study completed the iPCQ at their second attendance, preferably with a 2- to 3-day interval.

       The iMTA Productivity Cost Questionnaire

      The iPCQ consists of 18 items and adopts a recall period of 4 weeks. In the introduction, 9 items (numbers A1 to A6 and 1 to 3) assess the date for reply and the following sociodemographic factors: age, sex, education level, work status, paid or unpaid work, profession, number of workdays, and work hours per week of paid work. Further, productivity costs are measured in 3 separate index scores with individual sum scores: absence from paid work (absenteeism), reduced productivity at paid work (presenteeism), and productivity loss in unpaid work. To calculate productivity costs, 8 core items are used. The value of absenteeism is calculated from items 2, 3, 4, and 6; presenteeism from items 2, 3, 8, and 9; and unpaid work productivity loss from items 11 and 12.
      • Bouwmans C.
      • Hakkaart-van Roijen L.
      • Koopmanschap M.
      • Krol M.
      • Severens H.
      • Brouwer W.
      Manual: iMTA Productivity Costs Questionnaire.
      The costs of productivity loss are valued in hours; hence, they can be translated by a standard cost price of productivity per hour.
      The 3 items (items 4 to 6) measuring productivity costs due to short- and long-term absence from paid work originate from the PRODISQ
      • Koopmanschap M.A.
      PRODISQ: a modular questionnaire on productivity and disease for economic evaluation studies.
      and the SF-HLQ
      • van Roijen L.
      • Essink-Bot M.L.
      • Koopmanschap M.A.
      • Bonsel G.
      • Rutten F.F.
      Labor and health status in economic evaluation of health care. The Health and Labor Questionnaire.
      and identify the occurrence and length of absenteeism. The validity of these questions in terms of feasibility, reliability, and construct validity (comparison between long-term absence and register data) has been demonstrated in previous studies.
      • Bouwmans C.
      • De Jong K.
      • Timman R.
      • et al.
      Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P).
      • Meerding W.J.
      • IJzelenberg W.
      • Koopmanschap M.A.
      • Severens J.L.
      • Burdorf A.
      Health problems lead to considerable productivity loss at work among workers with high physical load jobs.
      The 3 items (items 7 to 9) measuring productivity costs owing to presenteeism at paid work are composed of questions from the PRODISQ and the SF-HLQ and aim to identify whether the responders suffered from health problems at work and, if so, for how many days. As well, the responders are asked to rate their work performance on days with productivity loss in comparison with function on normal working days using an 11-point rating scale. The reliability of these questions has been shown to be acceptable using a test–retest design.
      • Bouwmans C.
      • De Jong K.
      • Timman R.
      • et al.
      Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P).
      The items (items 10 to 12) about productivity costs from unpaid work were developed at the iMTA at Erasmus University Rotterdam. The responders are asked whether they can perform less unpaid work, such as volunteer work and household work, as a result of health problems, and to state how many hours it would take someone else to replace this unperformed work.
      • Bouwmans C.
      • De Jong K.
      • Timman R.
      • et al.
      Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P).
      The English and the Norwegian versions and the manual for the iPCQ are available from the iMTA at Erasmus University Rotterdam.
      • Bouwmans C.
      • Hakkaart-van Roijen L.
      • Koopmanschap M.
      • Krol M.
      • Severens H.
      • Brouwer W.
      Manual: iMTA Productivity Costs Questionnaire.

      Kanters T. Questionnaires for the measurement of costs in economic evaluations, www.imta.nl/questionnaires. 2018.

       Analysis

      All data analyses were performed using SPSS version 24 (SPSS Inc, Chicago, IL), and the Vassarstats kappa was calculated using http://vassarstats.net/kappa.html. The measurement properties of the Norwegian iPCQ were tested as follows.

       Data quality

      Proportions of missing data and floor or ceiling effects were described. Floor or ceiling effects were considered to be present if greater than 15% of participants reported the lowest or highest possible score.
      • Terwee C.B.
      • Bot S.D.
      • de Boer M.R.
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.

       Content validity

      To assess content validity, the COSMIN group recommends evaluating relevance, comprehensiveness, and comprehensibility.
      • Terwee C.B.
      • Prinsen C.A.C.
      • Chiarotto A.
      • et al.
      COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
      In the present study, we asked, with open questions, 10 patients with musculoskeletal disorders about the comprehensibility of the iPCQ (are the instructions, items, and responses understood as intended; are the items appropriately worded; and do the response options match the question?). Two researchers and 1 clinician, with no conflict of interest, were asked about the relevance and the comprehensiveness of the iPCQ (are all included items relevant for the construct of productivity cost and the target population; are all key elements of productivity costs included?).

       Construct validity

      To assess construct validity, the COSMIN group recommends evaluating structural validity and internal consistency, followed by hypothesis testing.
      • Terwee C.B.
      • Prinsen C.A.C.
      • Chiarotto A.
      • et al.
      COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      In the present study, we expected the iPCQ to cover the 3 components of productivity costs: absenteeism, presenteeism, and unpaid work productivity costs. To confirm the underlying structure of the questionnaire and to investigate structural validity of the iPCQ, confirmatory factor analysis was conducted.
      • Pallant J.
      SPSS Survival Manual: A Step by Step Guide to Data Analysis Using IBM SPSS.
      A computed factor loading expresses the strength or magnitude of an association between a given item and a factor. The loading ranges from 0 to 1; the higher the value, the more an item is associated with a factor (a component).
      • Pallant J.
      SPSS Survival Manual: A Step by Step Guide to Data Analysis Using IBM SPSS.
      Based on the original structure of the questionnaire, we expected that the core items would load onto 3 components: absenteeism (items 2, 3, 4, and 6), presenteeism (items 2, 3, 8, and 9), and unpaid work productivity costs (items 11 and 12). Furthermore, we hypothesized that the internal consistency was sufficient for all 3 components of the iPCQ. Internal consistency of the components was assessed using interitem correlation. For the components to be considered sufficiently reliable, the interitem correlation should be greater than 0.4.
      Finally, we hypothesized that high productivity costs, assessed with the 3 index scores of the iPCQ, were negatively correlated with low health-related quality of life
      • Kigozi J.
      • Lewis M.
      • Jowett S.
      • Barton P.
      • Coast J.
      Construct validity and responsiveness of the single-item presenteeism question in patients with lower back pain for the measurement of presenteeism.
      • Lamers L.M.
      • Meerding W.J.
      • Severens J.L.
      • Brouwer W.B.
      The relationship between productivity and health-related quality of life: an empirical exploration in persons with low back pain.
      and positively correlated with low physical workload,
      • Koopmanschap M.A.
      PRODISQ: a modular questionnaire on productivity and disease for economic evaluation studies.
      • Canjuga M.
      • Hammig O.
      • Bauer G.F.
      • Laubli T.
      Correlates of short- and long-term absence due to musculoskeletal disorders.
      • IJzelenberg W.
      • Molenaar D.
      • Burdorf A.
      Different risk factors for musculoskeletal complaints and musculoskeletal sickness absence.
      • d’Errico A.
      • Viotti S.
      • Baratti A.
      • et al.
      Low back pain and associated presenteeism among hospital nursing staff.
      low psychosocial work environment,
      • Canjuga M.
      • Hammig O.
      • Bauer G.F.
      • Laubli T.
      Correlates of short- and long-term absence due to musculoskeletal disorders.
      • IJzelenberg W.
      • Molenaar D.
      • Burdorf A.
      Different risk factors for musculoskeletal complaints and musculoskeletal sickness absence.
      • d’Errico A.
      • Viotti S.
      • Baratti A.
      • et al.
      Low back pain and associated presenteeism among hospital nursing staff.
      • Steenstra I.A.
      • Verbeek J.H.
      • Heymans M.W.
      • Bongers P.M.
      Prognostic factors for duration of sick leave in patients sick listed with acute low back pain: a systematic review of the literature.
      • Kresal F.
      • Suklan J.
      • Roblek V.
      • Jerman A.
      • Mesko M.
      Psychosocial risk factors for low back pain and absenteeism among Slovenian professional drivers.
      • Janssens H.
      • Clays E.
      • de Clercq B.
      • et al.
      Association between psychosocial characteristics of work and presenteeism: a cross-sectional study.
      and much pain
      • Kigozi J.
      • Lewis M.
      • Jowett S.
      • Barton P.
      • Coast J.
      Construct validity and responsiveness of the single-item presenteeism question in patients with lower back pain for the measurement of presenteeism.
      • Canjuga M.
      • Hammig O.
      • Bauer G.F.
      • Laubli T.
      Correlates of short- and long-term absence due to musculoskeletal disorders.
      • Steenstra I.A.
      • Verbeek J.H.
      • Heymans M.W.
      • Bongers P.M.
      Prognostic factors for duration of sick leave in patients sick listed with acute low back pain: a systematic review of the literature.
      • Zhang W.
      • Bansback N.
      • Kopec J.
      • Anis A.H.
      Measuring time input loss among patients with rheumatoid arthritis: validity and reliability of the Valuation of Lost Productivity questionnaire.
      • Lardon A.
      • Dubois J.D.
      • Cantin V.
      • Piche M.
      • Descarreaux M.
      Predictors of disability and absenteeism in workers with non-specific low back pain: a longitudinal 15-month study.
      (divergent construct validity). These hypotheses were based on previous studies.
      • Koopmanschap M.A.
      PRODISQ: a modular questionnaire on productivity and disease for economic evaluation studies.
      • Kigozi J.
      • Lewis M.
      • Jowett S.
      • Barton P.
      • Coast J.
      Construct validity and responsiveness of the single-item presenteeism question in patients with lower back pain for the measurement of presenteeism.
      • Lamers L.M.
      • Meerding W.J.
      • Severens J.L.
      • Brouwer W.B.
      The relationship between productivity and health-related quality of life: an empirical exploration in persons with low back pain.
      • Canjuga M.
      • Hammig O.
      • Bauer G.F.
      • Laubli T.
      Correlates of short- and long-term absence due to musculoskeletal disorders.
      • IJzelenberg W.
      • Molenaar D.
      • Burdorf A.
      Different risk factors for musculoskeletal complaints and musculoskeletal sickness absence.
      • d’Errico A.
      • Viotti S.
      • Baratti A.
      • et al.
      Low back pain and associated presenteeism among hospital nursing staff.
      • Steenstra I.A.
      • Verbeek J.H.
      • Heymans M.W.
      • Bongers P.M.
      Prognostic factors for duration of sick leave in patients sick listed with acute low back pain: a systematic review of the literature.
      • Kresal F.
      • Suklan J.
      • Roblek V.
      • Jerman A.
      • Mesko M.
      Psychosocial risk factors for low back pain and absenteeism among Slovenian professional drivers.
      • Janssens H.
      • Clays E.
      • de Clercq B.
      • et al.
      Association between psychosocial characteristics of work and presenteeism: a cross-sectional study.
      • Zhang W.
      • Bansback N.
      • Kopec J.
      • Anis A.H.
      Measuring time input loss among patients with rheumatoid arthritis: validity and reliability of the Valuation of Lost Productivity questionnaire.
      • Lardon A.
      • Dubois J.D.
      • Cantin V.
      • Piche M.
      • Descarreaux M.
      Predictors of disability and absenteeism in workers with non-specific low back pain: a longitudinal 15-month study.
      In general, there is some inconsistency in the literature, but to the best of our knowledge, it appears that most available studies demonstrate a low correlation between these variables and productivity costs. Spearman’s rho was used in all correlation analyses because the scales were not normally distributed. Correlation coefficients less than 0.3, between 0.3 and 0.6, and greater than 0.6 were considered low, moderate, and high, respectively.
      • Andresen E.M.
      Criteria for assessing the tools of disability outcomes research.

       Reliability

      The test–retest reliability of the continuous variables (items 2, 3, 4 second part, 6, 8, 9, 11, and 12) and the 3 index scores was assessed with the intraclass correlation coefficient (ICC) using 2-way random, average agreement. The acceptable level of ICC was set to >0.70.
      • Terwee C.B.
      • Bot S.D.
      • de Boer M.R.
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.
      In addition, Cohen’s unweighted kappa was used for dichotomous variables (items 4 first part, 5, 7, and 10) of the iPCQ. Kappa values were categorized according to Altman: poor (0 to 0.2), fair (0.21 to 0.40), moderate (0.41 to 0.60), good (0.61 to 0.80), and very good (0.81 to 1.00).
      • Altman D.G.
      Practical Statistics for Medical Research.

      Results

      A total of 115 patients with a mean age (SD) of 46 (9) years were included in the cross-sectional study, and a sample of 62 participants completed the retest questionnaire. The median time interval between test and retest was 3 days (range, 1-10 days). Almost all included patients were in paid work (90%), and more than half had been on sick leave during the previous 4 weeks. On average, they reported moderate pain, and the most frequently reported pain area was the back region. Study sample characteristics are shown in Table 1.
      Table 1Patient demographic characteristics and clinical status.
      Validity study (n = 115)Test-retest study (n = 62)
      Missing, n (%)Missing, n (%)
      Male, n (%)36 (31.3)0 (0)23 (37.1)0 (0)
      Age in years, mean (SD)45.6 (9.3)0 (0)46.3 (8.5)0 (0)
      Education level high, n (%)67 (58.2)0 (0)35 (56.5)0 (0)
      Mother tongue Norwegian, n (%)100 (87.0)0 (0)53 (85.5)0 (0)
      Work status, n (%)0 (0)0 (0)
       Employed or self-employed (paid job)104 (90.4)0 (0)55 (88.7)0 (0)
       Sick leave during past 4 weeks79 (68.7)1 (0.9)40 (64.5)0 (0)
       Rehabilitation, work disability7 (6.1)0 (0)4 (6.5)0 (0)
      Pain period in days, median (range)700 (5-10 950)26 (22.6)720 (5-10 950)15 (13.0)
      Pain severity last week (NRS 0-10), median (range)5 (1-9)3 (2.6)5 (1-9)1 (1.6)
      Pain location, n (%)0 (0)0 (0)
       Lower limbs70 (60.9)36 (58.1)
       Back80 (69.6)47 (75.8)
       Neck57 (49.6)26 (41.9)
       Upper limbs55 (47.8)24 (38.7)
       >2 pain areas45 (39.1)23 (37.1)
      Health-related quality of life (SF-36 0-100), median (range)
       Mental health70.0 (10-100)1 (0.9)75.0 (10-95)1 (1.6)
       Physical function75 (30-100)0 (0)70 (30-95)0 (0)
      Physical workload (PWQ 0-100), median (range)
       Heavy physical workload20.8 (0-86.1)7 (6.1)26.4 (0-86.1)4 (6.5)
       Long-lasting posture and repetitive movement50.0 (0-100)3 (4.8)50.0 (0-100)3 (4.8)
      QPS Nordic (1-5), median (range)
       Control of decisions2.8 (1-5)7 (6.1)2.8 (1-5)3 (4.8)
       Authorizing management3.3 (1-5)7 (6.1)3.7 (1-5)6 (9.7)
       Role conflict2.3 (1-5)5 (4.4)2.3 (1-5)4 (6.5)
       Fair leadership4.0 (1-5)13 (11.3)4.0 (1.3-5)9 (14.5)
      NRS indicates Numeric Rating Scale; PWQ, Physical Workload Questionnaire; QPS Nordic, General Nordic questionnaire for psychological and social factors at work; SF-36, 36-Item Short-Form Health Survey.

       Data quality

      The proportion of missing data was relatively small: less than 10% for all items (ranging from 0% to 9.6%). There were no floor or ceiling effects for any of the items. All continuous variables in the iPCQ and the sum scores for the 3 index scores had a skewed distribution. Descriptive statistics for the core items and the index scores are listed in Table 2.
      Table 2Descriptive statistics, including missing data, for the iPCQ core items and the index scores (n = 115).
      The index score of absenteeism is calculated from core items 2, 3, 4, and 6; presenteeism from core items 2, 3, 8, and 9; and productivity loss unpaid work from core items 11 and 12.18 NA indicates not applicable, due to the structure of the iPCQ.
      Core items (label, wording, and response format)Missing, n (%)NA, n (%)Median (range)
      #2. Weekly work hours
      How many hours a week do you work? Count only the hours that you get paid (. . . hours)3 (2.6)11 (9.6)37.5 (7.5-52)
      #3. Weekly workdays
      How many days a week do you work? (. . . days)2 (1.7)11 (9.6)5 (2-7)
      #4. Number of days absenteeism short term
      Have you missed work in the last 4 weeks as a result of being sick? (No; Yes I have missed . . . days)5 (4.3)35 (30.4)14 (1-21)
      #6. Numbers of days absenteeism long term
      Did you miss work earlier than the period of 4 weeks due to being sick? This is referring to one whole uninterrupted period of missed work as a result of being sick. (No, Yes). If yes, when did you call in sick? (day, month, year)3 (2.6)46 (40.0)175 (18-586)
      #8. Number of workdays with disability
      How many days at work were you bothered by physical or psychological problems? Only count the days at work in the last 4 weeks. (. . . workdays)0 (0)83 (72.2)11 (3-28)
      #9. Effective score completed work
      On the days that you were bothered by these problems, was it perhaps difficult to get as much work finished as you normally do? On these days how much work could you do on average?0 (0)83 (72.2)7 (3-10)
      Look at the figures below. A 10 means that you were able to do as much work as you normally do. A 0 means that you were unable to do any work on these days. (0-10, Likert-type scale)
      #11. Number of days less unpaid work
      How many days did this happen? Only count the days in the last 4 weeks. (…. days)7 (6.1)59 (51.3)15 (1-28)
      #12. Number of hours less unpaid work
      Imagine that somebody, for example, your partner, family member, or friend, helped you on these days, and he or she did all the unpaid work that you were unable to do for you. How many hours on average did that person spend doing this on these days? (On average . . . hours on these days)11 (9.6)59 (51.3)20 (2-280)
      Index scores
       Absenteeism, hours9 (7.8)35 (30.4)1155 (8-4688)
       Presenteeism, hours1 (0.9)86 (74.8)25 (5-105)
       Productivity loss unpaid work, hours12 (10.4)59 (51.3)327 (6-7840)
      iPCQ indicates Institute for Medical Technology Assessment Productivity Cost Questionnaire.
      The index score of absenteeism is calculated from core items 2, 3, 4, and 6; presenteeism from core items 2, 3, 8, and 9; and productivity loss unpaid work from core items 11 and 12.
      • Bouwmans C.
      • Hakkaart-van Roijen L.
      • Koopmanschap M.
      • Krol M.
      • Severens H.
      • Brouwer W.
      Manual: iMTA Productivity Costs Questionnaire.
      NA indicates not applicable, due to the structure of the iPCQ.

       Content validity

      Overall, the iPCQ has sufficient content validity. The included items are relevant and cover all domains of productivity costs, except compensation mechanisms and part-time sick leave. Moreover, the questionnaire was understood as intended by the 10 patients evaluating comprehensibility. Nevertheless, when going through all responses in the validity study, we found some deviations in the question about “number of hours with less unpaid work.” Some patients recorded hours per day, whereas others recorded the total number of hours in the 4-week period.

       Construct validity

      The confirmatory factor analysis revealed a 3-component solution accounting for 82% of the total variance in the data. The first, second, and third components explained 31%, 29%, and 23% of the total variance, respectively. Furthermore, the confirmatory factor analysis displayed, as expected, that core items 2, 3, 4, and 6 load on component 1; that core items 2, 3, 8, and 9 load on component 2; and that core items 11 and 12 load on component 3 (Table 3).
      Table 3Confirmatory factor analyses with item loading.
      Component
      Item123
      Weekly work hours0.770.55
      Weekly workdays0.770.57
      Numbers of days absenteeism short term0.81
      Numbers of days absenteeism long term0.70−0.39
      Numbers of workdays with disability0.820.34
      Effective score completed work0.85
      Number of days less unpaid work0.89
      Number of hours less unpaid work0.92
      Factor loading greater than 0.3 is reported.
      The internal consistency and the level of interitem correlation were acceptable for the 3 components with values of 0.46, 0.42, and 0.62 for absenteeism, presenteeism, and productivity costs from unpaid work, respectively.
      The results for the divergent construct validity are presented in Table 4. As expected, the correlation coefficients have low values (<0.30) between the 3 index scores of the iPCQ and almost all of the health- and work-related variables. There was a moderate correlation between absenteeism and control of decisions and between presenteeism and heavy physical workload. A total of 91% of our hypotheses were confirmed.
      Table 4Correlation of iPCQ domains with other health-related variables.
      iPCQ domainnCorrelation coefficient
      Calculated as Spearman’s rho.
      (P value)
      Absenteeism
       Physical function health related QOL (SF-36)95−0.107 (.301)
       Mental-health-related QOL (SF-36)94−0.210 (.042)
       Heavy physical workload (PWQ)950.157 (.128)
       Long-lasting posture and repetitive movement (PWQ)950.165 (.110)
       Pain intensity last week (NRS)920.194 (.064)
       Psychosocial work environment (QPS Nordic)
       Control of decisions89−0.336 (.001)
       Authorizing leadership92−0.087 (.409)
       Role conflict930.035 (.736)
       Fair leadership870.212 (.048)
      Presenteeism
       Physical function health-related QOL (SF-36)43−0.058 (.713)
       Mental-health-related QOL (SF-36)43−0.202 (.194)
       Heavy physical workload (PWQ)430.341 (.025)
       Long-lasting posture and repetitive movement (PWQ)430.113 (.472)
       Pain intensity last week (NRS)420.133 (.399)
       Psychosocial work environment (QPS Nordic)
       Control of decisions40−0.033 (.840)
       Authorizing management430.054 (.733)
       Role conflict430.297 (.053)
       Fair leadership39−0.239 (.143)
      Unpaid work productivity loss
       Physical function health-related QOL (SF-36)103−0.182 (.067)
       Mental-health-related QOL (SF-36)103−0.179 (.071)
       Pain intensity last week (NRS)1000.195 (.052)
       Number of pain locations (McGill pain drawing)1030.205 (.038)
      iPCQ indicates Institute for Medical Technology Assessment Productivity Cost Questionnaire; NRS, Numeric Rating Scale; PWQ, Physical Workload Questionnaire; QPS Nordic, General Nordic questionnaire for psychological and social factors at work; SF-36, 36-Item Short-Form Health Survey.
      Calculated as Spearman’s rho.

       Reliability

      Table 5 shows the test–retest reliability results for the 3 index scores and the core items. According to the ICCs, reliability was greater than the recommended minimum standard for the 3 index scores and all core items, with the exception of item 8 (number of workdays with disability; 0.34). The ICCs for the iPCQ index scores ranged from 0.89 to 0.99. Kappa values of the 4 dichotomous items of iPCQ ranged from 0.62 to 0.84. Item 7 for work despite disability during the last 4 weeks scored the lowest, and item 5 for absenteeism before the last 4 weeks scored the highest.
      Table 5Test–retest reliability results for the iPCQ core items and the index scores.
      Core items (label) and index scores (n)Median test (interquartile range)Median retest (interquartile range)% AgreementICC (95% CI)
      Weekly work hours (48)37.5 (37.5-40)37.5 (37.5-40)91.70.92 (0.86-0.96)
      Weekly workdays (52)5 (5-5)5 (5-5)96.20.88 (0.79-0.93)
      Number of days absenteeism short term (34)13.5 (12-20)12 (12-20)79.40.92 (0.83-0.96)
      Number of days absenteeism long term (29)197 (86-312)199 (78-315)86.20.99 (0.99-0.99)
      Number of workdays with disability (18)16 (8-20)19 (10-20)72.20.34 (−0.89-0.76)
      Effective score completed work (20)7 (5-8)7 (5-8)75.00.98 (0.94-0.99)
      Number of days less unpaid work (21)15 (7-23)16 (5-24)52.40.88 (0.69-0.95)
      Number of hours less unpaid work (20)20 (9-29)11 (9-24)85.00.99 (0.99-0.99)
      Index scores
       Absenteeism long term, hours (23)1576 (698-2363)1592 (555-2393)0.99 (0.98-0.99)
       Absenteeism short term, hours (2)34.3 (16-53)53.5 (32-75)0.89 (−0.04-1.0)
       Presenteeism, hours (16)23 (10-49)31 (14-48)0.91 (0.74-0.97)
       Productivity loss unpaid work, hours (19)294 (48-500)80 (50-560)0.98 (0.94-0.99)
      % agreement indicates (number of identical/total answers) × 100; CI, confidence interval; ICC, intraclass correlation coefficient agreement, 2-way random, average measures; iPCQ, Institute for Medical Technology Assessment Productivity Cost Questionnaire.

      Discussion

      In this study, we assessed the measurement properties of the Norwegian version of the iPCQ, and the overall results were good with respect to data quality, content and construct validity, and reliability.
      The iPCQ is a relatively new questionnaire, and there are only 2 previous studies with which we can compare our results.
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      • Beemster T.T.
      • van Velzen J.M.
      • van Bennekom C.A.M.
      • Reneman M.F.
      • Frings-Dresen M.H.W.
      Test-retest reliability, agreement and responsiveness of productivity loss (iPCQ-VR) and healthcare utilization (TiCP-VR) questionnaires for sick workers with chronic musculoskeletal pain.
      The data quality in our study, showing little missing data, is in line with the original study of Bouwmans et al,
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      in which the iPCQ was tested in the general population.
      Our evaluation of content validity showed good comprehensibility of the iPCQ, with the exception of item 12 (number of hours with less unpaid work). Some patients recorded hours per day, whereas others recorded the total number of hours in the 4-week period. Hence, a small adjustment was made to the questionnaire after finishing this study, and participants are now specifically requested to record hours per day. Further, the included items were considered to be relevant and to cover the main domains of productivity costs. Nevertheless, there are 2 exceptions: The iPCQ does not cover compensation mechanisms or part-time sick leave. Compensation mechanisms may influence the total value of productivity costs.
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      • Krol M.
      • Brouwer W.
      • Rutten F.
      Productivity costs in economic evaluations: past, present, future.
      • Zhang W.
      • Anis A.H.
      Health-related productivity loss: NICE to recognize soon, good to discuss now.
      Nonetheless, the extent to which these mechanisms affect final productivity costs remains unclear, and adjusting for them currently seems premature.
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      • Krol M.
      • Brouwer W.
      • Rutten F.
      Productivity costs in economic evaluations: past, present, future.
      • Zhang W.
      • Anis A.H.
      Health-related productivity loss: NICE to recognize soon, good to discuss now.
      Nevertheless, it would be possible to include items covering part-time sick leave, thereby increasing the usefulness of the iPCQ. In the iPCQ-VR, Beemster et al
      • Beemster T.T.
      • van Velzen J.M.
      • van Bennekom C.A.M.
      • Reneman M.F.
      • Frings-Dresen M.H.W.
      Test-retest reliability, agreement and responsiveness of productivity loss (iPCQ-VR) and healthcare utilization (TiCP-VR) questionnaires for sick workers with chronic musculoskeletal pain.
      showed that part-time sick leave could be reliably measured in patients with nonspecific musculoskeletal pain.
      Our assessment of construct validity confirmed a 3-component solution, which is similar to the original study of the iPCQ.
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      Bouwmans et al
      • Bouwmans C.
      • Krol M.
      • Severens H.
      • et al.
      The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
      distinguished between the 3 components based on a theoretical rationale. Our study is the first to confirm that the core items load as expected and that the 3 components accounted for as much as 82% of the total variance in the data. The internal consistency was acceptable for the 3 components. Furthermore, construct validity was supported by the hypothesis testing, which has not been reported previously.
      In the present study, the iPCQ showed good reliability for the index scores and all of the individual items, except item 8 (number of workdays with disability). The study of Beemster et al
      • Beemster T.T.
      • van Velzen J.M.
      • van Bennekom C.A.M.
      • Reneman M.F.
      • Frings-Dresen M.H.W.
      Test-retest reliability, agreement and responsiveness of productivity loss (iPCQ-VR) and healthcare utilization (TiCP-VR) questionnaires for sick workers with chronic musculoskeletal pain.
      supports a good reliability of the items related to long-term sick leave and a low reliability of the item covering number of workdays with disability. Regarding items related to short-term sick leave (item 4) and presenteeism (item 9), our results indicated a higher reliability than was found by Beemster et al.
      • Beemster T.T.
      • van Velzen J.M.
      • van Bennekom C.A.M.
      • Reneman M.F.
      • Frings-Dresen M.H.W.
      Test-retest reliability, agreement and responsiveness of productivity loss (iPCQ-VR) and healthcare utilization (TiCP-VR) questionnaires for sick workers with chronic musculoskeletal pain.
      This difference might be explained by a different time interval between test and retest, as Beemster et al
      • Beemster T.T.
      • van Velzen J.M.
      • van Bennekom C.A.M.
      • Reneman M.F.
      • Frings-Dresen M.H.W.
      Test-retest reliability, agreement and responsiveness of productivity loss (iPCQ-VR) and healthcare utilization (TiCP-VR) questionnaires for sick workers with chronic musculoskeletal pain.
      used an average of 20 days compared with 3 in this study. The reliability of items 11 and 12, covering productivity costs from unpaid work, have not been tested previously.
      The main limitation of this study is that we did not compare the index scores for absenteeism against public register data, which is recommended.
      • Zhang W.
      • Anis A.H.
      Health-related productivity loss: NICE to recognize soon, good to discuss now.
      We are, however, conducting a new study in which this comparison will be carried out. A second potential weakness is the hypothesis testing in the present study. Because construct validity is concerned with how well an instrument captures the intended construct,
      • Tang K.
      Estimating productivity costs in health economic evaluations: a review of instruments and psychometric evidence.
      • Terwee C.B.
      • Bot S.D.
      • de Boer M.R.
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.
      it is questionable to what degree divergent hypothesis testing alone can be used to provide evidence for construct validity. We could have tested the iPCQ against questionnaires that measure the same construct; however, there is no gold standard regarding the measurement of productivity cost, and different questionnaires often result in varied estimates, especially of presenteeism.
      • Tang K.
      Estimating productivity costs in health economic evaluations: a review of instruments and psychometric evidence.
      • van Roijen L.
      • Essink-Bot M.L.
      • Koopmanschap M.A.
      • Bonsel G.
      • Rutten F.F.
      Labor and health status in economic evaluation of health care. The Health and Labor Questionnaire.
      • Meerding W.J.
      • IJzelenberg W.
      • Koopmanschap M.A.
      • Severens J.L.
      • Burdorf A.
      Health problems lead to considerable productivity loss at work among workers with high physical load jobs.
      • Braakman-Jansen L.M.
      • Taal E.
      • Kuper I.H.
      • van de Laar M.A.
      Productivity loss due to absenteeism and presenteeism by different instruments in patients with RA and subjects without.
      • Zhang W.
      • Gignac M.A.
      • Beaton D.
      • et al.
      Productivity loss due to presenteeism among patients with arthritis: estimates from 4 instruments.
      • Beaton D.E.
      • Tang K.
      • Gignac M.A.
      • et al.
      Reliability, validity, and responsiveness of five at-work productivity measures in patients with rheumatoid arthritis or osteoarthritis.
      A third potential weakness of this study is that we have tested measurement properties of a generic questionnaire only in a sample of patients with musculoskeletal disorders in Norway. Obviously, testing measurement properties in different settings and including patients with different disorders is recommended. A fourth weakness of this study is the lack of data on eligible study participants who declined to participate. Because of limited resources, it was not possible to record information on all patients attending the rehabilitation clinic during the data collection period.
      The main strength of the present study is that it is the first to test the content and construct validity as well as reliability of the original iPCQ and that this testing was conducted in line with COSMIN guidelines.
      • Terwee C.B.
      • Bot S.D.
      • de Boer M.R.
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.

      Conclusion

      This study showed that the iPCQ has good measurement properties for measuring productivity costs among patients with musculoskeletal disorders receiving rehabilitation in a secondary outpatient clinic in Norway. We can recommend using the Norwegian version of the iPCQ for clinical and research purposes on patients with musculoskeletal disorders. Because the iPCQ is a generic instrument, further studies should validate it in other patient populations.

      Acknowledgments

      Ingeborg Landstad and Unicare Friskvern contributed to data collection.
      Source of financial support: This study was supported by Oslo Metropolitan University.

      References

        • Global Burden of Disease Study Collaborators
        Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013.
        Lancet. 2015; 386: 743-800
        • Bevan S.
        Economic impact of musculoskeletal disorders (MSDs) on work in Europe.
        Best Pract Res Clin Rheumatol. 2015; 29: 356-373
        • Burdorf A.
        Economic evaluation in occupational health--its goals, challenges, and opportunities.
        Scand J Work Environ Health. 2007; 33: 161-164
        • Bouwmans C.
        • Krol M.
        • Severens H.
        • et al.
        The iMTA Productivity Cost Questionnaire: a standardized instrument for measuring and valuing health-related productivity losses.
        Value Health. 2015; 18: 753-758
        • Krol M.
        • Brouwer W.
        • Rutten F.
        Productivity costs in economic evaluations: past, present, future.
        Pharmacoeconomics. 2013; 31: 537-549
        • Tang K.
        Estimating productivity costs in health economic evaluations: a review of instruments and psychometric evidence.
        Pharmacoeconomics. 2015; 33: 31-48
        • Zhang W.
        • Bansback N.
        • Anis A.H.
        Measuring and valuing productivity loss due to poor health: a critical review.
        Soc Sci Med. 2011; 72: 185-192
        • Beemster T.T.
        • van Velzen J.M.
        • van Bennekom C.A.M.
        • Reneman M.F.
        • Frings-Dresen M.H.W.
        Test-retest reliability, agreement and responsiveness of productivity loss (iPCQ-VR) and healthcare utilization (TiCP-VR) questionnaires for sick workers with chronic musculoskeletal pain.
        J Occup Rehabil. 2019; 29: 91-103
        • Guillemin F.
        • Bombardier C.
        • Beaton D.
        Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines.
        J Clin Epidemiol. 1993; 46: 1417-1432
        • Beaton D.E.
        • Bombardier C.
        • Guillemin F.
        • Ferraz M.B.
        Guidelines for the process of cross-cultural adaptation of self-report measures.
        Spine (Phila Pa 1976). 2000; 25: 3186-3191
        • Terwee C.B.
        • Bot S.D.
        • de Boer M.R.
        • et al.
        Quality criteria were proposed for measurement properties of health status questionnaires.
        J Clin Epidemiol. 2007; 60: 34-42
        • Nunnally J.O.
        Psychometric Theory.
        McGraw-Hill, New York, NY1978
        • Escalante A.
        • Lichtenstein M.J.
        • White K.
        • Rios N.
        • Hazuda H.P.
        A method for scoring the pain map of the McGill Pain Questionnaire for use in epidemiologic studies.
        Aging (Milano). 1995; 7: 358-366
        • Von Korff M.
        • Jensen M.P.
        • Karoly P.
        Assessing global pain severity by self-report in clinical and health services research.
        Spine (Phila Pa 1976). 2000; 25: 3140-3151
        • Ware Jr., J.E.
        • Sherbourne C.D.
        The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection.
        Med Care. 1992; 30: 473-483
        • Bot S.D.
        • Terwee C.B.
        • van der Windt D.A.
        • et al.
        Internal consistency and validity of a new physical workload questionnaire.
        Occup Environ Med. 2004; 61: 980-986
        • Dallner M.
        • Elo A.-L.
        • Gamberale F.
        • et al.
        Validation of the general Nordic questionnaire (QPSNordic) for psychological and social factors at work.
        Nordic Council of Ministers, ed. Vol Nord 200:12. Nordic Council of Ministers, Copenhagen, Denmark2000
        • Bouwmans C.
        • Hakkaart-van Roijen L.
        • Koopmanschap M.
        • Krol M.
        • Severens H.
        • Brouwer W.
        Manual: iMTA Productivity Costs Questionnaire.
        • Koopmanschap M.A.
        PRODISQ: a modular questionnaire on productivity and disease for economic evaluation studies.
        Expert Rev Pharmacoecon Outcomes Res. 2005; 5: 23-28
        • van Roijen L.
        • Essink-Bot M.L.
        • Koopmanschap M.A.
        • Bonsel G.
        • Rutten F.F.
        Labor and health status in economic evaluation of health care. The Health and Labor Questionnaire.
        Int J Technol Assess Health Care. 1996; 12: 405-415
        • Bouwmans C.
        • De Jong K.
        • Timman R.
        • et al.
        Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P).
        BMC Health Serv Res. 2013; 13: 217
        • Meerding W.J.
        • IJzelenberg W.
        • Koopmanschap M.A.
        • Severens J.L.
        • Burdorf A.
        Health problems lead to considerable productivity loss at work among workers with high physical load jobs.
        J Clin Epidemiol. 2005; 58: 517-523
      1. Kanters T. Questionnaires for the measurement of costs in economic evaluations, www.imta.nl/questionnaires. 2018.

        • Terwee C.B.
        • Prinsen C.A.C.
        • Chiarotto A.
        • et al.
        COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
        Qual Life Res. 2018; 27: 1159-1170
        • Mokkink L.B.
        • de Vet H.C.W.
        • Prinsen C.A.C.
        • et al.
        COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
        Qual Life Res. 2018; 27: 1171-1179
        • Pallant J.
        SPSS Survival Manual: A Step by Step Guide to Data Analysis Using IBM SPSS.
        6th ed. Allen & Unwin, Sydney, Australia2016
      2. Nunnally J. Bernstein I. Psychometric Theory. 3rd ed. McGraw-Hill, New York, NY1994
        • Kigozi J.
        • Lewis M.
        • Jowett S.
        • Barton P.
        • Coast J.
        Construct validity and responsiveness of the single-item presenteeism question in patients with lower back pain for the measurement of presenteeism.
        Spine (Phila Pa 1976). 2014; 39: 409-416
        • Lamers L.M.
        • Meerding W.J.
        • Severens J.L.
        • Brouwer W.B.
        The relationship between productivity and health-related quality of life: an empirical exploration in persons with low back pain.
        Qual Life Res. 2005; 14: 805-813
        • Canjuga M.
        • Hammig O.
        • Bauer G.F.
        • Laubli T.
        Correlates of short- and long-term absence due to musculoskeletal disorders.
        Occup Med (Lond). 2010; 60: 358-361
        • IJzelenberg W.
        • Molenaar D.
        • Burdorf A.
        Different risk factors for musculoskeletal complaints and musculoskeletal sickness absence.
        Scand J Work Environ Health. 2004; 30: 56-63
        • d’Errico A.
        • Viotti S.
        • Baratti A.
        • et al.
        Low back pain and associated presenteeism among hospital nursing staff.
        J Occup Health. 2013; 55: 276-283
        • Steenstra I.A.
        • Verbeek J.H.
        • Heymans M.W.
        • Bongers P.M.
        Prognostic factors for duration of sick leave in patients sick listed with acute low back pain: a systematic review of the literature.
        Occup Environ Med. 2005; 62: 851-860
        • Kresal F.
        • Suklan J.
        • Roblek V.
        • Jerman A.
        • Mesko M.
        Psychosocial risk factors for low back pain and absenteeism among Slovenian professional drivers.
        Cent Eur J Public Health. 2017; 25: 135-140
        • Janssens H.
        • Clays E.
        • de Clercq B.
        • et al.
        Association between psychosocial characteristics of work and presenteeism: a cross-sectional study.
        Int J Occup Med Environ Health. 2016; 29: 331-344
        • Zhang W.
        • Bansback N.
        • Kopec J.
        • Anis A.H.
        Measuring time input loss among patients with rheumatoid arthritis: validity and reliability of the Valuation of Lost Productivity questionnaire.
        J Occup Environ Med. 2011; 53: 530-536
        • Lardon A.
        • Dubois J.D.
        • Cantin V.
        • Piche M.
        • Descarreaux M.
        Predictors of disability and absenteeism in workers with non-specific low back pain: a longitudinal 15-month study.
        Appl Ergon. 2018; 68: 176-185
        • Andresen E.M.
        Criteria for assessing the tools of disability outcomes research.
        Arch Phys Med Rehabil. 2000; 81: S15-S20
        • Altman D.G.
        Practical Statistics for Medical Research.
        Chapman and Hall, London, UK1991
        • Zhang W.
        • Anis A.H.
        Health-related productivity loss: NICE to recognize soon, good to discuss now.
        Pharmacoeconomics. 2014; 32: 425-427
        • Braakman-Jansen L.M.
        • Taal E.
        • Kuper I.H.
        • van de Laar M.A.
        Productivity loss due to absenteeism and presenteeism by different instruments in patients with RA and subjects without.
        RA. Rheumatology (Oxford). 2012; 51: 354-361
        • Zhang W.
        • Gignac M.A.
        • Beaton D.
        • et al.
        Productivity loss due to presenteeism among patients with arthritis: estimates from 4 instruments.
        J Rheumatol. 2010; 37: 1805-1814
        • Beaton D.E.
        • Tang K.
        • Gignac M.A.
        • et al.
        Reliability, validity, and responsiveness of five at-work productivity measures in patients with rheumatoid arthritis or osteoarthritis.
        Arthritis Care Res (Hoboken). 2010; 62: 28-37