Esther W. C. van der Meer1, Johanna M. van Dongen2, Cécile R. L. Boot1,3, Joost W. J. van der Gulden4, Judith E. Bosmans2 and Johannes R. Anema1,3,5

1Department of Public and Occupational Health, EMGO Institute for Health and Care Research, VU University Medical Center, 2Department of Health Sciences and the EMGO Institute for Health and Care Research, Faculty of Earth and Life Sciences, VU University Amsterdam, 3Body@Work, Research Center Physical Activity, Work and Health, TNO-VU University Medical Center, Amsterdam, 4Department of Primary and Community Care, Centre for Family Medicine, Geriatric Care and Public Health, Radboud University Nijmegen Medical Centre, Nijmegen, and 5Research Center for Insurance Medicine AMC-UWV-VU University Medical Center, Amsterdam, The Netherlands

The aim of this study was to evaluate the cost-effectiveness of a multifaceted implementation strategy for the prevention of hand eczema in comparison with a control group among healthcare workers. A total of 48 departments (n = 1,649) were randomly allocated to the implementation strategy or the control group. Data on hand eczema and costs were collected at baseline and every 3 months. Cost-effectiveness analyses were performed using linear multilevel analyses. The probability of the implementation strategy being cost-effective gradually increased with an increasing willingness-to-pay, to 0.84 at a ceiling ratio of €590,000 per person with hand eczema prevented (societal perspective). The implementation strategy appeared to be not cost-effective in comparison with the control group (societal perspective), nor was it cost-beneficial to the employer. However, this study had some methodological problems which should be taken into account when interpreting the results. Key words: economic evaluation; implementation; hand eczema; healthcare workers; randomized controlled trial.

Accepted Nov 12, 2015; Epub ahead of print Nov 16, 2015

Acta Derm Venereol 2015; XX: XX–XX.

Cécile R. L. Boot, Department of Public and Occupational Health, EMGO Institute for Health and Care Research, VU University Medical Center, Van der Boechorststraat 7, NL-1081 BT Amsterdam, The Netherlands. E-mail: crl.boot@vumc.nl

Healthcare workers handle many irritants in the course of their work, such as water, detergents and gloves (1, 2). These irritants can damage the skin barrier function, which may lead to hand eczema (HE) (3). In a recent study, almost half of Dutch healthcare workers reported having symptoms related to HE (4). HE can have negative occupational consequences for the involved worker in terms of increased on-the-job productivity losses (i.e. presenteeism) (5), sick leave and increased turnover rates (5, 6). These negative occupational consequences have a large economic impact on employers and society as a whole. The annual societal costs of occupational HE are estimated to be approximately 9,000 €/patient (7). Most of these costs stem from the productivity losses associated with having HE, followed by healthcare costs (7).

Considering the negative economic consequences of HE and its high prevalence among healthcare workers, prevention of HE among this population is of utmost importance. To accomplish this goal, the Netherlands Society of Occupational Medicine (NVAB) developed a guideline for the prevention of HE in an occupational setting (8). A multifaceted implementation strategy was subsequently developed to implement this guideline among healthcare workers (9). This strategy was evaluated in a randomized controlled trial; the Hands4U study (9). Recently, the effects of the multifaceted implementation strategy on HE and preventive behaviour related to HE were evaluated, showing a positive effect on preventive behaviour and a negative effect on the prevalence of HE (10, 11).

Besides investigating the effects of an intervention, it is also important to investigate its financial consequences. This will give employers insight into the financial return of an occupational health intervention, which appeared to be a key deciding factor in the decision of whether to implement an intervention (12). This is of particular importance in the healthcare sector, where decision-makers are more inclined to invest their money in improving patient care rather than in improving the health of their own workers, as they are on a tight budget (12). The employer’s financial return for an intervention can be calculated by means of a return-on-investment (ROI) analysis. Health outcomes, however, are difficult to monetize and therefore cannot be included in a ROI analysis. Moreover, not only the employer may benefit from the intervention, but also other stakeholders (e.g. health insurance companies). Therefore, it is also important to perform cost-effectiveness analyses (CEAs) from the societal perspective, in which all costs are included irrespective of who pays or benefits from them.

The aim of the present study was, therefore, to explore the cost-effectiveness and financial return of the multifaceted implementation strategy in comparison with a minimal implementation strategy. Cost-effectiveness analyses were performed from both the societal and employer’s perspective. ROI analysis was performed solely from the employer’s perspective.

MATERIALS AND METHODS (for complete details see Appendix S11)

Methods

This study was performed alongside a 2-armed randomized controlled trial with 12 months follow-up. The study population was recruited from several hospitals and nursing homes from different regions of the Netherlands. Departments were included as a whole and could participate if most workers within the department handled irritants during work. Randomization took place at the level of departments. The study protocol was approved by the medical ethics committee of the VU University Medical Center in Amsterdam (9).

Intervention group: multifaceted implementation strategy. The intervention group received the multifaceted implementation strategy. This strategy contained the following components: education on (the prevention of) HE, participatory working groups, role models, reminders (posters), and a leaflet setting out the evidence-based recommendations from the guideline “Contact Dermatitis” of the NVAB (8).

Control group. Workers in the control group received the same information leaflet as those in the intervention group. This leaflet was considered to be a minimal implementation strategy.

Data collection.Baseline questionnaires were sent to all workers in the participating departments. Workers who completed the baseline questionnaire also received questionnaires at 3, 6, 9 and 12 months follow-up.

Baseline characteristics. The following variables were assessed at baseline: age (years), sec (male/female), number of working hours per week, number of working years in present job, having patient-related tasks (yes/no), education level (low/middle/high), co-worker support, decision authority, having an atopic predisposition, and skin exposure to irritants in leisure time due to hobbies or care-related tasks outside work. A detailed description of these characteristics can be found elsewhere (11).

Effect measures. The Nordic Occupational Skin Questionnaire – 2002 (NOSQ-2002) was used to measure the 3-month self-reported prevalence of HE (13, 14).

A partly modified version of the NOSQ-2002 was used to assess compliance with the NVAB guideline (13, 14). With this aim, a sum score was created, ranging from 0 to 5, where 0 means that a participant did not comply with the NVAB guideline and 5 means that a participant fully complied with the NVAB guideline.

Costs. Healthcare, absenteeism, and presenteeism were assessed at baseline, and after 3, 6, 9 and 12 months using questionnaires.

Workers who indicated that they had (symptoms related to) HE during the previous 3 months were asked to complete a questionnaire assessing their utilization of primary healthcare (e.g. general practitioner, allied health professionals, complementary medicine), secondary healthcare (e.g. medical specialists), and both prescribed and over-the-counter medications. Use of primary and secondary healthcare services was valued using Dutch standard costs (17).

Sickness absence was measured using a slightly modified question of the PROductivity and DISease Questionnaire (PRODISQ) (19), asking participants to report their total number of sick leave days due to HE during the previous 3 months. For the societal perspective, we estimated absenteeism costs by means of the Friction Cost Approach (FCA), with a friction period of 23 weeks and an elasticity of 0.8 (20). For the employer’s perspective, the Human Capital Approach (HCA) was used, in which costs are not truncated to the friction period and no elasticity factor is applied.

A slightly modified version of the PRODISQ was used to assess productivity losses at work due to HE (i.e. presenteeism) (19). The questions from the PRODISQ measured the following: the number of days participants went to work while having HE during the previous 3 months; the quality of their work on those days measured on a 11-point scale (0: worst quality; 10: same quality as usual); and the amount of work that was performed on those days measured on a 11-point scale (0: could not do anything; 10: could do the same as usual).

To estimate intervention costs, a bottom-up micro-costing approach was used. This means that detailed data were collected regarding the quantity of resources consumed as well as their unit prices (21). Intervention costs included all costs related to the implementation of the multifaceted implementation strategy.

Statistical analyses

Analyses were performed according to the intention-to-treat principle. Descriptive statistics were used to compare baseline characteristics between the intervention and control group. Missing data were multiply imputed. Imputations were performed per study group (intervention/control) using Predictive Mean Matching and Fully Conditional Specification in IBM SPSS (v20, Chicago, IL, USA). In total, 20 complete datasets were needed to reach a loss of efficiency below 5% (22, 23). Datasets were analysed separately, as specified below. Pooled estimates were subsequently calculated using Rubin’s rules (24). Cost-effectiveness analyses and ROI analyses were performed using Stata (V12, Stata Corp, College Station, TX, USA). We considered p < 0.05 as statistically significant.

CEAs. CEAs were conducted using HE and the compliance measure as outcome measures from both the societal and employer’s perspective. Effectiveness at 12-month follow-up was analysed using linear multilevel analyses, adjusted for baseline values. A 2-level structure was used for all outcome measures (i.e. worker, department). Cost differences between study groups were calculated for total as well as disaggregated costs. Differences in costs were estimated using linear multilevel analyses. To estimate 95% confidence intervals (95% CIs) around cost differences bias-corrected (BC) bootstrapping with 5,000 replications was used. Bootstrap replications were stratified for departments, to account for the clustering of data (25). Incremental cost-effectiveness ratios (ICERs) were calculated by dividing the adjusted differences in costs by those in effects. Bootstrapped incremental cost-effect pairs (CE-pairs) were plotted on cost-effectiveness planes (CE-planes) (26). To provide a summary measure of the joint uncertainty of costs and effects, cost-effectiveness acceptability curves (CEACs) were constructed that show the probability of the intervention being cost-effective in comparison with usual care for a range of ceiling ratios (i.e. the maximum amount of money decision-makers are willing to pay per unit of effect) (27).

ROI analysis. ROI analysis was performed solely from the employer’s perspective. Within these analyses, costs were defined as intervention costs, and benefits as the difference in monetized outcome measures (i.e. absenteeism and presenteeism costs) between the control and intervention group during follow-up, with positive benefits indicating reduced spending. Three ROI-metrics were calculated: (i) net benefits (NB); (ii) benefit cost ratio (BCR); and (iii) ROI (28–30). Subsequently, the probability of the intervention resulting in a positive financial return to the employer was estimated by determining what proportion of bootstrapped ROI-estimates was positive (i.e. NB > 0, BCR > 1 and ROI > 0%) (28–30).

Sensitivity analyses. Three sensitivity analyses were performed to assess the robustness of the results. In the first sensitivity analysis (SA1), The World Health Organization Health and Work Performance Questionnaire (WHO-HPQ) was used for estimating presenteeism costs (31, 32). In the second sensitivity analysis (SA2), absenteeism costs were valued using the HCA instead of the FCA for the societal perspective. In the third sensitivity analysis (SA3), presenteeism costs were excluded.

RESULTS

Participants

Of the 1,649 participants, 773 were allocated to the control group and 876 to the intervention group (see Fig. 1). Baseline characteristics of the study population are described in Table I. There were some meaningful differences between both groups for having a high level of education (control 52%; intervention 57%), performing patient-related tasks (control 81%; intervention 69%) and having HE at baseline (control 10%; intervention 7%). Of the 1,649 participants, 995 (60%) completed the questions on HE at least 4 times (out of a total of 5), 593 (36%) completed at least 4 out of 5 cost questionnaires, and 1,153 (70%) completed the questions on the compliance measure at least 2 out of 3 times. Participants with complete data were statistically significantly older and reported statistically significantly less often that they had an atopic predisposition than participants without complete data. These variables were included in the imputation model.

Fig. 1. Participants in the trial based on the primary outcome measure.

Effectiveness and costs

At 12-month follow-up, no statistically significant differences were found in the presence of HE (–0.06; 95% CI –0.16–0.05) and in the compliance measure (0.21; 95% CI –0.24–0.66) when comparing the intervention group with the control group. The costs of the multifaceted implementation strategy were €114 per employee (see Table SI1). All costs were statistically significantly higher in the intervention group compared with the control group (Table SII1). The largest cost difference between the intervention and control group was found for presenteeism costs (€2,764; 95% CI €2,478–3,057). Moreover, total costs in the intervention group were statistically significantly higher than in the control group from both perspectives (societal: €3,427; 95% CI €2,837–4,044; and employer’s: €3,318; 95% CI €2,649–4,035) (Table SIII1).

Societal perspective: cost-effectiveness

The ICER for HE was –59,174 (Table SIII1), meaning that the mean costs of the intervention to prevent an episode of HE in one participant were €59,174. The incremental CE-pairs were mainly located in the northeast quadrant of the CE-plane (74%), meaning that the intervention was more costly and more effective compared with the control group in preventing an episode of HE. The CEAC presented in Fig. S11 shows that the probability of the multifaceted implementation strategy being cost-effective in comparison with the control group was 0.06 if societal decision-makers are not willing to invest in the prevention of an episode of HE. This probability gradually increases with increasing values of willingness-to-pay, to a maximum of 0.84 at a ceiling ratio of €590,000.

For the compliance measure, an ICER of 16,068 was found, meaning that for every 1-point improvement on the compliance measure, the intervention costs €16,068 in comparison with the control group. The incremental CE-pairs were located mainly in the northeast quadrant of the CE-plane (77%), indicating that the intervention was more costly and more effective in improving compliance compared with the control group. For the compliance measure, the probability of the strategy being cost-effectiveness was 0.06 if societal decision-makers are not willing to invest in improving compliance with the guideline “Contact Dermatitis”. This probability gradually increased with increasing values of willingness-to-pay, to a maximum of 0.82 at a ceiling ratio of €494,000.

Employer’s perspective: cost-effectiveness

The ICER for HE was –57,299 (see Table SIII1). This means that the costs of the intervention were €57,299 for every episode of HE prevented in comparison with the control group. The incremental CE-pairs were located mainly in the northeast quadrant of the CE-plane (71%), i.e. the intervention was more costly and more effective compared with the control group in preventing HE. From the employer’s perspective, the probability of the multifaceted implementation strategy being cost-effective compared with the control group was 0.09 if employers are not willing to invest in the prevention of an episode of HE. The probability of the multifaceted implementation strategy being cost-effective in comparison with the control group increases with an increasing willingness-to-pay, to a maximum of 0.84 at a ceiling ratio of €580,000.

An ICER of 15,559 was found for the compliance measure. This indicates that, compared with the control group, an improvement of one point on the compliance measure is associated with an additional cost of €15,559 to the employer. The incremental CE-pairs were located mainly in the northeast quadrant of the CE-plane (73%), suggesting that the intervention was more costly and more effective compared with the control group in improving compliance. The probability of the strategy being cost-effective was 0.09 if employers are not willing to invest in improving the compliance with the guideline “Contact Dermatitis”. This probability gradually increased with an increasing willingness-to-pay, to a maximum of 0.82 at a ceiling ratio of €480,000.

Employer’s perspective: financial return

The total intervention’s employer’s benefits, NB, BCR, and ROI were negative during follow-up, meaning that the investments were larger than the benefits (Table SIV1). The intervention’s probability of financial return was 0.12.

Sensitivity analyses

The results of SA1, SA2 and SA3 differed to some extent from those of the main analyses. In SA1, the total cost difference was no longer statistically significant from the employer’s perspective (€454; 95% CI –€452–€1,343) (Table SIII1). In addition, although the results of SA1 and SA3 pointed in the same direction as those of the main analyses, the costs were lower and the cost-effectiveness results from these analyses were therefore slightly more favourable (Tables SIII1 and SIV1). Fig. S11 shows the CEACs from the societal perspective for SA1, SA2 and SA3. For SA1 and SA3, the CEAC were more favourable, as the probability that the intervention is cost-effective in comparison with the control group increases to a maximum of 0.85 at ceiling ratios of €210,000 in SA1, and €190,000 in SA3. Furthermore, the probabilities of financial return to the employers were higher in SA1 and SA3 than for the main analysis, but still remained low, namely 0.36 and 0.34, respectively.

DISCUSSION

The multifaceted implementation strategy was not considered cost-effective from either the societal or employer’s perspective in comparison with the control group, as reasonable probabilities of cost-effectiveness were obtained only at very high willingness-to-pay values. It is highly unlikely that societal decision-makers or employers are willing to pay such large amounts of money per person prevented from having HE and/or per 1-point improvement on the compliance outcome. In addition, the employer’s investments for the strategy outweighed their benefits and the associated probability of financial return was low (i.e. 0.12). Thus, the intervention was also not cost-beneficial to the employer.

Interpretation of the findings

There may be several potential explanations for the multifaceted implementation strategy’s lack of cost-effectiveness and financial return. First, the results showed that the difference in productivity-related costs between the intervention and control groups were large. The implementation strategy might have influenced the participants’ reporting of absenteeism and presenteeism due to HE, as those participating in the education sessions were made aware of the fact that their work could lead to HE. Making the workers more aware of this relationship might lead to more reporting of absenteeism and presenteeism related to HE. Secondly, workers might label their presenteeism and absenteeism differently after the implementation strategy. In the Netherlands, there is no need for workers to indicate whether absenteeism and presenteeism is work-related, as workers receive sickness or disability benefits regardless of the nature of their complaints (33, 34). After the intervention, the workers might understand the connection between their health complaints and related absenteeism and presenteeism, and report future absenteeism and presenteeism in relation to HE. A third explanation for the findings of the present study could be related to the setting in which the trial was performed. We aimed to prevent HE among healthcare workers in the workplace. However, recent reviews on the financial return of workplace wellness programmes concluded that there is no strong evidence of cost savings due to worksite health promotion programmes (35, 36). This lack of strong evidence might indicate that cost savings are difficult to establish when implementing primary prevention strategies in the workplace. Primary prevention continues to be difficult, as there is a long delay between the change in behaviour and the actual reward (better health) (37), making it difficult for people to continue their healthy behaviour. In addition, the cost savings of primary prevention might also be delayed, making it difficult to measure cost savings due to workplace health promotion programmes after just one year.

It is noteworthy that the findings of the present study deviated slightly from those of the accompanying effectiveness paper (11); i.e. after 12 months, there was no statistically significant difference between the intervention and usual care group regarding the prevalence of HE, whereas this difference was statistically significant in favour of the usual care group in the effectiveness paper. This difference can be explained by the fact that multiple imputation was used in the present study for the handling of missing data, whereas the effectiveness paper accounted for the missing data by using logistic multilevel analyses with random coefficients (38).

Robustness of the results

Results from the sensitivity analyses resulted in slightly more favourable cost-effectiveness results than those of the main analyses due to smaller cost differences between the treatment groups. However, overall the conclusions regarding the cost-effectiveness of the multifaceted implementation strategy as well as its financial return are in line with the main analyses. Some differences in results between the sensitivity analyses and the main analysis, however, are noteworthy. When the WHO-HPQ was used to estimate presenteeism costs instead of the modified-PRODISQ, the intervention was still not considered cost-effective in comparison with the control group, but cost differences were much smaller. This difference is probably explained by the fact that the WHO-HPQ and the PRODISQ conceptualize and measure presenteeism in different ways. The WHO-HPQ measures general work performance, whereas the modified-PRODISQ measures work performance in relation to health complaints. In addition, the PRODISQ measures presenteeism by 2 questions, namely a question on the quality of work as well as its quantity, whereas the WHO-HPQ solely measures general work performance. We opted for the modified-PRODISQ instead of the WHO-HPQ, because the PRODISQ’s conceptualization of presenteeism is most in line with the study by Zhang et al. (39), namely that health problems reduce both the quantity and quality of work. Furthermore, as HE was the primary interest of the study, measuring productivity in relation to HE seemed to be the most suitable choice compared with measuring general work performance. In addition, the costs for healthcare and absenteeism were also measured in relation to HE, making it a logical choice also to measure productivity in relation to HE. On the other hand, the WHO-HPQ might be less influenced by an increased awareness of HE among study participants compared with the PRODISQ, as the WHO-HPQ measures general work performance without relating this work performance to HE.

Strengths and limitations

This is the first study evaluating the cost-effectiveness of a multifaceted implementation strategy to prevent HE as well as its financial return in comparison with usual care. A strength of this study is that the risk of contamination between study groups was minimized by performing randomization at the level of departments. In addition, by performing linear multilevel analyses in combination with bootstrapping we were able to account both for the resulting clustering of data and the skewed distribution of cost data.

A first limitation of this study is the amount of missing data. For example, only 40% of the study population completed at least 4 out of the 5 cost questionnaires. Although missing data were imputed using state-of-the-art techniques, the results are probably less reliable and valid than when a complete dataset would have been obtained. Therefore, the results of the present study must be interpreted with caution. A second limitation is that all outcome measures were based on self-reports. As the multifaceted implementation strategy was designed, among others, to increase awareness of HE and to explain how work could lead to HE, this might have influenced the reporting of the study participants, as argued above. Therefore, we cannot be sure whether our results are a reflection of the intervention or merely a product of increased awareness. A third limitation of the study was that medical costs were assessed only for workers who reported having HE (symptoms). However, as the multifaceted implementation strategy was aimed at primary prevention, it is likely that participants in the intervention group without HE also purchased, for instance, moisturizers, in an effort to prevent HE. As a result, the medical costs in the intervention group might have been slightly underestimated.

Implications for practice and/or research

First, there is a need for more economic evaluations in the field of HE (prevention). Only one earlier study investigated the economic consequences of a HE intervention, in which the patients received integrated care (40). Giving organizations insight into financial implications of an intervention is considered to be important for the decision-making process about whether to invest in an intervention (12). Not performing an economic evaluation in our study would have led to a more positive recommendation regarding further implementation of the multifaceted implementation strategy, as our effect evaluation showed positive results regarding compliance with the NVAB guideline (10, 11). After the economic evaluation, we cannot recommend widespread implementation of the strategy without performing more research to investigate the effects. Therefore, we would recommend studies on HE to incorporate an economic evaluation in the study design.

Conclusion

This economic evaluation showed that a multifaceted implementation strategy for the prevention of HE was not cost-effective in comparison with care as usual, from both the societal and employer’s perspective. In addition, it did not result in a positive financial return to the employer.

ACKNOWLEDGEMENTS

This study is supported by a grant from The Netherlands Organisation for Health Research and Development (ZONMW).

The authors declare no conflict of interest.

1http://www.medicaljournals.se/acta/content/?doi=10.2340/00015555-2287

REFERENCES