Informal caregivers are unpaid helpers, e.g. family, friends, and neighbours, who assist adults with disabilities (9). Informal caregivers provide 4 times as much care as formal caregivers to individuals requiring assistance (10). Perceived caregiving burden is negatively associated with care-recipients’ functional independence and caregivers’ quality of life (11). The caregiving role can entail high levels of stress that may lead to burnout (12, 13). The replacement value of unpaid contributions by informal caregivers in the USA has been estimated at $450 billion annually (14). In light of all of the assistance that informal caregivers provide, the potential for burnout is a serious challenge to many healthcare systems. It has been asserted that AT reduces the need for caregiver assistance, thus decreasing the likelihood of caregiver burnout (8).

Although many measures of caregiver burden exist (15), most do not assess the outcomes that are attributable to AT provision. Caregiver burden measures typically assess relatively global problems, such as financial stress (16), social distress (e.g. problems interacting with family) (17), emotional burden (e.g. resentment or feelings of embarrassment about the care receiver’s behaviour) (18), health (e.g. lack of sleep) (19), and time demands (i.e. loss of free time) (20). At the same time, most of these measures include relatively few items pertaining to day-to-day demands on caregivers (e.g. the physical strain of providing care (21), the need for providing supervision (18), and concerns about the potential for care-recipient accidents or injuries (22), which AT is intended to address. Thus, a measure targeting the impact of AT usage on caregiver burden would fill a measurement gap that could enhance the cost-benefit justification for AT recommendations.

Demers et al. (23) created a conceptual framework to describe the effects of AT on users’ informal caregivers. The framework was created by means of an iterative, 4-step development process (23). First, important caregiver outcome variables were identified based on semi-structured interviews with caregivers and on a review of published empirical studies. Secondly, the variables were analyzed by means of a process of conceptual mapping and primary components were identified. Thirdly, these components were anchored using a general model of caregiving: the Stress Process Model (24). Fourthly, rehabilitation experts were consulted in order to refine the conceptual framework. According to the framework, the impact of AT varies with the type of device and the amount and manner of its use. The use of AT moderates the relationship among the characteristics of primary stressors (e.g. areas of assistance, forms of assistance, safety, and effort), the features of secondary stressors (e.g. role overload and the elective use of time), and broader caregiver outcomes (e.g. health and social participation).

Based on this framework, the Caregiver Assistive Technology Outcome Measure (CATOM) was created to capture the impact of AT provision on the user’s caregiver, i.e. to discriminate among caregivers who experience different amounts of burden and to evaluate the contributions of AT usage to these differences.

The CATOM is an 18-item tool that has 3 parts. In Part 1, caregivers identify all of the activities with which they help. They then identify 1 activity that is most demanding (physically and/or emotionally) to the caregiver, which is amenable to AT intervention. In Part 2, caregivers use a Likert scale to rate the burden they currently experience with 14 aspects of the selected activity. In Part 3, caregivers rate 4 aspects of the overall caregiving burden they experience. The CATOM can be used before and after provision of AT to capture its impact on the burden experienced with that specific activity.

The overall purpose of this paper is to report on the development and preliminary evaluation of the CATOM. The description reported here is based on the process outlined by Streiner & Norman (25). This multiple-phase project had 3 main objectives:

• to select items and appropriate response scales for inclusion in Parts 2 and 3 of the measure, and to create a preliminary version of the tool;

• to revise the measure and to create and revise an administration manual; and

• to assess the internal consistency, test-rest reliability, standard error of measurement (SEM), minimal detectable change (MDC), convergent validity, and utility of the measure and to document the presence of any floor and ceiling effects, and the distribution of its scores.

METHODS

Preliminary measure development

We reviewed existing measures to identify potential items that fit our conceptual framework (i.e. that assessed activity-specific primary stressors, secondary stressors, and caregiver outcomes (23)). We amalgamated/adapted items from these measures and developed new items to create a preliminary version of the tool.

Cognitive interviewing

Cognitive interviewing was used to reduce item misinterpretation (26). Participants were asked to “think aloud” while completing the measure. Examples of probing questions that were asked following the presentation of some items include:

• What are you thinking about when answering this question?

• Do you think this question is useful to understand a caregiver’s situation?

• What does the word “equipment” mean to you? Are you comfortable with the use of this word?

The interviews were recorded and transcribed verbatim. Content analysis identified common caregiver problems and areas of concern. Following the cognitive interviews, we also discussed the relevance of all of the items in the preliminary version, as the scale included multiple questions to assess the domains of the conceptual framework. Items were removed if they were deemed not relevant to caregivers or redundant. Experienced AT practitioners reviewed pre-final versions of the measure and administration manual. The study was approved by the local research center and university ethics board.

Psychometric testing

The CATOM was pilot tested as part of a multi-site, delayed intervention experimental trial that included sites in Montréal, Quebec and Vancouver, British Columbia (27). Forty-four caregiver-assistance user dyads were randomized to receive either an immediate or a 6-week delayed AT intervention.

Inclusion criteria for assistance users were: age 65 years or older, with a physical disability, and receiving 2 or more hours of informal care per week. Inclusion criterion for caregivers was: age 19 years or older. Assistance users and caregivers were excluded if they had cognitive impairments that prevented them from completing the study measures or providing informed consent. The intervention focused on an activity that was perceived as important by the assistance user and his or her caregiver. The first part of the CATOM was used to help identify the activity to be targeted for the intervention. It was then used to measure the activity-specific caregiver burden associated with this activity before and after the intervention.

Assistance user and caregiver demographic information was collected that included age, sex, diagnoses, and living situation. Additional data were collected using 5 measures. Assistance-users’ functional status was measured using the Functional Autonomy Measurement System (FAMS), a 29-item tool that assesses the performance of basic and instrumental activities of daily living, communication, and cognition (28). Total scores vary from 0 (complete independence) to –87 (complete dependence). Two-week test-retest reliability for the measure, calculated using an intraclass correlation coefficient (ICC) is 0.95. Caregiver health status was measured using the Visual Analog Scale from the 5-item Euro-Qol (EQ-5D) (where 0 = worst imaginable, 100 = best imaginable), for which the 1–4 week test-retest reliability is high (ICC of 0.90) (29).

The Individually Prioritized Problem Assessment (IPPA) measured assistance users’ perceived difficulty performing specific activities on a 5-point scale (where 1 = no difficulty, 5 = too difficult to perform) (30, 31). The IPPA compares favourably to the Sickness Impact Profile and the European Quality of Life (EuroQOL) scale in detecting change following AT provision (32). Assistance users’ accomplishment performing the activity selected to be the target of the intervention was measured using the 10-point response scale from the Life Habits scale (Life-H), where 0 = ”not performed” and 9 = ”performed with no difficulty and with no assistance” (33). The Life-H subscale having items that are most similar to activities targeted in this study, i.e. personal care, has high test-retest reliability (ICC = 0.97) (33). Activity-specific and overall caregiver burden were measured using Parts 2 and 3 of the CATOM, respectively. The administration time of the CATOM was recorded as a measure of utility. The study was approved by the local research center and university ethics boards.

Analysis

To describe the samples, means and standard deviations were calculated for continuous variables and frequencies and percentages were calculated for nominal variables. The mean and standard deviation of completion times in minutes were calculated as indicators of utility.

The internal consistency of the CATOM was assessed by calculating Cronbach’s alpha separately for Parts 2 and 3 of the scale. The delayed intervention design permitted us to look at the 6-week, test-retest reliability of scores of participants waiting to receive the intervention, excluding caregivers who had a significant change in health status or who provided care to someone whose health status changed significantly. To assess reliability for participants in the delayed intervention group, ICCs (2,1), SEM, and MDC (using the formula MDC95 = SEM × √2 × 1.96) were calculated. A Cronbach’s alpha of ≥ 0.7 < 0.9 is considered to be moderate, and an ICC of > 0.75 is considered to be excellent (34).

The distributions of all pre-intervention scores were examined for possible floor and ceiling effects, and the 1-sample Kolmogorov-Smirnov test was used to assess the normality of distributions of Parts 2 and 3 scores. A floor or ceiling effect is indicated if 15% of respondents have a minimal or maximal score, respectively (35).

To examine convergent validity (i.e. that a measures varies as hypothesized with related constructs), data from the entire sample were used to calculate correlations among users’ perceived task difficulty (measured using the IPPA) and accomplishment performing the task (measured using the Life-H) and caregivers’ perceived burden as measured by Parts 2 and 3 scores. The expectation was that the perceived difficulty and accomplishment scores would be moderately correlated with Part 2 scores, but would not be significantly correlated with Part 3 scores because of the targeted nature of the intervention. We anticipated that Part 2 of the measure, which captures activity-specific burden, would be moderately correlated with Part 3, which looks at overall burden. For the purpose of this analysis moderate correlations were considered to be > 0.3 and < 0.6 (36).

RESULTS

Preliminary item development

We reviewed 241 items from 12 existing measures in order to devise the original set of items. These measures included the Caregiver Burden Interview (37), Montgomery burden inventories (20), Caregiver Strain Index (21), Caregiver Burden Inventory (18), Caregiver Appraisal Measure (19), Caregiver Reaction Assessment (17), Strain Scale (22), Filial Anxiety Scale (20), Financial Impact Scale (16), Cost of Care Index (38), Impact on Social Activities (39), and Family Impact of Assistive Technology Scale (40). A total of 53 items informed the development of the preliminary version of the measure. Included were 4 items from the Caregiver Burden Interview (37), 7 from the Montgomery burden inventories (20), 3 from the Caregiver Strain Index (21), 7 from the Caregiver Burden Inventory (18), 6 from the Caregiver Appraisal Measure (19), 3 from the Caregiver Reaction Assessment (17), 6 from the Strain Scale (22), 1 from the Filial Anxiety Scale (20), 2 from the Financial Impact Scale (16), 8 from the Cost of Care Index (38), 3 from the Impact on Social Activities (39), and 3 from the Family Impact of Assistive Technology Scale (40).

Based on the clinical and research experiences of our team, and the presence of many duplicate items that tapped into the same construct, the initial pool of items was reduced to 29 items that reflected all of the domains of our conceptual framework regarding the impacts of AT on users’ informal caregivers (23). This preliminary version included 9 items that assessed primary stressors, 11 that addressed secondary stressors, and 9 that evaluated broader caregiver outcomes.

A response scale was chosen that decreases the likelihood of eliciting a social desirability bias (41). Using a 2-step approach, respondents are first asked a “yes/no” question about a specific burden-related concept, e.g. “Do you provide physical assistance to your (spouse or the person your provide care for) when (he/she) takes a bath?” If the answer is “no”, a score of 5 is assigned. If the answer is “yes”, participants are asked about the frequency with which they provide that particular assistance. Items are rated on an ordinal scale with 4 levels of frequency (1 = nearly always, 2 = frequently, 3 = sometimes, 4 = rarely).

Optionally, if baseline data are unavailable or response shift is of interest, retrospective responses can also be elicited. In this mode, respondents are asked to describe the perceived degree of change over a specified period. This judgment is elicited for items comprising Parts 2 and 3 using a 5-point ordinal scale (1 = a lot more, 2 = a little more, 3 = the same, 4 = a little less, and 5 = a lot less). An illustrative item is the following one: “Since your spouse received his walker, do you need to provide more physical assistance to him when he takes his bath, less physical assistance, or is it the same as before?” If the answer is “less,” the evaluator asks, “Is that a lot less or a little less?” These retrospective responses are identified as constituting Parts 2b and 3b, respectively.

Cognitive interviewing

A non-probabilistic sample of 6 informal caregivers of older adults with physical disabilities (3 with multiple sclerosis and 3 with stroke) participated in cognitive interviews using the preliminary version of the measure. The 2 male and 4 female participants had a mean age of 64 years (standard deviation (SD) ). Five were spouses and 1 was a daughter. They had been caregivers for a mean of 7 years (SD 3) and were providing 22 h of care per week (SD 19).

Based on the cognitive interviews 5 items were re-worded to achieve greater clarity. Based on further discussion with participants 11 items were removed because participants felt they were redundant (n = 9) or not relevant (n = 2).

An initial manual of administration was drafted to include a description of the CATOM’s characteristics, its theoretical foundation, content areas, rating system, and detailed instructions for administration. A pre-final version of the CATOM and its administration manual were then reviewed by 5 occupational therapists with AT expertise. The 4 female and 1 male occupational therapists had a mean age of 41 years (SD 10) and a mean of 16 years of practice experience (SD 9). Collectively, their feedback suggested minor changes to enhance the clarity of both the CATOM and its manual, all of which were incorporated prior to launching the pilot intervention study.

Table I describes the 18 retained items in the final version of the CATOM. Part 2 includes 14 questions to measure the frequency of caregiver burden associated with specified activities. Part 3 includes 4 items to measure global outcomes associated with care provision, i.e. how informal caregiving affects caregivers’ psychological health, participation, and social relationships.

Psychometric testing

Descriptive statistics. Descriptive statistics for demographic and outcome variables are presented in Table II. Six-week, test-retest data were available for 15 caregivers in the delayed intervention group. Two caregivers were excluded from the test-retest analysis for health-related reasons. For both samples, assistance users’ mean age was 83 years, and all had difficulty performing some functional tasks. Caregivers in the combined samples had a mean age of 71 years, and they provided 11–15 h of care per week.

Internal consistency. The Cronbach’s alpha was 0.80 for Part 2 and 0.76 for Part 3. The Cronbach’s alpha for the retrospective change scores was 0.83 for Part 2b and 0.66 for Part 3b.

Reliability, SEM and MDC. The resulting ICCs (2,1) for Parts 2 and 3 scores were 0.89 (95% CI 0.64–0.96) and 0.86 (95% CI 0.60–0.95), respectively. The SEM and MDC for Parts 2 and 3 scores were 3.02 and 8.35, and 1.35 and 3.73, respectively.

Floor and ceiling effects and distributions. With respect to the baseline distributions for Part 2, no participant had the lowest score and only 1 (2.4%) had the highest score. For Part 3, no participant had the lowest score, but 11 (26.8%) had the highest score. For Parts 2 and 3, p-values of the 1-sample Kolmogorov-Smirnov test were 0.62 and 0.26, respectively, indicating that the null hypothesis (that the distributions are normal) need not be rejected.

Convergent validity. As expected, Parts 2 and 3 were only moderately correlated (r = 0.56, p < 0.01). As hypothesized, there was a moderate correlation between activity-specific caregiver burden and accomplishment (r = 0.53, p < 0.01) and between activity-specific caregiver burden and difficulty performing the dyad-identified activity (r = –0.31, p < 0.05). Part 3 scores were not significantly correlated with either accomplishment or difficulty.

Utility. The CATOM took a mean of 13 min to administer (SD 7).

DISCUSSION

The CATOM is the first instrument designed specifically to assess the effects of AT provision on users’ informal caregivers. Given that AT is frequently provided to decrease the burden that informal caregivers experience (2, 8), and caregiver burden is a serious concern for the healthcare system (39, 42), the new instrument captures a decidedly important construct. This is congruent with the Consortium for Assistive Technology Outcomes Research taxonomy of AT outcomes (43), which identifies the effects of AT on caregivers as critical aspects of the social significance of these devices.

The CATOM underwent a rigorous and iterative process of development. Its content validity is supported by 4 factors: (i) it is anchored by an empirically derived conceptual framework as a basis for the measure (23); (ii) the initial pool of items was culled from existing measures of caregiver burden; (iii) a preliminary version of CATOM items was evaluated by informal caregivers, who provided feedback as part of a cognitive interviewing process; (iv) AT practitioners also provided feedback on a pre-final version to enhance the clarity of items and the instructions for the measure’s administration.

The preliminary psychometric properties of the instrument appear to be promising. Test-retest reliability data suggest that scores are relatively stable 6 weeks after administration (i.e. ≥ 0.86). ICCs above 0.75 have been described as excellent (44), although the confidence intervals are somewhat wide, perhaps due to the small number of participants in the reliability sample. The ceiling effect noted for Part 3 of the measure also likely reflects the relatively small study sample since assistance users in the intervention group did not require extensive help from caregivers. Furthermore, moderate correlations among Part 2 CATOM scores, assistance user difficulty, and assistance user accomplishment lend support for the convergent validity of the measure.

Application of the CATOM will fill an important gap in our understanding of the outcomes of AT provision, which is frequently justified because it is expected to result in an increase in users’ functional independence, i.e. reduced dependence on assistance from others (8). However, most outcome measures fail to address whether these devices are beneficial to caregivers. By administering the CATOM concurrently with established measures of functional independence, separate estimates become available of the extent to which users’ day-to-day independence is affected by the provision of AT. This information is critical to provide a complete, complementary picture of the outcomes of AT provision on users and their caregivers.

AT is provided in the context of many different models of service delivery. The psychometric study embodied an innovative model in which caregivers, prospective users, and service providers actively collaborate in the process of AT provision (27). That model contrasts with the prevailing one in which caregivers have less, if any, direct involvement in the process. Other models differ in terms of how devices are paid for, the credentialing of the providers, and the settings in which services are delivered. The CATOM can be readily adapted to assessing caregiver impacts in connection with any of these models, thus enabling their relative effectiveness to be compared more broadly.

With the CATOM, a caregiver’s burden can be understood from 2 viewpoints: (i) the specific areas of care that are entailed (e.g. pertaining to the recipient’s mobility or personal hygiene), and (ii) the aggregate care being provided, i.e. the individual’s overall caregiving burden. AT provision may reduce the caregiving burden associated with one or more specific user activities, but fail to reduce the caregiver’s overall burden. One way that this can occur is by caregivers simply substituting new areas of care for the ones addressed by the use of AT. The CATOM is designed expressly to clarify how changes in the burden associated with specific user activities relate to changes in overall care burden. This enables studies to be undertaken regarding how caregiving patterns are affected by factors such as caregivers’ health status, age, or gender, or by the types of disability or extent of disability that characterize the recipients of care.

The CATOM highlights the importance of including the perspectives of the device user and his or her caregiver in the AT prescription process. By gathering information about the impact of AT on users’ caregivers the measure emphasizes how funding decisions and the model of service delivery ought to consider not just the impact on the care recipient, but also the effect on the caregiver. Reducing caregivers’ burden by means of recipients’ use of AT (e.g. stair-climbing devices) may have both short- and long-term effects on mitigating caregivers’ physical and psychological stress and preventing them from developing disabling conditions that in turn may reduce their capability for rendering care. In some cases, the type of device prescribed may have a greater direct impact on the caregiver, rather than the AT user. For example, although a mechanical and an electric lift both facilitate transfers; the former is far more physically demanding on the care provider. In light of concerns about caregiver burnout (11, 12, 45), the information provided by the CATOM is critical for informing both policy and practice.

Future directions

Additional research is needed for further validation of the CATOM. This could include studies with other populations, e.g. assistance users with more profound functional impairments. Given that the measure was designed for a study to assess the effects of single interventions that target a specific problematic activity, additional work is necessary to develop and test versions of the tool for evaluating multiple problematic activities. This would enhance the ecological validity of the measure for application to community care settings where multiple problematic activities are frequently addressed simultaneously (4).

Strengths and limitations

Several strengths and limitations of the CATOM development process should be noted. Caregiver and practitioner involvement throughout the process provide some assurance of content validity and likely foster increased acceptance of the measure by individuals in both roles. Furthermore, the Stress Process Model provided a strong conceptual basis for the measure. The relatively small sample size created broad confidence intervals that would have been narrowed with a larger sample. Collecting additional activity-specific caregiver information would have facilitated supplemental validity-related hypothesis testing. Finally, given the age of the AT user sample, further testing is warranted to see if the instrument has similar properties among caregivers of younger AT users.

Conclusion

The CATOM is a promising new tool that assesses the effect of AT provision on users’ caregivers. The measure was developed by means of a multi-step, iterative process that involved: (i) item selection to create a preliminary version of the measure, (ii) cognitive interviewing to identify ambiguous items, and (iii) preliminary psychometric testing. The CATOM demonstrates excellent 6-week test-retest reliability and varies in expected directions with other measures. This study lays the groundwork for additional testing of the measure and its further development.

ACKNOWLEDGMENTS

This study was funded by the National Institute on Disability and Rehabilitation Research (grant #H133A060062). Personal financial support was provided for Dr Mortenson by a Banting post-doctoral fellowship.

REFERENCES