Taran Paulsen, MSc, Margreth Grotle, PhD, Andrew Garratt, PhD and Ingvild Kjeken, PhD

From the National Resource Center for Rehabilitation in Rheumathology, Diakonhjemmets Hospital, Oslo, Norway

OBJECTIVE: To develop and assess the reliability and validity of a patient-reported measure of hand activity performance for persons with rheumatoid arthritis (MAP-Hand).

METHODS: The development of the measure included a literature review, semi-structured interviews with 60 patients with rheumatoid arthritis, and testing of face and content validity by video-observation and classification of the initial items according to standardized methods. Further testing followed 2 surveys of 176 and 134 patients with rheumatoid arthritis and included Rasch analysis and comparing MAP-Hand scores with other measures of symptoms and functional ability. Test-retest reliability was assessed in 35 stable patients with rheumatoid arthritis.

RESULTS: Most of the initial 31 items had good face and content validity. Following Rasch analysis the measure was reduced to 18 items, which had good evidence for unidimensionality, a broad range of item difficulty, good person separation and ordered thresholds in a 4-point scale. The test-retest intraclass correlation coefficient was 0.94 (95% confidence interval 0.89, 0.97), indicating high reliability. The results of validity testing generally followed the a priori hypotheses, with MAP-Hand scores having moderate to high correlations with scores for the other measures.

CONCLUSION: The MAP-Hand is an 18-item patient-reported measure of hand activity performance, which showed good evidence for reliability and validity in patients with rheumatoid arthritis.

Key words: rheumatoid arthritis; hand function; activity performance; patient-reported measure; Canadian Model of Occupational Performance; International Classification of Functioning; Disability and Health; psychometric testing.

J Rehabil Med 2010; 42: 636–644

Correspondence address: Ingvild Kjeken, National Resource Center for Rehabilitation in Rheumatology, Diakonhjemmet Hospital, PO Box 23, Vinderen, NO-0319 Oslo. E-mail: ingvild.kjeken@diakonsyk.no

Submitted August 28, 2009; accepted April 27, 2010

INTRODUCTION

Rheumatoid arthritis (RA) is a chronic systemic disease, characterized by the presence of synovial inflammation and destruction of bone and cartilage (1). The joints most commonly involved are the hand joints, including the metacarpophalangeal (MCP), proximal interphalangeal (PIP) and wrist joints, causing decreased grip strength and joint mobility, and pain and hand deformities (2). This, in turn, has functional consequences in terms of limited ability to perform daily activities (3, 4).

To properly assess and treat hand function, health professionals need valid, reliable and responsive measures that are feasible for use in clinical practice (5). The structured review of the literature that informed the current research found 5 patient-reported measures of hand function for patients with RA (6–10). Appendix II summarizes the source of content, validity, reliability and responsiveness. Typically the measures comprise pre-defined multi-item scales generated by health professionals with little item consistency between the scales. However, during the past 2 decades it has been increasingly recognized that patients are the most valid sources to report on functional limitations and hence should inform the development of outcome measures (11).

Appendix II shows that existing measures do not follow conceptual models or recognized classification systems, which has implications for content validity. The development of the hand function measure reported below was based on the client-centred Canadian Model of Occupational Performance (CMOP) to include the patient perspective (12), and the International Classification of Function, Disability and Health (ICF) as a widely accepted terminology of health and functioning (13). Based on the CMOP’s definition of occupation, the ICF’s definition of activity performance and the functional unit of the hand, hand function in activity performance is understood as ”performance of a task or action that to a great extent involves use of the anatomical structures and physiological functions of the fingers, thumb, wrist and forearm. Hand function in activity performance is part of a person’s total activity performance and gives meaning to the individual looking after one’s self, enjoying life, and contributing to the social and economic fabric of his or her community”.

With the exception of the ABILHAND (7, 14), existing hand function measures have not been evaluated using item response theory (IRT). Rasch analysis is one form of IRT that has been increasingly applied in the field of patient-reported outcomes in order to assess the unidimensionality (the extent to which items measure a single construct, e.g. hand function), item difficulty (the relative difficulty of the items when compared with one another) and person separation (the extent to which items distinguish between distinct levels of functioning) of a measure (15). This study was designed to develop a patient-reported measure of hand function in activity performance (MAP-Hand) for persons with RA that is appropriate for clinical practice and research. Following the application of Rasch analysis, the MAP-Hand was assessed for test-retest reliability and construct validity.

PATIENTS AND METHODS

There were several stages in the development and testing of the hand instrument, which included different patient samples (Fig. 1).

Fig. 1. Stages of the study. TCOP: Taxonomic Code of Occupational Performance; COPM: Canadian Occupational Performance Measure.

MAP-Hand development

Literature review. Structured searches of the PubMed, EMBASE, and Cochrane Library databases informed both the decision to develop a new measure and the methods of development and evaluation upon which the new measure was based. The findings shown in Appendix II are based on searches up until June 2008 for all years and languages, using keywords relevant to patient-reported outcome measures. Measures that did not include a scale specific to hand function in activity performance were excluded. The names of identified measures were used in further searches. Information extracted from the articles followed published reviews and recommended criteria for the selection of instruments in clinical trials (16).

Two common features of the 5 existing measures identified in the review were a lack of theoretical basis for the development and an item generation process based primarily on existing scales and the views of health professionals. Furthermore, only one has been evaluated using item response theory. Therefore, it was considered appropriate to develop a new patient-reported measure of hand activity performance for persons with RA that followed recommended methods for development and testing with patients as the primary source in the development of items.

Item generation – semi-structured interviews. To ensure that the items were relevant for patients with RA, semi-structured interviews were conducted by TP and IK with 60 patients from the inpatient and outpatient clinics of the rheumatology department at Diakonhjemmet Hospital in Oslo. The patients had been diagnosed with RA by physicians at the hospital, had problems performing daily activities due to limited hand function, and could communicate well in Norwegian. Exclusion criteria were reduced hand function due to earlier injuries or other diseases, hand surgery during the last 3 months, or having a psychiatric or cognitive disorder.

The Canadian Occupational Performance Measure (COPM) is conceptually grounded on the CMOP, and is a patient-specific measure of activity performance (17). According to the COPM-manual, the administration of the COPM is a stepwise procedure, starting with an interview where the patients define their occupational performance problems within the following 9 areas: personal care, functional mobility, community management, paid/unpaid work, household management, play/school, quiet recreation, active recreation, and socialization. Each problem is written down under one of the activity categories in the COPM test form. Patients are then asked how important it is to be able to perform the activities on a scale from 1 (not important at all) to 10 (extremely important). Finally, patients rate up to 5 of the most important activities for Performance and Satisfaction with performance on scales from 1 (not able to do it, not satisfied at all) to 10 (able to do extremely well, extremely satisfied).

In this study, the following modifications of the COPM interview guide were made: patients were asked to describe which occupational problems they experienced due to arthritis in the joints of the hands, rather than their occupational problems in general. Before starting the scoring procedures, patients were also asked if they wanted to add any other activity problems, thereby ensuring that the whole domain of activity performance important to patients with RA was covered. Finally, scoring of Performance and Satisfaction were substituted by a rating of how often the patient performed each of the 5 prioritized activities on a 5-point scale of daily, 3–4 times weekly, weekly, monthly and never. The activities prioritized by patients were analysed by TP and IK according to the 9 activity categories of the COPM. Activities described by more than 20% of patients were included in the initial version of the MAP-Hand.

MAP-Hand testing

Face and content validity – videotapes and classification of items. Face and content validity were assessed by analysing videotapes of handgrips used by 3 persons with different level of functional ability (1 healthy hospital staff member, 1 patient with moderate functional limitations and 1 patient with severe limitations) when performing the activities included in the initial MAP-Hand item pool. It was hypothesized that an item pool with activities that require a wide variety of hand grips would ensure that the final measure covered a broad spectrum of activity performance of the hand, thereby protecting against floor- and ceiling-effects. Functional ability of the test persons were rated according to grip strength, measured with the Grippit instrument (18), and the Arthritis Impact Measurement Scales 2 (AIMS2) subscale for hand and finger function (10). Sollerman’s classification of the 8 main handgrips into which a normal grip pattern can be categorized (19) was used to analyse the items. A modification sign was applied when a handgrip did not fully meet the criteria for a specific grip. Also, 2 new codes were added: “twisting of the forearm” and “other”, the last code covering hand grips that did not fall into any of the other categories. The recorded activities were individually coded by 2 physiotherapists (ERH and TP) and 2 occupational therapists (IK and MH). Results were compared and disagreement resolved by consensus.

Face and content validity of the items were also assessed with the Taxonomic Code for Occupational Performance (TCOP) (20). TCOP divides occupational performance into a 7-level hierarchical framework where each higher level subsumes all the characteristics of those below it and each higher level has one more dimension of complexity than that below it. The 9 activity categories of the COPM were used to assess whether the items covered all dimensions of activity performance. Coding of the items according to TCOP and COPM was done individually by TP and IK. Results were compared and uncertainties resolved by consensus.

Construct validity (Rasch analyses) and test-retest reliability. The initial version of the MAP-Hand comprised 31 items with a 5-point scale of: no difficulty, slight difficulty, moderate difficulty, great difficulty and not able to do, in addition to a not applicable option. Scaling properties were assessed following a survey of 176 patients, of which 68% were women to match the RA population in general (21). Inpatient participants answered the questionnaires during the hospital stay (74 patients), whereas a postal survey was used for participants enrolled in the Oslo Rheumatoid Arthritis Register (ORAR) (22). The latter included 171 patients; 102 (60%) responded and met the inclusion criteria. All participants were asked to complete and return the MAP-Hand in addition to sociodemographic information and health status measurements (Table I). Rasch analysis was used for item reduction and preliminary evidence of construct validity of the initial MAP-Hand.

Physical functional ability was assessed using the American College of Rheumatology 1991 revised criteria for the classification of functional status in RA (ACR functional classification) (23). The Modified Health Assessment Questionnaire (MHAQ) was used to measure activity limitations (24). Disease parameters included patient’s global assessment of disease activity, pain and fatigue on a 100-mm visual analogue scale (VAS).

The revised MAP-Hand and the same additional questions were included in a second postal survey of 176 patients, of whom 134 (76%) responded. The MAP-Hand score is the mean of the final 18 items, with allowance for up to 4 missing items. Test-retest reliability was assessed in a random sample of 50 patients, who were asked to complete a second questionnaire within 1–2 weeks. Finally, 5 persons with RA were asked to comment on the final version of the measure, including the wording and ordering of items.

The regional ethics committee for medical research in Norway approved all parts of the study protocol. Written informed consent to participate was obtained.

Statistical analysis

Respondents and non-respondents were compared in relation to clinical and socio-demographic variables. All items were analysed in frequency tables. Items with a third or more missing data were considered for removal from the questionnaire, and items in which a third or more patients replied “did not apply to me” were not included in further analyses.

Rasch analysis is a probabilistic model that tests the extent to which the observed pattern of person and item responses fits the pattern expected by the model (15, 25). Unidimensionality was assessed together with overall model fit, individual person fit and item fit, the threshold order of ordinal scale, item difficulty, differential item functioning, and person separation. The likelihood-ratio test was used to assess whether using the rating scale model or the unrestricted model was appropriate. The overall fit of the model was given by a Bonferroni-adjusted χ2 item-trait interaction statistic. A non-significant probability value indicated that there was no substantial deviation from the model, and that the hierarchical ordering of the items was consistent across all levels of the underlying trait. The Person Separation Index, which is equivalent to Cronbach’s alpha, indicates the power of a scale to discriminate amongst respondents with different levels of the trait being measured. Values of 0.8 and 0.9 indicated that the scale could statistically discriminate between at least 2 and 3 groups, respectively. The individual person fit and item fit were assessed by inspecting the means (SD: standard deviation) of the separate fit residuals. Mean values of approximately 0 (SD = 1) were expected if the items and persons fitted the model. Misfitting items were identified by fit residuals of generally greater than plus or minus 2.5 or a significant χ2 probability value (15). For each item the ordered set of response thresholds and the category probability curves were assessed. Disordered thresholds were corrected by rescoring the items.

To test for bias across groups of respondents, DIF was assessed in relation to age (≤ 48, 49–60, ≥ 60 years) and gender. With uniform DIF the difference between groups in the probability of affirming an item (or category) between groups is constant across the scale (analysis of variance (ANOVA) main effect). With non-uniform DIF the difference between groups varies across the scale (ANOVA interaction effect).

Principal components analysis (PCA) was applied to 2 subsets of the residuals to assess unidimensionality. The absence of any meaningful pattern in the residuals supports the assumption of local independence and unidimensionality of the scale. The component loadings of 2 subsets of items were compared and a paired t-test was used to determine if the associated person estimates were significantly different from that for all items. If the person estimate was different between the subset and the full item scale, this would indicate a breach of the assumption of local independence and unidimensionality.

Test-retest reliability was assessed by the intraclass correlation coefficient (ICC). An ICC of 0.80 or more was considered to be good test-retest reliability.

The relationship between the MAP-Hand scores and those for the other health measurements was assessed using data from the second postal survey. In tests of construct validity, high correlations were expected between the MAP-Hand and the hand and finger function subscale and the arm subscale of AIMS2; the correlation with the hand and finger subscale was expected to exceed that for the arm subscale. A moderate correlation was hypothesized between the MAP-Hand and the fatigue scale, and high correlations with the joint pain and global disease assessment scales. Weak, moderate and high correlations were defined as r < 0.3, 0.3 < r < 0.6, and r > 0.60, respectively (26). Analyses were undertaken in SPSS (version 14) and RUMM2020.

RESULTS

MAP-Hand development

Item generation by semi-structured interviews. The patient characteristics are shown in Table I. The patient interviews gave 259 different activities, 31 of which met the inclusion criteria of being described by at least 20% of the informants. These 31 activities were described a total of 653 times and prioritized 199 times. Most activities were in the areas of household management (36%) and personal care (25%). Seven of the 10 most highly rated activities in terms of their importance related to personal care. All 31 activities were described and rated as important by both men and women. Except for “carrying heavy objects”, all activities were performed at least weekly by 50% or more of the patients who rated the activity as important.

MAP-Hand testing

Face and content validity – videotapes and classification of items. The videotaped participants comprised: a healthy 47-year-old woman with a maximum dominant hand grip force of 292 Newton (103% of normative values) and AIMS2 score of 0; a 53-year-old woman with RA with moderate functional limitations (maximum grip force of 164 Newton (66% of normative values), and AIMS2 score of 2.5); and a 58-year-old man with RA with severe functional limitations, (a maximum grip force of 122 Newton (25% of normative values) and AIMS2 score of 3). The video analysis revealed that all but the activity of typing on a computer could be coded by the modified version of Sollerman’s classification of handgrips. Most frequently used was the 5-finger pinch grip for 19 activities, while the extension grip was used the least for 4 activities. The RA patients had more use of modified and bimanual grips, and more loading of the proximal, stronger parts of the hands compared with the healthy individual. For the patient with severe functional limitations, some activities could not be accomplished including “opening jam jars”. Of the 31 activities, all 3 individuals performed 21 bimanually. According to the TCOP, all activities were classified as an activity, coded as either tasks or actions. According to the 9 occupational categories in the COPM, the items represented all occupational areas except play/school. No new activity category was included.

Construct validity (Rasch analyses) and test-retest reliability. The characteristics of patients taking part in the 2 surveys are shown in Table I. The mean age of respondents in the second survey was 60 years (SD = 13), compared with 50 for non-respondents (SD = 16), which was the only statistically significant difference.

A total of 176 patients responded to the initial survey of the MAP-Hand. The initial 31-item instrument had low levels of missing data (Table II). Two of the items were not relevant for 20% or more of the respondents; “turning steering wheel in cars” and “typing on a computer”. However, it was decided to retain “typing on a computer”, as this activity was considered an important hand related activity among younger people and also highly relevant within the occupational areas “paid/unpaid work”, “play/school”, “quiet recreation” and “socialization”.

The likelihood-ratio test was highly significant and hence the Unrestricted Rasch model was applied. The first Rasch analysis indicated some misfit to the Rasch model (χ2 = 93.35; Bonferroni-adjusted level of significance p = 0.02). The Person Separation Reliability Index was high, with a value of 0.97. Two items with fit statistics outside the acceptable level of ± 2.5; “picking up coins” and “zipping” did not fit the model. The item “washing hands” had disordered thresholds. The measure was subsequently reduced to 19 items (Table III). The threshold and category probability curves showed that there were some difficulties in discriminating between 5 response levels, and a 4-level response category was adopted for the 19-item version.

Rasch analysis of the 19 items revealed that 2 of the 134 patients had extreme scores, the item “cutting raw vegetables” had a fit residual of –2.684 (p = 0.008), indicating that this item was redundant, and the item “typing on a computer” had disordered thresholds. After excluding these 2 patients, deleting the former and rescoring the latter item, the 18-item measure (Appendix I) was found to have a good fit to the Rasch model (χ2 42.81, p-value 0.20) (Table III). All 18 items had satisfactory fit residuals in the range 1.99–1.37, and the item difficulty level ranged from –2.3 to 2.5 logits for the most difficult and least difficult items respectively. The mean logits of the 132 patients were –1.56 (SD 1.81). The person separation reliability index was high, at 0.93.

The calibrated person and item difficulty maps of the 18-item measure showed no floor effect for the MAP-Hand (Fig. 2). The threshold ordering for the 18 items worked properly after collapsing the 2 highest scores on the item “stirring food in a pot”.

Analysis of DIF in relation to age and gender showed a uniform DIF for gender for items “tying shoelaces” and “carrying heavy objects”, indicating that there was a gender difference in the response to these 2 items. When adjusting for the uniform DIFs by splitting the 2 items for females and males the fit statistics were still excellent. There were also non-uniform DIFs in relation to age for the items “slicing bread using a knife” and “peeling raw vegetables”.

Fig. 2. Person-item threshold location distribution for the 18-item measure. SD: standard deviation.

The 2 subsets of items with positive and negative loadings on the first residual component were separately fitted to the Rasch model and the person estimates were obtained. The difference between these 2 subsets was not statistically significant, supporting the unidimensionality of the final 18-item measure.

Test-retest reliability was based on 35 patients (70%) who reported their hand function as stable in the test period. The mean difference between the 2 scorings was 0.032 (SD 0.171), and the ICC coefficient for re-scoring was 0.94 (95% CI 0.89, 0.97).

The results of testing for construct validity showed that the 18-item MAP-Hand had significant correlations within the ranges hypothesized, the largest being for the AIMS2 subscales at 0.66 for the arm subscale, and 0.78 for the hand and finger subscale. A moderate correlation of 0.39 was found with fatigue, and larger correlations of 0.65 and 0.63 were found for joint pain and patient global disease assessment, respectively.

DISCUSSION

The MAP-Hand, an 18-item patient-reported measure of hand function in activity performance, is based on the views of patients, was evaluated against accepted classification systems and has undergone a rigorous process of initial testing. The results show that the MAP-Hand is a valid and reliable instrument for use in patients with RA.

The measure was administered to participants with RA and performed well in relation to data quality and the Rasch model. The final version of the measure comprises a variety of handgrips, including twisting of the forearm and both unimanual and bimanual activities. It further covered the hierarchy of difficulty across age groups and sexes, as well as the COPM activity dimensions and the TCOP tasks and actions. As such, the MAP-Hand has undergone a more scientifically rigorous process of development than the existing instruments found in our review based on systematic searches of the literature.

The current definition of hand function in activity performance was based on 2 relevant models. The ICF model and CMOP ensured that a standard vocabulary for health and functioning and the patient’s perspective were included. Furthermore, the selection of items based on interviews with the patients ensured that the MAP-Hand had the patient-group as a primary source of content in its development.

Purposeful sampling was chosen so that the content of the measure was relevant to both sexes. With 42% men and a 20% limit required to enter the initial item set, item relevance and hence content validity for both sexes was assured. The top priority activity was “wiping after using the toilet”, which is not included in any of the existing measures of hand function identified by our structured review, indicating that patients and health professionals might have different priorities (see Appendix II). The fact that 7 of the 18 items in the MAP-Hand are not covered by existing measures, underlines the need for taking account of patient views in item development and the need for a new measure of hand activity performance that has content validity from the perspective of the individual patient.

In the study sample of 134 patients with RA, the 18-item scale covered a broad range of item difficulty, with evidence for both unidimensionality and for assessing the different levels of hand functioning. “Carrying heavy objects” and “tying shoelaces” had a uniform DIF for gender. This could be explained by the fact that men with RA maintain significantly more grip strength than women (27, 28), and the possible cultural trait of men of around 60 years of age being more frequent users of shoes with laces than females.

From a patient perspective, the goal and outcome of the rehabilitation intervention should reflect the activity limitations important to the individual (29). The considerable variation in activities found by this study has also been reported by others (30, 31). To ensure that the individual patient views are included in the rehabilitation process, the MAP-Hand could be supplemented with an open scale measure, such as the Patient-Specific Functional Scale (PSFS), in which they can add other activities of importance to daily functioning (32). This would provide patients and clinicians with important information that may form a basis for the goal-setting process, planning of interventions and evaluation of outcomes.

With the exception of “typing on a computer”, all activities could be coded by our modified version of Sollerman’s handgrips. This supports the validity of the activities as being hand-related, though no precise grip pattern was revealed, a finding also shown by others (33). However, the activities constituting the MAP-Hand were performed with a broad spectrum of handgrips, which hypothetically will protect against a floor- and ceiling-effect of the measure. In a recent study, the results showed that computer use was significantly affected in persons with arthritis, which again limited their ability to participate in work and home activities (34). Although not meeting the criteria of being a handgrip, it was therefore decided to include the item “typing on a computer” because of future relevance to a large proportion of the population in aspects of education, home- and working life.

The validation process included comparisons based on supporting hypotheses in relation to the widely used disease-specific instrument AIMS2, as well as with patient reported pain, fatigue and disease activity. The correlation between MAP-Hand and AIMS2 were high, supporting good construct validity, and the other hypotheses were also confirmed. However, the new measure should be assessed for responsiveness and compared with existing measures in RA. The psychometric properties of the MAP-Hand in other rheumatic diseases, including osteoarthritis, should also be considered. Such a process should include applying the same interview and testing procedures and may result in a version of MAP-hand with two components, one with items that are broadly applicable and one with items that are disease-specific.

The study participants were recruited from Diakonhjemmet Hospital and the ORAR register. Although geographic representation was somewhat limited, the clinical and sociodemographic characteristics of the informants were quite broad, and hence the MAP-Hand is likely to have content that is representative to western culture more generally. However, for a measure to be widely accepted, it must be validated in different cultural settings and by different investigators.

To conclude, the results from this study provide substantial support for the data quality, reliability and validity of the MAP-Hand. The measure is acceptable to patients, taking only 2 min to complete. The MAP-Hand has been translated into English following recommended criteria (35) and is shown in Appendix I.

ACKNOWLEDGEMENTS

The authors would like to thank all the participants with RA for sharing their time and experience on hand function in daily activities, and Gro Demiri and Ulf Bjørkli for contributing to the video recordings. We also would like to thank Professor Nina Vøllestad at the University of Oslo for her input in the design of the study, the health professionals at Diakonhjemmet Hospital for data entry and assistance, Else Resser Heyerdahl and Merete Hermann for analysing and coding the video recordings, and Hilde Iren Flaatten for library assistance.

This study was supported by grants from the Norwegian Fund for Post Graduate Training in Physiotherapy and Dr Jan A. Pahle’s research legacy.

REFERENCES