MAPPING THE STROKE IMPACT SCALE (SIS-16) TO THE INTERNATIONAL CLASSIFICATION OF FUNCTIONING, DISABILITY AND HEALTH

Carolina Moriello, MSc1,2, Kerry Byrne, MA3, Alarcos Cieza, PhD4, Caryn Nash, BOT1,2, Paul Stolee, PhD5 and Nancy Mayo, PhD1,2

From the 1Division of Clinical Epidemiology, 2School of Physical and Occupational Therapy, McGill University, Montreal, 3Rehabilitation Sciences, University of Western Ontario, Ontario, Canada, 4Department of Physical Medicine and Rehabilitation, University of Munich, Germany and 5Lawson Health Research Institute, St Joseph’s Health Care London, Ontario, Canada

OBJECTIVE: To demonstrate how the International Classification of Functioning, Disability and Health (ICF) can be used to create coded functional status indicators specific for stroke from a simple stroke-specific functional index, the Stroke Impact Scale-16 (SIS-16).

SUBJECTS: Nineteen professionals for the mapping portion and 8 persons with stroke for the cognitive debriefing portion.

METHODS: Participants were asked to identify appropriate codes for the corresponding items of the SIS-16 following a structured protocol for mapping measures to the ICF. A Delphi technique was used in order to reach consensus for as many items as possible. In addition, cognitive debriefing was conducted with persons with stroke.

RESULTS: A total of 13 items had Functional Status Indicators endorsed (8 items at the 4 digit level and 5 items at the 3 digit level). There were 3 items that did not reach consensus. The cognitive debriefing sessions demonstrated the differences in interpretation from the persons with stroke and the intentions by the developers.

CONCLUSION: This study has shown how the ICF can capture most items from functional status measures, such as the SIS-16. Furthermore, the items can be used to map onto a standard coding framework, illustrating the potential for increased use of Functional Status Indicators.

Key words: ICF, mapping, cerebrovascular accident, methods, functional status indicator.

J Rehabil Med 2008; 40: 102–106

Correspondence address: Carolina Moriello, Division of Clinical Epidemiology, Ross Pavilion R4.27, Royal Victoria Hospital Site, 687 Pine Avenue West, Montreal, QC, H3A 1A1, Canada. E-mail:carolina.moriello@mcgill.ca

Submitted November 14, 2006; accepted September 11, 2007.

Introduction

The specific impairments, activity limitations and participation restrictions experienced by persons with stroke is not captured by research at the population level; instead estimates about these consequences are derived from targeted research (1–9). A further challenge with identifying the consequences of stroke is that there are many different clinical measurement tools, each with their specific conceptual framework, content, wording of items and response options. As a result, communication about function is non-standardized and is referenced to the particular measure used to assess function.

While the summary scores from these functional measures are not easily interpretable, many of items on these indices represent specific functions and could be used to create functional status indicators (FSI), which would then be relevant for population-level reporting. Coding these FSI would have the added benefit of making the information compatible with the administrative databases, enhancing capacity for health services and population health research (10).

The International Classification of Functioning, Disability and Health (ICF) provides a universal conceptual framework, an international common language, and a standard coding system for describing and coding functioning, disability and health. All of the positive components are grouped under the umbrellas term “functioning”: body structure/function refers to the anatomical/physiological component of a body (e.g. joint/limited range of motion); activity is the execution of a task (e.g. walking); and participation refers to the involvement of an individual in life situations from the perspective of society (e.g. working). All of the negative components of health, impairments, activity limitations and participation restrictions are grouped under the umbrella term “disability” (11).

The ICF provides a hierarchical coding structure that categorizes 1424 unique classification units, or categories of health outcomes. Each category defines a meaningful set of related physiological functions, anatomical structures, actions, tasks, or areas of life at the most granular level and are referred to as FSI. Codes are preceded by the following letters: b, signifying impairments of body function; s, signifying impairments of body structure; d, signifying activities and participation; and e, signifying environmental factors. Each letter is followed by 1, 2, 3 or 4-digit codes, representing the degree of detail captured (11).

It has been demonstrated recently that FSI can be produced from standard outcome measures by mapping specific items to the World Health Organization’s (WHO) ICF (10, 12). The information from the coded FSI provided more specific information about the needs of persons with stroke than did the summary scores (10, 12) and improved prediction of future health events (12). Both of these studies used a generic health status measure, the Measuring Outcomes Study Short-Form 12 (SF-12) and the 12 items yielded 8 coded FSI, 2 in the physical domain, 4 in the emotional domain and 2 in the participation domain (10, 12). However, it is also important to be able to capture the level of functioning for persons with stroke. A commonly used outcome measure of functional status is the Stroke Impact Scale-16 (SIS-16).

The purpose of this paper is to demonstrate how the ICF can be used to create coded FSI specific for stroke from a simple stroke-specific functional index, the SIS-16 (13, 14). Specifically, the objectives of this study were to: (i) identify the extent to which the items of the SIS-16 yield FSI; and (ii) identify ambiguities in the wording of the SIS-16 that would impact on creating FSI.

Methods

A structured ICF Mapping Protocol was developed for the purposes of this study. This protocol was based on the mapping rules developed by Cieza et al. (15) and incorporating a Delphi technique to arrive at a consensus on the best code for each item.

The steps in mapping protocol are outlined in Table I. Nineteen professionals participated in independently assigning alphanumeric codes to the items of the SIS-16. The mapping exercise was performed by e-mail, allowing for the inclusion of professionals from various areas of expertise (occupational therapists (n = 6), physical therapists (n = 7), exercise physiologists (n = 2), psychology/behavior science (n = 2), rehabilitation (n = 1) and epidemiologists (n = 1)) and from different parts of the world (Montreal, Quebec, Canada (n = 13); London and Kitchener Ontario, Canada (n = 3) ; Pittsburgh (n = 1) and Florida (n = 1), USA; Munich, Germany (n = 1)). The participants had on average 7.5 years (range 0–30 years) experience in stroke clinically and 6.8 years (range 0–22 years) in stroke research. With respect to the ICF there were a minimal number of people using ICF in a clinical setting, but on average 4.3 years experience of using it in research (range 0–10 years). All participants received a training package prior to this mapping exercise. The presentation explained the terminology of the ICF along with the rules for mapping, which are explicitly outlined in the accompanying articles (15, 16). The participants were asked to identify the ICF codes that they felt best corresponded to each of the SIS-16 items.

We used the Delphi technique to arrive at a consensus as to the best code for each item. Key features of the Delphi process are: (i) iterative administration of the survey questionnaires with feedback between rounds, so that respondents are able to review responses from earlier survey rounds and modify their own responses as appropriate; (ii) anonymity, to avoid respondents being unduly affected by influential or forceful panel members; and (iii) statistical aggregation of results to determine the consensus response of the group (17, 18).

The percentage of agreement was calculated for all suggested codes. Agreement greater than or equal to 70% was selected a priori as a threshold that needed to be met in order for a code to be endorsed. If a code was endorsed at a 4-digit level, then the 3-digit root of that code was also endorsed. For items with less than 70% agreement, the Delphi technique (19) was used to attempt to arrive at a consensus. The exercise consisted of asking the raters to review the codes where the agreement level was below 70% and to assess if they would still implement their originally selected code or if they would now choose one of the codes suggested by another member of the panel. This process was repeated until the 70% agreement threshold was reached for an item, or it was determined that agreement would not be achieved.

Two techniques of cognitive debriefing were used in order to identify and evaluate sources of inconsistency in the coding that may arise from different interpretation of the items. The first technique, the think-aloud technique, requires that the person “think-aloud” as they are answering the question. The interviewer reads the question and records notes on the process that the person is using to arrive at the answer. The next step is the verbal probing technique, which is used to understand the cognitive processing used to answer the question. The interviewer asks for more specific information that is relevant to the person answering the question (20, 21). These cognitive debriefing sessions were conducted with 8 persons with stroke and for 6 of the 16 items where there was a range of agreement. The respondents were 8 persons living with a stroke in the community (mean age 72 years; age range 34–89 years; 50% women and men) participating in an ongoing study at the McGill University Health Center (MUHC). They ranged from 3 to 12 months post onset of stroke, with various sequelae of stroke (diminished functional walking capacity, decreased muscle strength, and fatigue). The items for which cognitive debriefing was carried out were: SIS item 3: get to the toilet on time; SIS item 4: control your bladder; SIS item 6: stand without losing your balance; SIS item 7: go shopping; SIS item 8: do heavy household chores; and SIS item 15: get in and out of a car.

Results

Mapping

Table II lists the items of the SIS-16, along with the results from the mapping exercise. The percentage of agreement ranged from 79% to 100% for the items endorsed at the 4-digit level. Table II also gives when in the consensus process this level of agreement was obtained. The percentage of agreement ranged from 89% to 100% for the items that were endorsed at the 3-digit level. These items did not reach consensus at the 4-digit level.

A majority of items were endorsed in the initial round of mapping, at a 3-digit level. However, in the first round of Delphi technique, 7 items were being endorsed at a more specific level (4-digit). There were no additional codes endorsed after the second round of Delphi technique. Due to the lack of further agreement in the second round of Delphi, there were no further rounds performed.

A total of 3 items (“control bladder”, “control bowels”, and “get in and out of car”) did not reach consensus after the second round of Delphi (Table II). Table III shows the suggested codes for the items. The codes at the end of the second round of consensus were d5300 (Regulating urination: coordinating and managing urination, such as by indicating need, getting into the proper position, choosing and getting to an appropriate place for urination, manipulating clothing before and after urination, and cleaning oneself after urination), b620 (Urinary function: functions of discharge of urine from the urinary bladder) and b6202 (Urinary continence: functions of control). It seems that there is inconsistency to the interpretation of the item being a body function or an activity. The same issue arises with the “control bowels” item.

Table III also demonstrates the suggested codes for “get in and out of car” from the initial round of coding to the second round of Delphi. There is a wide variability in the suggested codes for this item. Although there are a wide range of codes, it appears that there are inconsistent views on this task as being either a transfer (d420) or an issue of changing body position (d410).

Cognitive debriefing

Table IV summarizes the respondent’s (persons with a stroke) interpretation of the items and the definition of the ICF endorsed. The endorsed ICF code for “getting to the toilet on time” refers to the process of eliminating waste. The administration guide of the SIS-16, emphasizes that this question is associated with movement, if the respondent has the physical ability to get to the bathroom quickly enough. The respondents, however, seem to be responding to this item with respect to control over urine and not having accidents. They responded in the same way to the SIS item referring to bladder control, suggesting that these two items are the same in the minds of respondents.

For the SIS item “shopping”, the endorsed ICF code chosen is focused on an outcome, procuring goods. The respondents, however, are responding to the process of procuring these goods. Different parts of the process were identified as being important and of varying difficulty: cognitive (remembering and paying), physical (walking, carrying, standing), fine motor (paying), and transportation (driving or taking the bus). The administration guideline for the SIS-16 refers to any type of shopping, but does not include the transportation aspect of the task. The SIS item is definitely process oriented, thus matches the respondents’ concept, but the item is not specific enough to identify which parts of the process are being questioned. This would lead to difficulty in interpreting the scores on this item across people and over time if different processes were being considered.

For the SIS item standing without losing balance, the endorsed ICF code refers only to staying in a standing position and is not specific for identifying balance as a reason for not being able to stand. The respondents, however, identify specific and different reasons for their responses to this item: wavering, dizziness, losing balance and pain, and of these dizziness (b2401) and pain (b280) have ICF codes. Thus, the ICF code seems appropriate for what the people are identifying, but the SIS item has 2 specific constructs embedded, standing and balance where balance is not being captured by the endorsed ICF code. This may be because the ICF does not provide a code for balance, so it is not codeable using the ICF.

For the SIS item referring to “household chores”, the ICF code is close to the wording of the SIS item and the people seem to be responding to similar constructs. Some respondents identified fatigue as the reason for difficulty, rather than loss of ability.

For the SIS item “getting in and out of the car”, there was no ICF code identified. This could be because of the large number of processes as identified by the people in responding to this item. The SIS item seems to be focused more on the outcome of getting in or out of the car, whereas the persons with stroke are focusing on very specific components of the task. An improved item would either be that the outcome is the desired object of the question or break down the process into multiple questions.

Discussion

Codes were endorsed for 13 items of the SIS-16 with 8 items coded at a 4-digit level, and 5 items coded at a 3-digit level only. As the SIS was developed with the ICF biopsychosocial model in mind (14), this high degree of compatibility with the ICF provides further support for the content validity of the SIS-16.

There were 3 items that were not coded using the ICF due to lack of agreement among coders. For the items, “control bladder” and “control bowels”, it seems that there were differing views on whether these should be considered a body function (b620, b525, respectively) or activity (d5300, d5301, respectively). There were many different codes initially suggested for the item, “get in and out of car”. This may be because there are numerous processes involved in this activity.

There are concepts that the ICF can not capture with a code, such as balance. The SIS-16 item, “stand without losing balance” was endorsed with a code; however this code refers to the standing portion of the item and not the balance. Therefore, the true essence of the item may not be captured. This demonstrates that it is easier to develop measures based on the ICF from the beginning, in order to ensure that the true essence of the items is being captured. Items should then also be written in a manner that the respondents will interpret the items as they are defined by the ICF. For example, although there was agreement on the ICF code for shopping, cognitive debriefing indicated that shopping was a very involved process involving everything from getting to the store, choosing items, and the cognitive processes of purchasing. A new item, more compatible with the ICF code might look like: How much difficulty would you have going to a store and purchasing a needed item?

Some of the challenges encountered in this mapping exercise relate to issues such as questionnaire development and application. Items that are ambiguous or easily misinterpreted will result in imprecise measurement and poor reliability. These items will also be difficult to code using the ICF. This suggests that it may be easier to develop measures based on the ICF from the outset, in order to facilitate consistent interpretation and coding. The items could then also be written that support interpretation of the items as they are identified by the ICF.

In order to further validate the codes endorsed, a sample of persons with stroke should be re-evaluated to determine whether the coded ICF categories assigned after mapping of the items actually apply to the individual and that information is not “lost in translation”. This final step of the protocol was not carried out, therefore making this a limitation of the study. This additional information would allow us further to validate the codes that were endorsed, making it definite that the appropriate code is being endorsed.

Another limitation of the study is the low number of stroke participants in the cognitive debriefing sessions. A larger number may have provided more detailed information regarding the items.

This study has shown how most items from a functional status measure, the SIS-16, can be mapped onto a standard coding framework, illustrating the potential for increased use of FSI in electronic health records, other computerized health information systems, and the ICF core sets. The study found the Delphi survey to be an efficient method to identify consensus codes, and to highlight items where consensus is unlikely. Cognitive interviewing techniques are helpful in showing the different ways that functional status items are interpreted by respondents. The study has also highlighted the challenges associated with attempts to apply standardized codes to items with ambiguous meanings or that reflect multiple functional processes. Our group intends to continue to explore the potential for mapping functional and quality of life measures onto the ICF.

References

1. Jonsson AC, Lindgren I, Hallstrom B, Norrving B, Lindgren A. determinants of quality of life in stroke survivors and their informal caregivers. Stroke 2005; 36: 803–808.

2. Mayo NE, Wood-Dauphinee S, Cote R, Durcan L, Carlton J. Activity, participation, and quality of life six months post-stroke. Arch Phys Med Rehabil 2001; 83: 1035–1042.

3. Jorgensen HS, Nakayama H, Raaschou HO, Vive-Larsen J, Stoier M, Olsen TS. Outcome and time course of recovery in stroke. Part I: outcome. The Copenhagen Stroke Study. Arch Phys Med Rehabil 1995; 76: 399–405.

4. Robinson-Smith G, Johnston MV, Allen J. Self-care self-efficacy, quality of life, and depression after stroke. Arch Phys Med Rehabil 2000; 81: 460–464.

5. Thorngren M, Westling B. Rehabilitation and achieved health quality after stroke. A population-based study of 258 hospitalized cases followed for one year. Acta Neurol Scand 1990; 82: 374–380.

6. Pohjasvaara T, Erkinjuntti T, Vataja R, Kaste M. Comparison of stroke features and disability in daily life in patients with ischemic stroke aged 55 to 70 and 71 to 85 Years. Stroke 1997; 28: 729–735.

7. Wilkinson PR, Wolfe CD, Warburton FG, Rudd AG, Howard RS, Ross-Russell RW, Beech RR. A long-term follow-up of stroke patients. Stroke 1997; 28: 507–512.

8. Wolfe CD, Taub NA, Woodrow J, Richardson E, Warburton FG, Burney PG. Patterns of acute stroke care in three districts of southern England. J Epidemiol Community Health 1993; 47: 144–148.

9. Taub NA, Wolfe CD, Richardson E, Burney PG. Predicting the disability of first-time stroke sufferers at 1 year. 12-month follow-up of a population-based cohort in southeast England. Stroke 1994; 25: 352–357.

10. Mayo NE, Nadeau L, Levesque L, Miller S, Poissant L, Tamblyn R. Does the addition of functional status indicators to case-mix adjustment indices improve prediction of hospitalization, institutionalization, and death in the elderly? Med Care 2005; 43: 1194–1202.

11. World Health Organization. International Classification of Functioning, Disability and Health, 2nd revision edn. Geneva: WHO; 2001.

12. Mayo NE, Poissant L, Ahmed S, Finch L, Higgins J, Salbach N, et al. Incorporating the International Classification of Functioning, Disability and Health (ICF) into an electronic health record to create indicators of function: proof-of-concept using the SF-12. J Am Med Inform Assoc 2004; 11: 514–522.

13. Finch E, Brooks D, Stratford PW, Mayo NE, editors. Physical rehabilitation outcome measures, 2nd edn. Hamilton: BC Decker Inc.; 2002.

14. Duncan PW, Lai SM, Bode RK, PereraS, DeRosa J. Stroke Impact Scale-16: A brief assessment of physical function. Neurology 2003; 60: 291–296.

15. Cieza A, Brockow T, Ewert T, Amman E, Kollerits B, Chatterji S, et al. Linking health-status measurements to the international classification of functioning, disability and health. J Rehabil Med 2002; 34: 205–210.

16. Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol 2005; 58 Suppl 3: 23–337.

17. Dalkey N, editor. Studies in the quality of life. Los Angeles: Institute of Government and Public Affairs, University of California; 1970.

18. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ 1995; 311: 376–380.

19. Stolee P, Esbaugh J, Aylward S, Cathers T, Harvey DP, Hillier LM, et al. Factors associated with the effectiveness of continuing education in long-term care. Gerontologist 2005; 45: 399–409.

20. Willis GB, Research Traingle Institute. Cognitive interviewing: a “how to” guide. 1999. Available from: http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf

21. Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res 2003; 12: 229–238.