Pearl Chung, MBBS, BMedSci1, Sarah Jin Hee Yun, BN2 and Fary Khan, MBBS, MD, FAFRM (RACP)1

From the 1Department of Rehabilitation Medicine, Royal Melbourne Hospital and 2Intensive Care Unit, Monash Hospital, Melbourne, Australia

OBJECTIVE: To compare the contents of participation outcome measures in traumatic brain injury with the International Classification of Functioning, Disability and Health (ICF) Core Sets for traumatic brain injury.

Method: A systematic search with an independent review process selected relevant articles to identify outcome measures in participation in traumatic brain injury. Instruments used in two or more studies were linked to the ICF categories, which identified categories in participation for comparison with the ICF Core Sets for traumatic brain injury.

RESULTS: Selected articles (n = 101) identified participation instruments used in two or more studies (n = 9): Community Integration Questionnaire, Craig Handicap Assessment and Reporting Technique, Mayo-Portland Adaptability Inventory-4 Participation Index, Sydney Psychosocial Reintegration Scale Version-2, Participation Assessment with Recombined Tool-Objective, Community Integration Measure, Participation Objective Participation Subjective, Community Integration Questionnaire-2, and Quality of Community Integration Questionnaire. Each instrument was linked to 4–35 unique second-level ICF categories, of which 39–100% related to participation. Instruments addressed 86–100% and 50–100% of the participation categories in the Comprehensive and Brief ICF Core Sets for traumatic brain injury, respectively.

CONCLUSION: Participation measures in traumatic brain injury were compared with the ICF Core Sets for traumatic brain injury. The ICF Core Sets for traumatic brain injury could contribute to the development and selection of participation measures.

Key words: ICF; ICF Core Sets for TBI; participation; traumatic brain injury; TBI.

J Rehabil Med 2014; 46: XX–XX

Correspondence address: Dr Pearl Chung, Department of Rehabilitation Medicine, Royal Melbourne Hospital, 34-54 Poplar Road, Parkville, Melbourne, VIC 3052, Australia. E-mail: pearl.chung@mh.org.au

Accepted Sep 17, 2013; Epub ahead of print Nov 13, 2013

INTRODUCTION

Participation is a key outcome in rehabilitation of persons with traumatic brain injury (TBI) (1). With an annual incidence of 10 million people worldwide (2), it is projected to become third greatest cause of global disease by the year 2020 (3). A recent study suggests TBI incidence may be as high as 790 per 100,000 person-years, for all TBI severity (4). The estimated cost of care and productivity loss due to TBI per year of US$406 billion (5) reflect devastating cognitive and behavioural impairments of persons with TBI for all age groups, and especially young adults, with long-term care needs and participation restriction. In addition, TBI can coexist with posttraumatic stress disorder (PTSD), and cardiopulmonary, burns or musculoskeletal injuries, which potentially confound or exacerbate early psychosocial consequences of TBI (6).

The introduction of World Health Organisation’s International Classification of Functioning, Disability and Health (ICF) (7) constructs of ‘participation’ and ‘participation restriction’ have gained prominence in TBI research. Defined as ‘involvement in a life situation’, participation represents how one functions in society with a health condition (p. 213 in ref 7).

Based on the ICF framework, ICF Core Sets have been developed for specific health conditions, including in TBI. The selection of the ICF categories for the ICF Core Sets for TBI incorporated preliminary studies involving of patients, health professionals, researchers, and international clinical data collection, followed by the decision-making and consensus process in 2010 (8). The Comprehensive and Brief ICF Core Sets for TBI consists of 138 and 23 categories, respectively. Further studies are needed to validate the ICF Core Sets for TBI from a multidisciplinary perspective. The Comprehensive ICF Core Sets represent the minimal standard of reporting for comprehensive assessments, whereas the Brief ICF Core Sets could be used in other settings where concise assessments of functioning are sufficient (9). The ICF Core Sets for TBI have many potential utility, including comparing and guiding selection of health status measures in TBI. Previous studies compared the ICF core sets for other health conditions with health status measures (9, 10). The ICF model encompasses the biopsychosocial model of health in a way which facilitates the evaluation of measures of less well-defined concepts such as participation.

Previous systematic searches for instruments measuring participation included all health conditions (11), or mixed aetiology of acquired brain injury (12–14). Other reviews were either narrative (6), reviewing all outcomes of TBI (15, 16), or used the ICF framework (13–15, 17, 18) without comparing with the ICF Core Sets for TBI (19). We are unaware of previous comparisons of outcome measures in participation in TBI with the ICF Core Sets for TBI. The main objective of this study was, therefore, (i) to carry out a systematic literature review to identify outcome measures in participation which were used in studies on persons with TBI, (ii) to examine the content of these measures in relation to the ICF, and (iii) to compare the participation contents of the outcome measures with the Comprehensive and Brief Core Sets for TBI as references.

METHOD

Systematic review to identify outcome measures

This study was a part of a broader systematic search undertaken by our group to evaluate outcome measures in adult persons with TBI. We searched PubMed, Medline, AMED, Biological Abstracts, Health and Psychosocial Instruments, and PsycINFO for all available literatures published from January 1992 to December 2012. The search identified original research in persons with TBI with participation as an outcome measure. The first stage of the literature search identified concepts related to participation, using search terms: “participation”, “handicap”, “productivity”, “reintegration”, “recreation”, “leisure”, “adjustment”, “integration”, “re-integration”, “soci*”, and “community”. The second stage identified participation aspects, using search terms: “satisfaction”, “problem”, “quality”, “accomplishment”, “performance”, “role”, and “self-efficacy”. The third stage of the search limited the search to standardised instruments using: “instrument*”, “measure*”, “interview*”, “scale*”, “questionnaire*”, and “assess*”. In addition, a previous search method was replicated to identify further relevant studies (11). All searches were then limited to “TBI” or “traumatic brain injur*”.

We included all original instruments which were used among adult persons with TBI to measure participation or closely related constructs, such as community integration (Fig. 1). Firstly, an examination of the objectives of the study as stated in the abstracts identified studies of participation or closely related concepts in persons with TBI. Secondly, an examination of the outcome measures in participation as stated in structured abstracts or in the full texts of the studies, where necessary, identified the instruments used. This also included original instrument developments and validation studies of outcome measures in participation. We excluded studies with: children, unclear number of persons with TBI, less than 50% of sample were persons with TBI, and less than 10 persons with TBI. We excluded study methods where: the diagnostic criteria for TBI were not stated; instruments for participation were administered incompletely; a surrogate measure of participation was used, such as return to work; or instruments used were not validated. We excluded study types of: non-English studies, theses, or conference proceedings. Reviews or discussion papers were not included for the identification of instruments, but were incorporated into the content analysis of outcome measures and the discussion. Two reviewers (PC, SY) independently assessed the eligibility of studies and identified the instruments. For practical reasons, we included instruments that had been completely administered in two or more separate eligible studies for further analysis. Subsequent search of the literatures identified the original instrument.

Fig. 1. PRISMA for systematic search result (31).

Content analyses of outcome measures

Comparative analyses of the identified instruments were made possible through ICF linking. The ICF categories are presented in a hierarchal nested structure with an alphanumeric code (7), where the letter represents the ICF component (Body Functions, b; Body Structures, s; Activities & Participation, d; and Environmental Factors, e), and the numbers represent the concepts as codes, with each additional numerical figure representing higher levels of categories, from first-level to fourth-level (7). This hierarchal structure makes it possible for ICF categories of a higher-level category to be simplified to a lower-level category without introducing error.

We searched the literatures to identify studies that previously linked the selected outcome measures to the ICF. We reviewed the methodology of linking used in each identified study to ensure that the standardised linking rules had been used (20, 21). The stated ICF categories for each instrument were retrieved and compared. To allow meaningful comparison of different levels of ICF categories used, such as first, second, third, or fourth-level ICF categories, the first-level ICF categories were removed, and the third or fourth-level ICF categories were simplified to the respective second-level ICF category.

For instruments not previously linked to the ICF, two authors (PC, FK) trained in the ICF linking rules independently performed the linking (20, 21). We extracted question items from the original instrument, and then identified all meaningful concepts, which were then linked to the most relevant second-level ICF category. Any concepts that could not be linked were coded as non-defined (nd), not-covered (nc), or non-defined health condition (nd-hc), according to the linking rules. Consensus was reached through discussion.

We reviewed the ICF properties of the instruments, by calculating content density and bandwidths, and content diversity indices (10, 22) (Box 1).

These were typically published with the original linking study, and where necessary, additional calculations were performed based on the published information only. Previous studies of the ICF bandwidth calculations were based on the concept that all-level ICF categories (n = 1454) should be included when comparing the number of categories used by a given instrument, including all-level ICF categories of the Activities & Participation domain (n = 393) (10, 23). In this study, we decided to use only the second-level ICF categories, and therefore the ICF bandwidth calculations were based on the available second-level ICF categories (n = 362), including second-level ICF categories of the Activities & Participation domain (n = 118). In the calculations, the original frequency of the ICF categories per instrument was not reported, firstly, because the linking to different levels of categories can affect the frequency of the ICF categories, and secondly, the main aim of this analysis was to compare the instruments with the ICF Core Sets for TBI, which is inherently a non-quantitative list. The Comprehensive and the Brief ICF Core Sets for TBI were obtained for comparative analysis of the identified outcome measures in participation (8).

Selection of ICF participation categories

There is no consensus for subdivision within Activities & Participation domain of the ICF for identifying ICF participation categories (24). The ICF permits its users to determine their own taxonomy within this domain, as relevant to their application (7), which has led to conflicting views (Fig. 2). This lack of agreement has been a major challenge for the conceptualisation and operationalisation of the ICF model (24). We accepted the suggested subdivision method by Whiteneck and Dijkers (24) which classified all items in d660 ‘assisting others’, d7 ‘domestic life’, d8 ‘major life areas’, and d9 ‘community, social and civic life’ as participation, which is also consistent with one of the ICF’s suggested subdivisions (7).

Fig. 2. Activities & Participation in the International Classification of Functioning, Disability and Health (ICF): Approaches to subclassification. d: Activities & Participation domain of the ICF; TBA: to be assessed. Note: ICF Option (1) and (2) divisions are examples, not the opinion of the ICF (7).

Comparison with the ICF Core Sets for traumatic brain injury

We compared the ICF categories in participation from the ICF Core Sets for TBI and the identified instruments, such that the ICF Core Sets for TBI were used as “yard sticks” for comparison (10). We also described the ICF categories in participation that appeared to recur among the identified instruments, and compared this with the ICF Core Sets for TBI.

RESULTS

Systematic review to identify outcome measures

From an initial total of 294 articles, 101 studies met the inclusion criteria. From these studies, a total of 20 instruments were identified (Fig. 1). The outcome measures used on two or more occasions were (n = 9) Community Integration Questionnaire (CIQ), Craig Handicap Assessment and Reporting Technique (CHART), Mayo-Portland Adaptability Inventory-4 Participation Index (M2PI), Sydney Psychosocial Reintegration Scale Version 2 (SPRS-2), Participation Assessment with Recombined Tool-Objective (PART-O), Community Integration Measure (CIM), Participation Objective Participation Subjective (POPS), Community Integration Questionnaire-2 (CIQ-2) and Quality of Community Integration Questionnaire (QCIQ). These instruments were accessible in the published literatures and are briefly described, including the number of studies using these instruments (Table I). Other instruments used on only one occasion (n = 11) were not further examined (Fig. 1). The kappa statistic of 91% for article selection; 85% for initial identification of all instruments used; and 100% agreement for identification of instruments used on two or more occasions suggested excellent overall agreement.

Content analyses of identified outcome measures

Some items in the instruments were already linked to the ICF categories, and did not require ICF linking. These outcome measures were: CIQ, CHART, M2PI, SPRS-2, CIM, and POPS (13, 15, 23, 25, 26). For CIQ-2 and QCIQ, their concepts were identical to CIQ, and they only differed from CIQ in the perspective of participation measured. Therefore, to avoid overrepresentation of identical concepts of CIQ, these two outcome measures were considered to be equivalent to CIQ for the ICF linkage and comparison with the ICF Core Sets for TBI. We performed the ICF linking of PART-O using the ICF linking rules, with complete agreement on the final list of ICF linked items.

A total of 412 meaningful concepts were identified from the 7 instruments (Table II). A total of 362 meaningful concepts were linked to the ICF categories, and meaningful concepts which could not be linked to the ICF categories (n = 50) were coded as nd, nc, or nd-hc (26, 27). The concept density was highest for POPS (d = 5.5), and lowest for CIM (d = 1.2). Bandwidths for all 7 instruments were highest for Activities & Participation components, ranging from 3% (CIM), to 26% (PART-O). Bandwidths for all of the other ICF domains were below 5% (Table II).

Selection of ICF participation categories

All unique ICF categories linked to the instruments and the ICF Core Sets for TBI were collated, from which all first-level ICF categories (n = 12) were removed, and all third and fourth-level categories were simplified to the respective second-level categories to generate final sets of ICF categories for each instrument for the content analyses. It was noted that all removed first-level ICF categories were represented by a corresponding second-level ICF category, except for d5 ‘self-care’ of the Brief ICF Core Sets for TBI. As d5 is not considered to be participation, it did not affect the content analyses.

In all 7 instruments, there were 155 ICF categories, from which 142 ICF categories were of the Activities & Participation domain (92%). From this, using the selection criteria for participation (24), 81 ICF participation categories (62%) (Table III), and 61 ICF activity categories (38%) (Table IV) were identified. Even though these outcome measures are specifically for measuring participation, over one-third of their ICF-linked categories represented activities rather than participation. For each of the instruments, the proportion of the ICF categories in participation from among all unique ICF categories used for linkage ranged from 39% (CHART) to 100% (CIM).

Comparison with the ICF Core Sets for traumatic brain injury

An examination of the ICF Core Sets for TBI using the selection criteria for participation (24) identified the ICF participation categories in the Comprehensive (n = 20) and the Brief (n = 4) ICF Core Sets for TBI (Table III). A comparison of the ICF participation categories of the instruments and the ICF Core Sets for TBI showed that the instruments covered 20% (n = 4, CIM) to 75% (n = 15, PART-O) of the Comprehensive ICF Core Sets for TBI, and 50% (n = 2, CIM and CHART) to 100% (n = 4, PART-O) of the Brief ICF Core Sets for TBI. Further, from all ICF participation categories in the instruments, 86% (n = 6, SPRS-2) to 100% (n = 4, CIM, and n = 13, POPS) could be found in the Comprehensive ICF Core Sets for TBI, and 17% (n = 2, CHART) to 50% (n = 2, CIM) could be found in the Brief ICF Core Sets for TBI.

Observations of the ICF categories in participation demonstrated recurring ICF categories in the seven instruments and the ICF Core Sets for TBI. Two ICF categories present in all seven instruments were: d920 ‘recreation and leisure’ and d750 ‘informal social relationships’. The ICF category present in six instruments was d850 ‘remunerative employment’. Five ICF categories present in five instruments were: d720 ‘complex interpersonal interactions’; d760 ‘family relationships’; d820 ‘school education’; d855 ‘non-remunerative employment’; and d910 ‘community life’. Several of these concepts were also captured by both the Comprehensive ICF Core Sets for TBI (d720, d750, d760, d850, d855, d910, d920) and the Brief ICF Core Sets for TBI (d720, d760, d920). CIM only captured four of these concepts (d760, d820, d850, d855). Other instruments contained six to eight of these recurring concepts, and all of these concepts were present in PART-O (Table IV).

DISCUSSION

To our knowledge, this is the first systematic review comparing outcome measures in participation in TBI with the ICF Core Sets for TBI. A critical examination of the ICF categories of the outcome measures in participation found participation categories represented less than two-thirds of all identified ICF categories. The ICF categories in participation in the selected outcome measures and the ICF Core Sets for TBI overlapped to varying degrees. A range of ICF categories in participation recurred in the instruments and the ICF Core Sets for TBI, in the areas of interpersonal interactions and relationships, education and employment, recreation and leisure, and community life.

Selection of ICF categories in participation

Despite being a prominent aspect of the ICF model, defining participation in the ICF for research and clinical applications continues to be a challenge. Wide variations in the terminology, definitions, and measurement methods remain problematic (24). As for terminology, ‘participation’ now replaces ‘integration’, ‘reintegration’, ‘normalisation’, and ‘participation restriction’ instead of ‘handicap’ as a single overarching term (27). The taxonomy also remains a challenge, as there is no consensus in the literatures for how to subdivide the ICF’s Activities & Participation domain despite several recommendations (11, 24). In this study, one of these recommendations for subdivision was adopted for defining ICF categories in participation without encountering difficulties, and this taxonomy could be used in future studies. Potential problems with conceptualisation of participation were also encountered through the ICF linkages of the outcome measures in participation. It was found that over one-third of all of the linked ICF categories related to activities rather than participation. Indeed, the proportion of the ICF categories in participation in each outcome measure was widely variable. This can be a major problem for instruments designed to demonstrate the efficacy of an intervention to improve participation. If outcome measures for participation incorporate measures of activities, such as independence with personal care, mobility or domestic activities of daily living, the results cannot be said to truly reflect changes in participation. Further work is needed to ensure the instruments are appropriate for measuring participation.

Comparison with the ICF Core Sets for traumatic brain injury

Our examination of outcome measures in participation in TBI demonstrated many similarities in concepts with the Brief and Comprehensive ICF Core Sets for TBI. Being a shorter instrument, the CIM covered the smallest number of ICF categories in participation from the Comprehensive ICF Core Sets for TBI; however, all its concepts were found in the Comprehensive ICF Core Sets for TBI. The Brief ICF Core Sets for TBI contained only 4 ICF categories in participation, and so limited its utility for comparing with the outcome measures in participation.

Recurring ICF categories were identified in the selected outcome measures in participation and the ICF Core Sets for TBI. Difficulties experienced by persons with TBI in lifestyle choices and relationships were captured through: d920 ‘recreation and leisure’, d720 ‘complex interpersonal interactions’, and d760 ‘family relationships’. Impact of cognitive impairment of TBI on education and vocation were recognised by: d850 ‘remunerative employment’, d820 ‘school education’, and d855 ‘non-remunerative employment’.

The instruments and the ICF Core Sets for TBI can be mutually informative. Many instruments have the benefit of pilot and validation clinical studies over the years. The strengths of the ICF Core Sets for TBI are in their rigorous methodology for development, and in the future, through clinical validation data. Careful considerations are needed when selecting outcome measures to ensure the contents of the instruments closely correlate with the outcomes of interest, which in this case, is participation. The ICF Core Sets for TBI may be further incorporated into the development and improvement of outcome measures in participation to ensure that important issues in participation in TBI will be measured.

Limitations

This study has some potential limitations. This is a systematic search with specific inclusion and exclusion criteria of published literatures, and may not fully represent what is actually used in clinical settings. Instruments that were used once were excluded from a practical perspective. Sample sizes of studies were not considered. Despite efforts to remove duplicate studies, repeated data may have been included. Dates of studies were not incorporated; and older instruments were more likely to be used more than once. Simplifying the ICF linked concepts to second-level ICF categories decreased the amount of information available for analysis, and collating data from various authors may have introduced error. The ICF Core Sets for TBI are not yet validated and may change.

Conclusion

TBI has long-lasting impact on participation. Rehabilitation goal is to minimise participation restriction, which enables the persons with TBI to function in society; decrease dependency; and improve life satisfaction and productivity. The ICF Core Sets for TBI contain ICF categories in participation in persons with TBI, which overlapped to varying degrees with all of the identified instruments. This is encouraging for both the instruments and the ICF Core Sets for TBI which are yet to be validated. Further work including the validation of the ICF Core Sets for TBI will be a priority for identifying the ICF categories in participation that are meaningful in TBI.

ACKNOWLEDGEMENTS

We thank Professor A Ekram for his assistance with the systematic search, and Dr B Amatya for his guidance on data analysis. The above mentioned persons were informed and approve of this acknowledgement.

REFERENCES