Patrik Arvidsson, MSc1, Mats Granlund, PhD1,2, Ingrid Thyberg, PhD3 and
Mikael Thyberg, PhD1,4

From the 1Swedish Institute for Disability Research, Örebro University, Örebro, 2CHILD, School of Health Science, Jönköping University, Jönköping, 3Rheumatology, Department of Clinical and Experimental Medicine, and
4Rehabilitation Medicine, Department of Medical and Health Sciences, Linköping University, Linköping, Sweden

OBJECTIVE: To explore internal consistency and correlations between perceived ability, performance and perceived importance in a preliminary selection of self-reported items representing the activity/participation component of the International Classification of Functioning, Disability and Health (ICF).

DESIGN: Structured interview study.

SUBJECTS: Fifty-five Swedish adolescents and adults with a mild intellectual disability.

METHODS: Questions about perceived ability, performance and perceived importance were asked on the basis of a 3-grade Likert-scale regarding each of 68 items representing the 9 ICF domains of activity/participation.

RESULTS: Internal consistency for perceived ability (Cronbach’s alpha for all 68 items): 0.95 (values for each domain varied between 0.57 and 0.85), for performance: 0.86 (between 0.27 and 0.66), for perceived importance: 0.84 (between 0.27 and 0.68). Seventy-two percent of the items showed correlations > 0.5 (mean = 0.59) for performance vs perceived importance, 41% > 0.5 (mean = 0.47) for perceived ability vs performance and 12% > 0.5 (mean = 0.28) for perceived ability vs perceived importance.

CONCLUSION: Measures of performance and perceived importance may have to be based primarily on their estimated clinical relevance for describing aspects of the ICF participation concept. With a clinimetric approach, parts of the studied items and domains may be used to investigate factors related to different patterns and levels of participation, and outcomes of rehabilitation.

Key words: clinimetrics; psychometrics; rehabilitation.

J Rehabil Med 2012; 44: 562–569

Correspondence address: Patrik Arvidsson, County Council of Gävleborg, Vuxenhabiliteringen, Lasarettsvägen 4, SE-803 24 Gävle, Sweden. E-mail: patrik.arvidsson@lg.se

Submitted February 25, 2011; accepted January 17, 2012

INTRODUCTION

Participation is an important issue in rehabilitation according to the perspectives both of professionals and of people with different disabilities (1–3). For persons with an intellectual disability, interventions that enhance participation may be especially important, but it is not clear how this aspect of functioning should be assessed (4, 5). The group is defined by a combination of impaired general intelligence levels and problems regarding so-called adaptive behaviour skills: in conceptual issues such as communication skills, functional academics, and self-direction, in social issues such as social and leisure skills, and in practical issues such as self-care, home living, community use, work, and health and safety (6, 7). The literature on the participation aspect of functioning (1) is sparse and somewhat disparate with respect to conceptual frameworks (5). Although participation and participation restrictions are specific aspects of functioning and disability, the concepts are broad and extensively debated (8). Therefore it is difficult to know what aspects should be covered by the items of an assessment instrument, in a context of clinical rehabilitation and applied rehabilitation research (2, 9–11). A strategy of linking existing instruments to the International Classification of Functioning, Disability and Health (ICF) is possible (12), but established instruments may contain a complex mix of ICF components and may not be clearly related to the concept of participation or an explicit interactional model of functioning and disability (4, 5, 13). In order to use the conceptual model and distinctions of the ICF, an alternative strategy is to start with ICF categories of participation, and proceed by exploring and developing adequate measurement properties.

Adhering to an integrative definition of disability (14), the ICF emphasizes the interaction between individual and environmental factors (1, 15). Among the components of functioning and disability, the ICF makes a general distinction between body structure/function and activity/participation. In addition, the ICF provides lists with specific categories of the components. These lists provide an interdisciplinary terminology in neutral terms, i.e. aspects of functioning, which may also be used to describe corresponding aspects of disability. The ICF categories of activity/participation are organized into 9 domains. Although the term activity denotes an individual perspective, while participation denotes a social perspective, the categories are specified in a common list. The structure of this list is based on conceptual distinctions, and exhibits an approximate order of increasing complexity with respect to expected sequences of actions and interactions with the social environment, both regarding the order of domains and within domains (8). Thus, the whole list of possible categories of participation is quite complex and heterogeneous (9). On the one hand, it would probably be an advantage to reduce this complexity with respect to the construction of instruments for measurement. On the other hand, it may be an advantage not to make a priori reductions, but to explore empirical relationships between different aspects in different health conditions and social environments. In order to capture the perspectives of persons with different health conditions, it seems reasonable to focus on self-reported participation (5). In adolescents and adults with a mild intellectual disability it is possible to collect self-reported data if the methods are adapted to their cognitive abilities (5, 16). To obtain a clinically useful instrument, it is necessary to make a selection of items from the extensive lists of ICF categories, and perhaps also a selection among the 9 domains of activity/participation (1). In this study, we did not make any a priori assumptions that certain “basic” domains of activity/participation should not be regarded as possible aspects of participation, i.e. involvement in life situations. This approach is compatible with option 4 according to Annex 3 of the ICF (1).

In any of the possible options regarding the 9 ICF domains, an important question is whether a measure based on a selection of items will reflect a common construct, i.e. participation. If the domains mainly reflect commonalities with respect to what people would be expected to be able to do, this would not necessarily be the most important factor influencing the consistency of measures focusing on the actual performance, which will also be influenced by, for example, what activities are perceived to be important from the perspective of an individual. In addition to questions concerning the appropriate selection of items from the activity/participation list, one may question how participation should be rated in terms of, for example, performed activity, frequency of performance, or subjective engagement operationalized as perceived importance. It is quite clear that participation does not mean just the ability to perform certain activities, not an enabling environment as such, and not just a general sense of belonging, but rather the actual involvement and performance of activities in life situations (1). Involvement might be distinguished from performance in that it also contains, at least implicitly, perceptions of involvement or the perceived importance of the activity. Thus, when trying to distinguish between the perspectives of activity and participation, it is reasonable to investigate the relations between ability, performance and importance. The relationship between ability and performance is complex, i.e. an individual’s potential to perform an activity in general or provided a certain situation, vs the actual frequency or duration of performance in interaction with the actual situations of that person (1, 17, 18), and it is not clear if there are certain domains that are less dependent on the interaction with the environment (8, 9). Thus, one question is whether there are different patterns of correlations between ability and performance regarding different domains of participation. An additional question is whether certain domains or items are perceived to be more or less important aspects of participation from the perspective of the person (8–10, 18).

The aim of this study was to explore the following questions in Swedish adolescents and adults with a mild intellectual disability: To what extent is there internal consistency in ratings of perceived ability, performance and perceived importance in a preliminary selection of self-reported items representing different domains in the activity/participation component of the ICF? To what extent do perceived ability, performance and perceived importance correlate?

METHODS

Inclusion process

Within the Swedish municipality of Gävle, persons aged 16–40 years who had any kind of societal service because of intellectual disability, and were estimated to have the ability to understand and answer the questions of the structured interview, were invited to participate in the study. To be included in societal services because of intellectual disability a formal diagnosis is requested, meaning that the person has had a significant low IQ (cut-off is usually set to 70) and before the age of 18 has had documented difficulties with adaptive behaviour. This implies that an included person may show a somewhat higher IQ at re-test in, for example, adult age (6, 19). At subsequent information sessions, the content, procedure and the conditions for taking part in the study were presented to groups of, in total, 106 potential participants, orally as well as in written text. They were asked to read the written text carefully, if necessary, together with a relative or professional of their choice. Fifty-two percent (n = 55) of those who were invited to participate answered yes. On the basis of the ethical considerations for this study, there was no attempt to collect data regarding the persons who did not want to participate.

Study group

Fifty-five informants were included, 58% men and 42% women. The mean age was 24 years (standard deviation (SD): 8), median 20. At the time of the study, the subjects IQs ranged from 56 to 78 with a mean of 68.5 and a median of 68, as assessed with the Wechsler Adult Intelligence Scale-III (19).

Sixty percent of the informants lived with parents and 27% with continuous access to professional support. Five percent lived with a partner. No-one lived alone without professional support. Fifty-eight percent of the informants attended special education and 42% had special occupational arrangements.

The study was approved by the Regional Ethics Committee of Linköping (dnr: 144-07).

Interview guide

The interview guide developed for this study included 68 items representing a selection of categories of the 9 ICF domains of activity/participation (Table I). The intention was that the selected ICF items should be relevant to persons with a mild intellectual disability (5, 6) and cover the 9 domains (1). The selection was based on the authors’ considerations concerning clinical relevance, trying to pay attention to issues used in the Adaptive Behavior Assessment System-II (7) and discussions during clinical implementations of the ICF model and the development of the ICF Universal Code sets for children and youth (20–24).

Data collection

The informants were interviewed by the first author (PA), a clinical psychologist with experience of testing and rehabilitation with respect to intellectual disability. The interview was conducted according to the order of 68 items given in Table I. In order to explore each item regarding the aspects of perceived ability, performance, and perceived importance, the interviewer asked, related to the present situation, about the following verbal alternatives and scored these as a 3-grade Likert scale: Perceived ability, I can do this fully without any help or support (2), can do it to some extent (1), cannot (0). Performance, I do this often (2), sometimes or seldom (1), never (0). Perceived importance, this is very important to me (2), somewhat important (1), not important at all (0). Regarding each item, the order of stating questions about ability, performance, and importance, respectively, was allowed to vary depending on the order of spontaneous answers given by the informants. There was time for reflection and reasoning in order to ensure that the informants understood the items and the distinctions between the aspects mentioned.

Statistics

Distributions were described using actual numbers and percentages with respect to each domain. Cronbach’s alpha was used to explore the internal consistency among the issues of the questionnaire representing 9 ICF domains of activity/participation. Cronbach’s alpha values were calculated for the total scale of 68 items and for each domain. In addition to the numerical data reported in the tables, the Cronbach’s alpha values were denoted in bold (> 0.7), plain numbers (0.5–0.7), and underlined (< 0.5), to indicate our a priori assumptions concerning values that might be regarded as high, reasonably high in a clinical perspective, or low (25, 26).

Spearman’s rank correlation was used to describe correlations between the different aspects (perceived ability, performance, and perceived importance) with respect to each item. Correlation coefficients 0.5 or higher were denoted in bold in the tables and p-values < 0.05 were regarded as significant. Statistical analyses were performed with Statistica version 8.

RESULTS

Distributions

The actual ratings for each domain are described in Table II.

Internal consistency

The Cronbach’s alpha values are shown in Table III. Regarding perceived ability, the Cronbach’s alpha value for the whole scale of 68 activities was 0.95, while the value for each domain of activities varied between 0.57 and 0.85. Regarding performance, the Cronbach’s alpha value for the whole scale was 0.86, while the value for each domain varied between 0.27 and 0.66. The low values (< 0.5) represented the domains of learning and applying knowledge, communication, mobility, and major life areas. Regarding perceived importance, the Cronbach’s alpha value for the whole scale was 0.84, while the value for each domain varied between 0.27 and 0.68. The low values represented the domains of learning and applying knowledge, communication, mobility, and self-care.

Correlations

In Table I, the correlations between perceived ability and performance, perceived ability and perceived importance, and performance and perceived importance are shown for each of the 68 items. In Table IV, the corresponding percentages of items with high correlations (0.5 or higher) are shown for each domain of activities, and for all 9 domains summarized.

Performance vs perceived importance. Seventy-two percent of the items showed high correlations between performance and perceived importance, with a mean correlation coefficient among all 68 items of 0.59 (median 0.64): for example, within the domain of learning and applying knowledge each item showed high correlations between these aspects. Within the domain of mobility each item showed high correlations except driving a car. Within the domain of domestic life each item showed high correlations, except residing as preferred. In contrast, there were no high correlations between these aspects within the domain of general tasks and demands.

Perceived ability vs performance. Forty-one percent of the items showed high correlations (0.5 or higher) between perceived ability and performance, with a mean correlation coefficient among all 68 items of 0.47 (median 0.45): for example, within the domain of domestic life each item showed high correlations, except residing as preferred. In contrast, there were no high correlations between these aspects within the domain of community, social and civic life.

Perceived ability vs perceived importance. Only 12% of the items showed high correlations between perceived ability and perceived importance, with a mean correlation coefficient among all 68 items of 0.28 (median 0.25): there were no high correlations within any of the first 4 domains, and within each of the domains 5–9 there were only 1 or 2 items with high correlations between these aspects.

DISCUSSION

In this study, a preliminary selection of items was used to explore conditions of the development of self-reported assessments of participation. Of course, different approaches would have been possible and a selection of participation items can always be questioned. On the other hand, it was based on the ICF and the results can be useful in further studies on, for example, measurement properties as well as clinical importance and levels of participation. The selected items correspond fairly well with the Universal Code sets developed by Ellingsen (24), but several items were used to represent single codes that could be contextualized as different life tasks. The internal consistency reported in Table III may be used to judge to what degree it makes sense to use the whole list of items included in the questionnaire, and the specific domains of activities, as measures of perceived ability, performance, and perceived importance, respectively. Regarding the list of items as summary scores, the Cronbach’s alpha values were high for each of the 3 self-reported aspects, i.e. perceived ability, performance, and perceived importance. The high value reflects that the included items may be regarded as one dimension of measurement, in the sense that although the items differ from each other they reflect a common construct. The value is dependent on both the homogeneity and the number of items. Regarding each specific domain, the Cronbach’s alpha values were high or reasonably high with respect to the ability aspect, while the values were below 0.5 regarding some domains with respect to the performance aspect and the importance aspect. Low values indicate that the selection of items does not give a reliable measure of a common construct based on psychometric criteria. With respect to performance, the majority of the low values were found among the first 4 domains. The discrepancy between alpha values for perceived ability and performance for these domains may illustrate a quite fundamental problem related to how the domains within activity/participation were constructed. It is probable that the latent constructs for the domains in the activity/participation component, and especially for these 4 domains, are based on commonalities regarding the ability an individual has for performing the acts listed. The acts described in these domains are not contextualized and thus are applicable on being involved in almost all the life situations listed in domains 5–9. Thus, there will be a high alpha value for those ratings of the latent construct that represent the theoretical underpinnings of the construct (ability), but not for ratings of other contextually dependent aspects of the construct.

Another example of the same phenomenon is the low value (0.3) regarding performance in the domain of major life areas, which could not reasonably be excluded from a construct of participation. Three of the 5 items included in this domain represented studying, apprenticeship, and working for payment, respectively. In a Swedish context, few persons with a mild intellectual disability would be expected to perform more than 1 of these 3 activities during the same period of life. Even if they are strongly related in terms of the ability aspect they might not be statistically related in terms of whether they will actually be performed by the individual or considered as important. This discussion highlights a general problem concerning assessments of participation: how can “full” participation be defined, and would such a measurement be reasonable without including the importance aspect (27)? In this study, we explored the importance aspect from the perspective of the person, but regarding, for example, work and employment it may also be necessary to pay attention to the perspective of society (14, 18).

For the development of overall measures of participation paying attention to both the frequency of performance and the perceived importance, improvements of Cronbach’s alpha values might be relevant. However, an alternative would be to question the relevance of using primarily a psychometric approach, e.g. using Cronbach’s alpha, to evaluate the appropriateness of the domains for measuring participation. With a clinimetric approach that also considers, for example, clinical relevance and relations to relevant conceptual models (25, 26), we suggest that items designed to measure different aspects of a phenomenon, (e.g. ability, performance, and importance) might represent the same construct because the 3 aspects reflect a life situation even if they only have low to moderate inter-correlations in general measurements. This approach would make it possible to define items in domains 5–9 as participation, accepting low alpha values. For the first 4 domains the lack of common denominators for life situations and the focus on acts rather than complex sequences of tasks would indicate activity and thus high alpha values only for perceived ability.

In this study, self-reported measurements with respect to each single item were also used to explore how people with a mild intellectual disability distinguish between the different aspects, i.e. perceived ability, performance, and perceived importance, for the same act or life situation. A possible interpretation of the rather high correlations between performance and perceived importance is that the respondents conceive performance and importance as closely related, but somewhat different, aspects of participation. Some activities may be performed almost automatically without considerations about the importance in relation to, for example, a larger project (8, 10, 17, 18, 27), and might thus be considered as activity. Other activities may imply a more conscious choice in relation to what is considered to be important from the perspective of the person. For example, the domain of general tasks and demands may represent the former, with no high correlations, while the domain of domestic life may represent the latter, with 88% high correlations between perceived importance and performance. Although the actual performance is reasonably an essential measure in the assessment of participation (1), it is possible that such measures should be weighted by including the importance aspect, as discussed above. If reflected upon by a person as important parts of a larger project, even quite basic activities, such as learning and applying knowledge, communication, and mobility, should perhaps be allowed to affect the total assessment of participation.

The finding that 41% of the items show high correlations between perceived ability and performance may be interpreted in different ways depending on assumptions about the perceived importance, environmental factors, and conceptual distinctions. If it is assumed that informants conceived ability and performance as quite distinct aspects; a distinction that was paid attention to during the interviews, one may discuss differences between the domains in terms of the interaction with environmental factors, or regarding more or less automatic behaviour vs intentional action (8). In a rehabilitation context, the way that distinctions between ability and performance are handled may be very important. For example, it may influence the financial living conditions for persons with disabilities because estimations of ability, as such, are crucial for decisions about disability pensions for those not participating in remunerative work (18). For the studied group with intellectual disabilities, discussions concerning special arrangements, with regard to, for example, work and a place to live, will also depend on estimations involving this distinction.

Regarding the domain of domestic life, the only item with low correlations between perceived ability and performance was related to acquiring and furnishing a place to live. In a clinical perspective, this is an important aspect of participation (5), and in terms of the distinctions discussed by Badley (8), a project such as acquiring a place to live may be quite dependent on interactions with environmental attitudes and socio-economic factors.

Among the domains with a rather high proportion of correlations between perceived ability and performance, the two domains general tasks and demands, and communication include items that could be conceived of as basic acts. As discussed by, for example, Coster & Khetani (9), such items may be embedded in more complex participation. Such basic activities would not be expected to vary very much with culture and would only be influenced by the environment to some extent (8). Some of these basic acts may be performed automatically if one has the ability, and be difficult to define in terms of frequency and duration. There have been quite extensive discussions concerning distinctions between, for example, basic actions and generated actions, such as “activities”, “tasks” and “life projects” (8, 17, 18, 27). In clinical assessments of the performance aspect of participation, it seems reasonable that an item should be possible to talk about in terms of frequency and duration. If so, it could be registered in the form of a diary.

The questions about education and work reflect items that are rather dependent on the social environment. Among these, education showed the highest correlation between perceived ability and performance, perhaps reflecting that some kind of education was the most consistent opportunity provided to this group and that many informants were recruited in this context. Within the domain community, social and civic life, there were no items with high correlations between perceived ability and performance, perhaps reflecting that the performance of such activities is highly dependent on interactions with the social environment (6). For example, a person may have the ability to visit restaurants and cafés, but social attitudes may constitute an environmental barrier (1, 14, 15, 28).

The low correlations between perceived ability and perceived importance indicate that the informants conceive ability and importance as quite distinct aspects. Further studies on relationships between these aspects are relevant to ICF-based intervention strategies (2).

In conclusion, a preliminary selection of items was used to explore how self-reported assessments of participation could be based on ICF categories. Within each specific domain, the Cronbach’s alpha values were high or reasonably high with respect to the ability aspect, while the values, on average, were below 0.5 with respect to performance and the importance aspect. It indicates that measures of performance and perceived importance may have to be based primarily on their estimated clinical relevance for describing aspects of the participation concept rather than on their psychometric properties. The high correlations between perceived importance and performance may indicate that respondents conceive performance and importance as closely related, but somewhat different, aspects of participation. The varied correlations between perceived ability and performance indicate that different items within domains, as well as different domains, are influenced by environmental factors to a varying extent. It is likely that measures of participation need to be contextualized, i.e. the first domains of the common list of the activity/participation component should perhaps not be used for measuring participation. With a clinimetric approach, relevant parts of the studied items and domains may be used to investigate factors related to different patterns and levels of participation, and to evaluate interventions related to participation.

AcknowledgEments

The study was funded by the County Council of Gavleborg, Sweden and FAS (The Swedish Research Council on Work and Social Life).

REFERENCES