Sibylle Juvalta, MSc1,2, Marcel W. M. Post, PhD1,3,4, Susan Charlifue, PhD5, Luc Noreau, PhD6,7, Gale Whiteneck, PhD5, Frédéric S. Dumont, PhD6 and Jan D. Reinhardt, PhD1,8,9

From the 1Swiss Paraplegic Research, Nottwil, 2Research Unit for Health Sciences, School of Health Professions, Zurich University of Applied Sciences, Winterthur, Switzerland, 3Rudolf Magnus Institute for Neuroscience and Center of Excellence in Rehabilitation Medicine, University Medical Center Utrecht and De Hoogstraat, Utrecht, 4Department of Rehabilitation Medicine, Center for Rehabilitation, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands, 5Craig Hospital, Englewood, CO, USA, 6Centre for Interdisciplinary Research in Rehabilitation and Social Integration, 7Rehabilitation Department, Laval University, Québec, Canada, 8Department of Health Sciences and Health Policy, University of Lucerne, Lucerne, Switzerland and 9Institute for Disaster Management and Reconstruction, Sichuan University and Hong Kong Polytechnic University, Chengdu, Sichuan Province, China

OBJECTIVE: To develop and pre-test the Nottwil Environmental Factors Inventory (NEFI), a questionnaire assessing the perceived impact of environmental factors on specific areas of participation (productive life, social life, and community life) experienced by people with spinal cord injury.

SUBJECTS/PATIENTS: Thirty-seven participants with spinal cord injury in Canada, Switzerland and the USA.

METHODS: A first draft of the NEFI was developed based on a new theoretical model, the International Classification of Functioning, Disability and Health (ICF) Core Sets for spinal cord injury, and expert consultation. Three rounds of cognitive testing were conducted to examine participants’ comprehension of the conceptual framework and items, to identify challenges in cross-cultural measurement, and iteratively to refine the questionnaire.

RESULTS: Participants were able to differentiate well between environmental factors influencing productive life and those influencing social life or community life, but not between environmental factors influencing social life and community life. Items intended to capture avoidance of participation due to barriers or overcoming of obstacles were generally well understood.

CONCLUSION: For people with spinal cord injury, the NEFI may help to identify limiting and helpful environmental factors, while considering avoiding and overcoming behaviours. Quantitative validation and exploration of the possible use of the NEFI in other diagnostic groups is recommended.

Key words: environmental factors; spinal cord injury; questionnaires; ICF; social participation.

J Rehabil Med 2015; 47: 618–625

Correspondence address: Jan D. Reinhardt, Swiss Paraplegic Research, Guido A Zäch Strasse 4, CH-6207 Nottwil, Switzerland. E-mail: jan.reinhardt@paraplegie.ch

Accepted Mar 27 2015; Epub ahead of print Jun 16, 2015

INTRODUCTION

The United Nations’ (UN) Convention on the Rights of Persons with Disabilities (1) and the International Classification of Functioning, Disability and Health (ICF) (2) emphasize the role of environmental factors (EFs) in creating disability experiences. The ICF provides a comprehensive and internationally agreed list of EFs that might have an impact on disability and can be rated as barriers or facilitators.

Several studies on associations between EFs and activity limitations and participation restrictions have been performed in people with spinal cord injury (SCI) (3–5). However, current evidence for the influence of EFs on disability in persons with SCI is inconclusive (6–8). This might indicate problems with the theory underlying the study of EFs, or problems with the methods or instruments used to measure EFs. A first problem may be inadequate coverage of EFs relevant to people with SCI, as instruments have not been developed specifically for SCI (7). Secondly, assessment of the association between EFs and participation is hampered, because people who do not participate in certain areas of life, such as work, might not rate EFs in those areas as barriers, despite the fact that some of them do not participate in these life areas because they want to avoid barriers (3, 7–9). Avoidance has been shown to be a strategy used by older adults with a disability to deal with environmental challenges (10). Similarly, there may be people with a busy social and community life who encounter and report numerous barriers, but who manage to overcome them, and this is not captured in current measures (3, 11). Finally, EFs may not affect each participation domain to the same degree, but rather may affect specific areas of participation. For example, EFs deemed important to work life may not have the same importance for leisure activities; currently no measure addresses the impact of EFs on different participation domains (8).

In response to these difficulties, we suggest a new theoretical framework outlining the influence of EFs on participation, which can better address the complex interplay between EFs and disability. As an extension of the ICF framework (2), we hypothesize that experienced participation may impact on the intention to participate and on avoiding and overcoming behaviours (12). Our proposed new framework is illustrated in Fig. 1.

The participation process starts with a person’s intention to participate, e.g. the desire to go to the cinema. In the attempt to participate, and in planning an activity, the person’s perception of facilitators and barriers plays a role, e.g. the person does not want to visit a cinema with stairs (perception of barrier), or they expect that there will be people who are willing to help them get up the stairs (perception of facilitators). These perceptions may be influenced by past attempts at participation, e.g. a person who has experienced that people are not willing to help, may be less ready to seek help. Finally, the person’s attempts to participate may be more (barriers are overcome and facilitators can be utilized) or less successful (barriers cannot be overcome and facilitators cannot be mobilized). The level of success in participation has, in turn, an impact on future participation intentions. On the one hand, participation intentions may be reduced if a person experiences unsuccessful participation attempts, lowering their self-efficacy and self-esteem. Barriers linked with these unsuccessful attempts may be perceived as insurmountable and therefore avoided. On the other hand, if barriers are overcome, this can positively affect future attempts to participate.

 

Fig. 1. Theoretical framework for the study of environmental factors.

This model provides the basis for a more sophisticated measurement of EFs in people with SCI. In addition, such a measure of EFs will be more valuable if it is valid in different countries and can be used for cross-national comparisons. With this aim, the simultaneous development of a questionnaire in each country would be superior to the current standard of development in a single country followed by translation into other languages (13).

The aim of this study was to simultaneously develop, test and refine the Nottwil Environmental Factors Inventory (NEFI) in persons with SCI in Canada, Switzerland and the USA.

METHODS

Design

After the initial development of the NEFI by an international expert group, 3 rounds of cognitive interviews were conducted to test and refine the NEFI in an iterative process. Cognitive interviewing is used in questionnaire development to assess the understanding of items by the target group, reduce sources of error, and reveal whether the same concept is understood differently in different countries or languages (13, 14). As theoretical background for the cognitive interviews we used the Cognitive Aspects of Survey Methodology approach (15). This approach argues that several cognitive processes are required to arrive at an answer to a question. As a consequence, probing questions representing those cognitive processes can be posed to the respondents in order to check whether the questions cause any problems.

Ethics

The study was approved by the ethics boards of Lucerne, Switzerland, the research ethics board of the Institut de réadaptation en déficience physique de Québec Canada, and the HCA-HealthONE IRB (USA). The study followed the principles of the Declaration of Helsinki (16).

Participants and recruitment

Inclusion criteria were: people with an acute onset of SCI, aged 18 years and over, and able to speak and understand English, French or German. Individuals with cognitive disorders were excluded.

Participants with SCI were recruited through the networks of researchers and associations for people with SCI. Also, participants were asked to indicate other potential participants (“snowball method”). Written informed consent was obtained from all study participants.

Instrument development

The selection of EFs to be included in the NEFI was based on 2 studies undertaken to determine the most important aspects of functioning from the perspective of persons with SCI, which were part of the development of the ICF Core Sets for SCI (17–19), an international qualitative focus group study (20), and an international quantitative study in which functioning problems of people with SCI were assessed using an ICF-based form (21). The 30 barriers or facilitators for the long-term context that were relevant to people in the chronic phase of SCI and that were ranked highest in both studies were selected as a pool for development of the questionnaire (Table I).

Five experts from Canada, the Netherlands, Switzerland, and the USA participated in development of the questionnaire. All were experienced in the development and psychometric testing of questionnaires on EFs and participation. The 30 selected EFs were critically reviewed by the experts. The EF “Other professionals” (e360) was judged to be insufficiently specific and was excluded. Several other EFs were considered too similar to be retained as separate items and were merged in order to shorten the questionnaire as much as possible. For example, the EFs e150, e160, e515 and e520 (Table I) were merged into a single EF “accessibility of public places and buildings”. Similarly, EFs on support and attitudes of neighbours, colleagues and acquaintances were combined.

The expert group considered the impact of EFs on participation to be variable between participation domains. Therefore, it was decided to rate the perceived influence of EFs for multiple participation domains (8, 22, 23). Three main ICF participation domains were selected: “productive life”, including education, paid employment, non-remunerative employment (chapter 8 of the ICF (8, 22, 23)), housework and caring for children or older persons (chapter 6 of the ICF (22, 23)), “social life”, corresponding to chapter 7 (8, 22, 23) on interpersonal interactions and relationships in the ICF and, finally, “community life”, covering recreation and leisure activities, religious activities and political activities (chapter 9 of the ICF (8, 22, 23)). English, French and German-language versions of the questionnaire were developed simultaneously, the English-language version serving as the “Rosetta’s Stone”. Thus, all versions of the questionnaire were first written in English, then translated into the target languages and finally compared. Multiple bi- and multi-lingual researchers were involved in the translation process to ensure consistency of meaning of concepts across languages.

Cognitive testing

A German version of the questionnaire was tested in Lucerne (Switzerland) by Swiss Paraplegic Research, an English version in Englewood, Colorado (USA) by Craig Hospital and a French Version in Québec (Canada) by the Center for Interdisciplinary Research in Rehabilitation and Social Integration (CIRRIS). All interviewers had received training in cognitive interviewing from the senior author during a questionnaire development conference in April 2010. The interviews took place at the respective research institutions or at the respondents’ homes. Before the interview, participants completed a questionnaire with demographic- and injury-related information and signed the informed consent form. During the interview, the participants had a copy of the questionnaire in order to facilitate the process. All interviews were restricted to a duration of 1 h.

A minimum of 2 rounds of cognitive testing was anticipated, the first with an emphasis on general concepts, e.g. understanding of the concepts of barriers and facilitators, and the second focused on the ordering and wording of the questions. In the first round, 3 participants were included in each of the 3 countries. A small sample would be sufficient in the first round, since major problems would be identified quickly (14). In the second round, 13 cognitive interviews were conducted; 5 in the USA, 5 in Canada and 3 in Switzerland. In the third round, 15 cognitive interviews were conducted; 5 in the USA, 5 in Canada and 5 in Switzerland.

Instruments

Basic data on demographic and SCI characteristics were collected.

The initial NEFI contained 14 EFs (listed in Table I) in each of the 3 participation domains. Most EFs were accompanied by examples. The influence of each EF was assessed with a general “entry” question. If the participant indicated any influence of an EF on the respective domain, subsequent questions were asked to assess how much this EF was helpful (facilitator) or limiting (barrier) and how the respondent would react when encountering this EF (i.e. avoiding participation in the respective domain vs overcoming the barrier). This structure (Fig. 2) represents the theoretical model presented in Fig. 1. Depending on the number of EFs influencing participation, completing the questionnaire encompassed between 42 and 168 items. Changes made after each round of cognitive testing are described in the Results section.

 

Fig. 2. Design of the Nottwil Environmental Factors Inventory (NEFI) questionnaire in the first cognitive testing round.

Probing questions representing the cognitive processes required to arrive at an answer to a question were used to check whether the questions caused any problems. During the cognitive interviews, the interviewer posed all questions related to each EF at one time. After each response, the interviewer asked probing questions intended to reveal the participants’ comprehension, retrieval, judgment and response processes (Table II). Additional questions were asked if the respondent did not know the answer, exhibited uncertainty, or requested more information.

Analysis

The interviews were audio-taped and summarized in the original language. The French and German summaries were translated into English in order to share the information. After each interview round, patterns of comprehension, retrieval, judgment and response were identified and key passages were extracted. The patterns found in each country, and possible modifications to the questionnaire, were discussed in a telephone conference.

RESULTS

Three rounds of cognitive testing were carried out. The demographic- and injury-related characteristics of the interview participants are shown in Table III.

First round of cognitive testing

In Canada and Switzerland, participants did not perceive a difference between social and community life. In the USA, this was less of a problem, but in terms of the impact of EFs little difference was found between these 2 life areas. Participants also tended to indicate the influence of an EF in general and not the influence on a specific participation domain.

Another source of confusion in Canada and Switzerland resulted from the entry question, which was effective only in the USA. The entry question was intended to reduce respondent burden. However, the term “influence” was interpreted by several participants as “effectively limiting”. As a consequence, they did not feel themselves to be influenced by an EF even if they experienced problems.

Furthermore, it did not make sense to some participants to answer first how much easier an EF made their life and subsequently to answer how much harder the same EF made their life, because it was taken as the same question.

There were no issues regarding the questions about overcoming and avoiding. A Swiss participant, for instance, said that attitudes of family and relatives made his social life a little more difficult. He subsequently answered that he sometimes avoided social life and sometimes overcame these problems.

In the USA and Switzerland, the EF “devices for moving around indoors” was unclear to participants. US participants also experienced difficulties with the EF “public policies and services” and had trouble keeping the time-frame “the last 4 weeks” in mind when answering.

Revisions-1. Based on these results, the version for the second cognitive testing round was revised as follows. The EF “devices for moving around indoors” was replaced by “devices for moving around” and examples were added. The EF “public policies and public services” was changed to “national policy and public services” and examples were added. All other EFs that were not yet specified by examples were complemented, e.g. we named attitudes such as tolerance or understanding. The EF “personal assistance” was expanded to include unpaid assistance, such as support from family and friends. The participation domains, as well as the concepts of EF barriers and EF facilitators, were explained and defined in an introduction. Furthermore, the participation domains social life and community life were merged into a single participation domain and, for better contrast between productive life and social life/community life, the questionnaire was sorted by EFs and not by life areas. Thus, every EF was presented consecutively for the 2 life areas. It was decided to drop the entry question due to difficulties finding a neutral formulation. Instead, no influence was added to the response scale, and “I did not encounter any problems” was added to the avoidance/overcoming item, so that participants who were not experiencing any problems with a specific EF could answer this item. For every EF, “inadequate EF” or “adequate EF” was added in order to highlight the distinction between the EF facilitators and the EF barriers items (Fig. 3).

 

Fig. 3. Design of the Nottwil Environmental Factors Inventory (NEFI) questionnaire in the second cognitive testing round.

Second round of cognitive testing

In Switzerland and the USA, the overall conceptual framework of the questionnaire was still perceived as difficult. The difference between an EF making life harder and an EF making life easier was not understood, or participants could not think of examples of how some EFs, mainly perceived as facilitators could make their life harder, such as medications or an adapted car. Thus, the amendment of “adequate EF” and “inadequate EF” was too unspecific. In terms of understanding of the EFs, the meaning of the EF “national policy and public services” was unclear in all 3 countries. Some of the EFs were also mixed up by participants. For instance, it was difficult for participants to differentiate between devices for moving around and transportation, and to distinguish the 4 EFs on attitudes.

Furthermore, the given time-frame of “the last 4 weeks” did not make sense to most participants; most of them could not recall a specific situation in this time period and answered more generally. The interviews in Switzerland were stopped after the third interview when it became clear that the questionnaire was too difficult to answer.

In Canada, participants found it difficult to recall the definitions of productive life and social life/community life presented in the introduction of the questionnaire, and therefore answered based on their own understanding of these concepts. It was further observed that the questions on overcoming/avoidance were answered on a “what if” basis rather than based on actual past experiences. In the USA, it was observed that people with lower educational levels experienced more difficulties answering the questionnaire.

Revisions-2. The EF “national policy and public services” was restricted to health services for a better understanding of this EF. The 4 EFs on attitudes were reduced to 3 by collapsing attitudes of family and attitudes of friends into 1 item.

The time-frame to be considered for the EFs was changed from 4 weeks to 12 months. It was further decided that the definitions of productive life and social/community life should be repeated in each item. Also, specific examples of barriers or facilitators for each EF were added to enhance comprehension. The avoidance/overcoming section was restricted to 1 item at the end of every barriers or facilitators section. In the barriers section, this item asked if the participant engaged less in productive life or social/community life because of the aforementioned barriers. The participant had the option to answer “no, because barriers were usually overcome” or “yes, activities in productive life or social/community life were sometimes, often or almost always avoided”. At the end of the facilitators section participants were asked if they would have participated less without those facilitators (Fig. 4).

Fig. 4. Design of the Nottwil Environmental Factors Inventory (NEFI) questionnaire in the third cognitive testing round.

Third round of cognitive testing

In Canada and Switzerland a generally good understanding of the EFs was observed and the understanding of barriers improved compared with the second round. This is illustrated by a Swiss participant who said her wheelchair made her social/community life much harder in the past 12 months, because she got a new one and her adjustment to it was extremely difficult. She even fell out of the new wheelchair and, as a consequence, did not dare to go out for a while because she felt insecure. The only problematic EF was health services. One Swiss participant and 3 Canadian participants could not find examples of how health services affected their productive life or social/community life, while 2 US participants suggested a re-wording of the question to indicate that the lack of health services could be detrimental while availability would be beneficial. There were also concerns that the questions on health services were very complex and could be divided into several different questions (rather than linking all the concepts in the question with “and”). In the USA, the EF on personal support/assistance and the EF on medication/supplies were not well understood because of difficulties with the wording of these items.

There was a good comprehension of the life areas of productive life and social/community life. The repetition of the definition in each item helped to recall that productive life was not only about paid employment.

In Canada and Switzerland the change from the barriers section to the facilitators sections required some additional explanation from the interviewer.

In the US, the term “inappropriate” to explain the EF barrier “personal support/assistance” was perceived as confusing. US participants also said, that a “not applicable” answer category is lacking, e.g. when they did not require a personal assistant.

Revisions-3. In general, the third version of the NEFI performed better than the second. Nonetheless, there were still some difficulties that required the presence of an interviewer for clarifying and explaining. Therefore, interviewer instructions were added to the questionnaire e.g. how to introduce and explain the questions. Secondly, a “not applicable” category was added to the answer options, which could later be re-coded as “no influence”, since something that is not applicable will be neither a barrier nor a facilitator. Thirdly, the EF on health services was dropped, since the link between facilitators or barriers in health services and participation in productive life or social/community life was not sufficiently evident to the participants. The current version of the NEFI is available in Appendix SI1.

DISCUSSION

The NEFI is an interviewer-administered instrument to assess the impact of EFs on participation of persons with SCI in productive life and social/community life. It is the first EF instrument that has been specifically developed for persons with SCI based on evidence from ICF Core Sets. The final version of the NEFI comprises 56 items with 13 EFs covering all 5 EF chapters of the ICF.

The NEFI allows the influence of EFs on 2 different participation domains, productive life and social/community life, to be specified. These 2 life areas were defined very broadly, i.e. including housework or voluntary work for productive life. This is in line with the concept of subjective participation (24, 25), since each participant can select and answer based on those examples that apply best to his/her situation.

The NEFI assesses the influence of EFs on participation during the past 12 months, since a time span of 4 weeks was not representative for most of the participants in the second cognitive interview round. This time-frame can be adopted if the NEFI is, for instance, applied in an intervention study.

Compared with other EF instruments, the NEFI allows investigators to assess EFs both as barriers and as facilitators, distinguishing between productive life and social/community life, as well as the extent to which persons with SCI may avoid participation or overcome environmental barriers on these main participation domains. The cognitive interviews supported the assumption that some people tend to avoid difficult situations, especially concerning accessibility. In contrast, persons tending to overcome barriers are more confident they will find a solution if problems arise. The NEFI thereby reflects the complex interplay between EFs and participation better than other EF questionnaires. However, this comes at a cost, since the cognitive interviews showed that the interviewees needed help from an interviewer to guide them through the questionnaire; thus the NEFI in its current form is less suitable for self-report.

As participants indicated that they would have answered differently immediately after their SCI, the NEFI may be sensitive to different stages of adjustment to SCI, with a presumed high influence of EFs in the early post-acute phase and a diminishing influence with increasing time post-injury.

The NEFI is conceptually based on the ICF, but expert consultations and cognitive interviews showed that EFs, as they are described in the ICF categories, do not match the everyday perception of people with SCI. For example, categories that are explicitly distinct from each other in the ICF, such as buildings for public use, and architectural systems, may be perceived as being the same to people with SCI. Therefore, ICF categories were combined during the development process in order to achieve better comprehension, with the disadvantage that some of the final NEFI items do not relate one-to-one to specific ICF-categories. A more general coverage of the EF chapters of the ICF has, however, been maintained.

Although the NEFI has been developed specifically for use with persons with SCI, the conceptual framework presented, including overcoming and avoiding behaviours, may be applied to other disabilities as well. While different physical, sensory, mental or emotional disabilities may present different environmental barriers that can be avoided or overcome with different facilitators, both the proposed behavioural theory of participation and the NEFI questionnaire development strategy would probably apply.

Limitations

The NEFI has only been tested with cognitive interview procedures; therefore, we currently do not have information on the duration of administration of the NEFI in a real-time interview situation. Moreover, the NEFI was tested in Switzerland, Canada and the USA. Its applicability to other countries cannot be taken for granted, especially regarding low-resource countries. Our interviews in the 3 countries revealed country-specific understandings and difficulties with the questionnaire. Eventually, convenience samples were used, and this may limit the generalizability of the findings.

In a next step, the NEFI questionnaire should be field-tested with a larger sample, in order to establish psychometric properties and scoring algorithms.

Conclusion

For people with SCI, the NEFI may help to identify limiting and helpful EFs, while considering avoiding and overcoming behaviours. Quantitative validation and exploration of its possible use in other diagnostic groups is recommended.

ACKNOWLEDGEMENTS

This study was funded by the Swiss Paraplegic Research, Nottwil, Switzerland. The authors appreciate the collaboration of the 3 study centres: Swiss Paraplegic Research and Swiss Paraplegic Center (Nottwil, Switzerland), Craig Hospital (Englewood, Colorado, USA), and the Center for Interdisciplinary Research in Rehabilitation and Social Integration (CIRRIS, Quebec, Canada). The authors would also like to thank Professor Dr Gerold Stucki and Mirjam Brach from Swiss Paraplegic Research for their valuable advice and continuous support of the project.

1http://www.medicaljournals.se/jrm/content/?doi=10.2340/16501977-1982

REFERENCES