Margreth Grotle, PT, PhD1, Andrew Garratt, PhD1, Ida Løchting, MSci1, Ingvild Kjeken, OT, PhD1, Mari Klokkerud, OT, MSci1, Till Uhlig, MD, PhD1,2 and Kåre Birger Hagen, PT, PhD1

From the 1National Resource Centre for Rehabilitation in Rheumatology, Department of Rheumatology, Diakonhjemmet Hospital and 2Faculty of Medicine, University of Oslo, Oslo, Norway

OBJECTIVE: To develop the Rehabilitation Patient Experiences Questionnaire for patients undergoing rehabilitation for rheumatological disorders.

METHODS: Development of the instrument was based on literature review and adaptation of the Patient Experiences Questionnaire. The instrument was piloted and then administered in a multicentre cohort study of 12 rehabilitation units.

RESULTS: The survey included 435 patients, of which 412 (94.7%) responded to the Rehabilitation Patient Experiences Questionnaire. Following principal component analysis, the initial 27 items were reduced to 18 items and 4 scales: rehabilitation care and organization, information and communication, availability of staff, and social environment. Item-total correlations ranged from 0.77 to 0.87. Cronbach’s alpha exceeded the criterion of 0.7, and was 0.87, 0.86, 0.78, and 0.77 for the 4 scales, respectively. Construct validity was supported by correlations between the 4 scales and responses to individual questions, which were largely in the direction as hypothesized. Overall, patients reported good experiences. There were statistical differences across the rehabilitation settings in staff availability (p = 0.001) and social environment (p = 0.002), but no difference in care and organization and information/communication (p > 0.05).

CONCLUSION: The 18-item Rehabilitation Patient Experiences Questionnaire is a promising outcome measure of experiences related to rehabilitation in patients with rheumatic diseases across different clinical settings.

Key words: outcome assessment, patient satisfaction, rehabilitation, quality of care, rheumatic disease.

J Rehabil Med 2009; 41: 576–581

Correspondence address: Margreth Grotle, National Resource Center for Rehabilitation in Rheumatology, Diakonhjemmet Hospital, PO Box 23 NO-0319 Oslo, Norway. E-mail: margreth. grotle@medisin.uio.no

Submitted June 26, 2008; accepted February 26, 2009

INTRODUCTION

Despite improvements in pharmacological and surgical treatment for patients with rheumatic diseases, special rehabilitation strategies are often required. The comprehensive evaluation of healthcare interventions should include not only an evaluation of the patient’s health status, but also patient’s experiences and satisfaction with the process and quality of the healthcare delivery (1). There is limited evidence regarding patients’ experiences related to the rehabilitation process and quality of care. One reason for this may be the lack of standardized instruments to assess patients’ experiences in a rehabilitation setting. Another reason may be that patient satisfaction, like other subjective concepts, such as quality of life, is difficult to define clearly, and thereby assess accurately. However, despite measurement challenges, there is growing evidence that important aspects of patient experiences and satisfaction related to healthcare delivery can be measured in a reliable and valid manner (2–6).

Previous studies have shown that patients’ satisfaction is related to the extent to which both general and condition-specific healthcare needs are met (7), the extent to which an individual’s expectations are fulfilled (5, 8), and the extent to which patients comply with treatment (9) and take an active role in their own care (10). Critics of patient satisfaction research, however, have drawn attention to the lack of a standard approach to measuring satisfaction (11, 12) and to the lack of reliability of satisfaction surveys (12). Ceiling effects have also been cited as a problem, with many studies reporting high levels of satisfaction, a particular problem when using single questions addressing overall satisfaction (3). One way to meet this challenge has been to ask patients to rate their experiences related to specific aspects of healthcare, such as the organization of care, communication, provision of information, and degree of involvement in health decisions (6). During the past decade patient experiences of hospital care (2, 5) and outpatient care (6) in Norway have been assessed using the Patient Experiences Questionnaire (PEQ). The hospital inpatient (2, 5) and outpatient versions (6) of the PEQ have demonstrated good reliability and construct validity. However, the PEQ has not been used in a rehabilitation setting. Rehabilitation settings differ from hospital settings in many respects, including the health status of the patients, the availability of staff, and the content and organization of treatment interventions.

Objectives

The aims of this study were to describe the development of the rehabilitation version of the PEQ (Re-PEQ), and to test validity and reliability of the questionnaire in patients undergoing rehabilitation for rheumatological disorders. In particular, we would like to test whether the Re-PEQ could be used in different rehabilitation settings.

METHODS

Development of the Rehabilitation Experiences Questionnaire

Searches of the literature up until January 2006 were conducted to identify existing questionnaires assessing patient experiences in patients with rheumatological diseases or other musculoskeletal diseases. The Anglo-American and Scandinavian literature were searched for aspects of patients’ experiences of rehabilitation care due to musculoskeletal disorders, in particular rheumatic diseases. The search words used were rehabilitation, rheumatologic rehabilitation, rehabilitation facilities, overall quality, access, competence, outcome, patient satisfaction, patient experiences, continuity of care, assessment instrument, and questionnaire. Different combinations of the search terms were explored, and titles and abstracts were read. Studies published in languages other than English or in any of the Scandinavian languages were excluded. The literature review did not identify any standardized questionnaire for assessing patient experiences in patients undergoing rheumatological rehabilitation. Therefore it was decided to adapt a Norwegian questionnaire, the PEQ (5), for a rehabilitation setting.

Items and domains in the original version of the PEQ (5) were discussed for their relevance to patients in rehabilitation settings in a focus group meeting with clinical staff (rheumatologist, nurse, physiotherapist, occupational therapist) and researchers. Based on this discussion, a pilot draft of the Re-PEQ was developed comprising 27 items assessing experiences related to receiving rehabilitation care. Each item was rated on 5-point categorical rating scale and rescored to 0–100, where 100 represented best experiences.

The first draft of the Re-PEQ was tested in a pilot study among patients having rehabilitation in 5 of the 13 involved institutions. All the included items were understandable to the patients; hence, there was no need to make any changes before implementing the questionnaire in the main study.

Additional items were included as means to test construct validity: 2 items concerning overall satisfaction with rehabilitation care, 1 item assessing expectations, and 12 items addressing perceived need for improvement in different aspects of rehabilitation care.

Data collection

The 27-item pilot draft of the Re-PEQ was administered to 435 patients with rheumatic disease (56 rheumatoid arthritis, 39 ankylosing arthritis, 261 osteoarthritis, 59 other rheumatic diagnoses, and 20 missing data on diagnosis), who had completed inpatient rehabilitation for at least one week (range 1–4 weeks) in one of 12 Norwegian rehabilitation institutions. Nine of the institutions were rehabilitation centres and 3 were inpatient hospital departments. The data collection was carried out over a 4-month period from September to December 2006. Patients completed the questionnaire at discharge and delivered the questionnaire in a sealed envelope to the research coordinator at each of the study sites.

One of the main differences between the rehabilitation settings in rehabilitation centres and hospitals in Norway concerns the type of professions providing the rehabilitation interventions; in rehabilitation centres rehabilitation is usually provided by physiotherapists, 1 or 2 medical doctors and nurses, whereas in the hospitals rehabilitation might be provided by physiotherapists, medical doctors, nurses, occupational therapists, social workers and, at 2 of the hospitals, a psychologist. The study was approved by the Norwegian Regional Committee for Medical Research Ethics and the Data Inspectorate.

Sociodemographic and health-related variables

Patients filled in a questionnaire that included sociodemographic and health-related variables during the first day of their rehabilitation stay. The medical doctor provided data on diagnosis. Patients completed the Health Assessment Questionnaire (13) at admission and discharge.

Data analysis and statistical methods

Items with more than 10% missing data were considered for removal from the questionnaire. Principal component analysis (PCA) was used to assess the underlying structure of the items. Components were extracted with an eigenvalue greater than 1. Items with poor factor loadings were considered for removal from the final questionnaire. Based on previous research findings relating to patient experiences in Norway, it was expected that items would contribute to different aspects of patient experiences, including communication, information, and organization (2, 5, 6).

The internal consistency of the resulting scales was assessed using item-total correlation and Cronbach’s α. For the scales to be considered sufficiently reliable for use in groups of patients, item-total correlation should be above 0.4 and the alpha value above 0.7 (14).

In assessing construct validity it was hypothesized that Re-PEQ scores would have moderate correlations (0.4–0.6) with the responses to 3 questions assessing patients' perceptions of overall confidence in, overall satisfaction with, and the extent to which expectations were fulfilled in relation to the rehabilitation institution (15, 16). Furthermore, it was hypothesized that the Re-PEQ scores would have negative correlations of a small to moderate level (0.2–0.5) with perceived need for improvement in 12 aspects of the rehabilitation institutions. Health status and outcome have been found to have consistently small but positive associations with patient experiences and satisfaction, and hence a small level of correlation was expected in relation to change in the Health Assessment Questionnaire (4, 15). Age has been found to have a small but consistent positive association with patient experiences (4, 15).

Differences in patient experiences across different rehabilitation settings were analysed by comparing the Re-PEQ sub-scores in patients who had a rehabilitation stay in rehabilitation centres and hospital departments, respectively.

Responsiveness is not relevant to assess as this questionnaire only involve a post-treatment assessment and not a pre-post assessment as with ordinary outcome measures.

Analyses were performed using the SPSS software, version 14.0 (SPSS Inc., Chicago, USA).

RESULTS

Data collection

Of the 435 patients who received rehabilitation care 412 (94.7%) returned the questionnaire. The mean age of respondents and non-respondents were 59.2 (standard deviation (SD) 10.9) years and 61.6 (13.5) years, respectively; this and the gender distributions were not statistically significant (p > 0.05).

Of the responding 412 patients, 337 had rehabilitation stay in rehabilitation centres and 75 in hospital departments. Patients’ health status at admission according to the Health Assessment Questionnaire was similar in the rehabilitation centres (mean 0.8, SD 0.5) and in the hospital departments (0.7, SD 0.5) with no statistical significant difference between the rehabilitation settings (p > 0.05). During the rehabilitation stay the changes in patients’ health status were mean –0.23 (SD 0.35) in the rehabilitation centres and –0.12 (SD 0.26) in the hospital departments, respectively, with a statistical significant difference across the 2 settings (p = 0.011).

Statistical analyses

The levels of missing data and descriptive statistics for the 27 Re-PEQ items are shown in Table I. Levels of missing data ranged from 0% to 47.7% for the items relating to social environment and need for contact with the institution after rehabilitation stay. Item means were generally skewed towards positive levels of experiences. Five of the 27 items regarding experiences related to receiving rehabilitation care were not included in the PCA: one item regarding need for contact with the rehabilitation institution after admission had 47.4% missing data, 2 items relating to next-of-kin experiences and one item relating to information if relapse(s) were not applicable to a high proportion of the patients Additionally, the item regarding care provided by occupational therapist(s) was not included in PCA because it had both high levels of missing data (29.2%) and not applicable responses (23.0%).

Following the results of PCA and item-total correlation, 4 more items were excluded. These related to technical equipment, general impression of the institution, the same personnel/staff taking care during the stay (staff continuity), and leisure-time activities. PCA with the remaining 18 items gave 4 scales of patient experiences, accounting for 64% of the total variance (Table II): rehabilitation care and organization, information and communication, availability of staff, and social environment. The component loadings were acceptable and ranged from 0.56 to 0.88. Table I shows that the levels of item-total correlation were acceptable and ranged from 0.55 to 0.75. The alpha values for the 4 factors met the criterion of 0.7, ranging from 0.77 to 0.88.

The results of the validity testing are shown in Table III. In general, the highest correlations were found for the first 3 scales of the Re-PEQ, whereas most of the correlations for the scale covering social environment were low and non-significant. As hypothesized, the 2 overall questions regarding satisfaction showed the highest correlations with the Re-PEQ scores, in particular for the scales of rehabilitation care and organization and information/communication. The correlations were lower for the scales of availability and social environment and in the range of 0.29–0.37. The correlations between the Re-PEQ scores and fulfilment of expectations, as well as change in health status, were lower than expected, in particular for the 2 scales of availability and social environment. For the variables reflecting need for improvement the correlations were generally moderate. Re-PEQ showed poor to moderate correlation with the 12 variables reflecting institution standard. As hypothesized, the correlation with changes in health status related to the patients’ rheumatic condition were of a small level and significant for the 3 scales of rehabilitation care and organization, information/communication and availability. As expected, the correlations with age were low, but in the direction expected, with older people having better experiences of care.

Overall, patients reported good experiences related to rehabilitation, both in the rehabilitation centres and the hospital departments (Table IV). When comparing patients’ scorings in the 4 subscales of Re-PEQ across the 2 rehabilitation settings there was no statistical difference with regard to care and organization and information/communication (p > 0.05). However, there were statistical differences across the rehabilitation settings in availability of staff and social environment. Patients at the rehabilitation centres reported better experiences with availability of staff compared with patients in hospital departments (p = 0.001), whereas patients at the hospital departments reported better experiences with the social environment than patients in the rehabilitation centres (p = 0.002).

DISCUSSION

The final version of the Re-PEQ is a brief 18-item questionnaire that assesses 4 aspects of patients’ experiences related to the rehabilitation process and quality of care; rehabilitation care and organization, information and communication, availability of staff, and social environment (Appendix I). The internal consistency of these 4 scales is good and the questionnaire is suitable for patients receiving rheumatological rehabilitation. The Re-PEQ was suitable to assess patients’ experiences in different clinical settings, such as rehabilitation centres and hospital departments.

The Re-PEQ is the first standardized questionnaire that is designed for assessing patient experiences in rheumatological rehabilitation, and thus meets the need for such an instrument that will facilitate a more comprehensive evaluation of healthcare interventions. By using the Re-PEQ, patient’s experiences and satisfaction with the process and quality of the healthcare delivery can be assessed in addition to patient’s health status, which is strongly recommended by the Consort guidelines (1). The brevity of Re-PEQ makes it suitable for use together with other health-related outcome measures both in clinical studies and in national surveys of healthcare services.

The development of the Re-PEQ was based on literature review, an existing questionnaire with good evidence for reliability and validity within a Norwegian population (5, 15) and clinical expertise that promotes content validity in terms of sufficient breadth and depth of coverage of the important aspects of rehabilitation care. The response rate of 94.7% was very good, suggesting that the questionnaire is acceptable to patients. Following consideration of data quality and applicability, 4 items were excluded from the PCA. The removal of a further 5 items served to improve the resulting component structure and internal consistency of the 4 scales.

The correlations between the 4 scales and responses to individual questions were largely of the magnitude hypothesized. In particular, the 4 Re-PEQ scales had moderate levels of correlation, with responses to 2 questions relating to confidence and overall satisfaction with the rehabilitation institutions. These results are in line with previous findings (4, 6, 16). The Re-PEQ had lower correlations with the extent to which patients’ expectations were met. In particular, the correlations with availability and social environment were very small and insignificant, indicating that expectations relate more to rehabilitation care and organization, information and communication. Other studies have shown that patient satisfaction is correlated to the extent to which an individual’s expectations are fulfilled (5, 8).

As expected the Re-PEQ scores were negatively correlated with need for improvements in the various aspects of the rehabilitation institutions, with the highest correlations for the 2 scales covering organization of care and information and communication. Age was not found to have a correlation with the 3 most important scales of the Re-PEQ, which contrasts with findings within the general literature relating to patient experiences and satisfaction (4). This could be because this group of patients is quite old and there is evidence for a negative correlation for patients above 65 years of age (17). The scale of social environment had a significant but negative correlation of 0.2, which suggests that the older patients in this group have poorer interactions with other patients. This might be because they are fewer in number and there are less people to interact with of a similar age. Alternatively, they may be relatively less mobile, which may afford them less opportunity for interaction with other patients.

The present study also showed that the Re-PEQ was suitable to assess patients’ experiences across 2 different rehabilitation settings, which are quite common in Norway and Scandinavia. Patients reported good experiences related to a rehabilitation stay, both in the rehabilitation centres and the hospital departments, in particular with regard to care and organization, information/communication, and availability of staff. The patients’ level of scores for social environment was somewhat lower, however, in particular in the rehabilitation centres. Furthermore, the patients in the hospital departments tended to be less satisfied with availability of staff. These aspects of care are important to feed back to the healthcare providers in order to improve the quality of healthcare delivery.

There are a number of limitations of this study that should be considered. Firstly, test-retest reliability of the instrument has not been assessed and should be carried out in future studies. Secondly, this study mainly involved patients with rheumatoid arthritis, osteoarthritis, or ankylosing spondylitis, hence the instrument should be tested for other rheumatic diagnostic groups to reassure the suitability for these patient groups. Thirdly, due to the design of the Re-PEQ, which only allows assessment after receiving treatment or rehabilitation, it is not possible to provide data on sensitivity to change or responsiveness. Therefore, the Re-PEQ should be used together with other outcome measures.

In conclusion, the 18-item Re-PEQ is acceptable to patients receiving rheumatological rehabilitation and showed good internal reliability and construct validity. Its brevity indicates that it can easily be used alongside other patient-reported outcomes, including health status and quality of life. The questionnaire is specific to rheumatic patients undergoing rehabilitation and is recommended for use with similar groups of patients within this type of setting.

ACKNOWLEDGEMENTS

This study was financed by the Health and Rehabilitation Organisation, Norway. The authors wish to thank Therese Bjor and Bjørn Yngvar Nordvåg for contributing in the focus group and the SPOR collaboration group, who collaborated in collecting the data for the present study. The SPOR collaboration group consisted of the following institutions in Norway: Bakke Senter, Borger bad, Grande rehabiliteringssenter, Jeløy kurbad, Martina Hansens hospital, Diakonhjemmet sykehus, Revmatismesykehuset, Ringen rehabiliteringssenter, Skogli helse-og rehabiliteringssenter, Sykehuset Østfold, Tonsåsen rehabilitering, Valnesfjord helsesportsenter, and Vikersund kurbad.

REFERENCES