Shanjia Chen, MSc1,2#, Jing Tao, PhD1,2,3#, Qian Tao, PhD4,5, Yunhua Fang, MSc1,2, Xiaoxuan Zhou, MSc1,2, Hongxia Chen, PT, MSc6, Zhuoming Chen, OT, PhD7, Jia Huang, OT, PhD1,2, Lidian Chen, PhD1,2,3 and Chetwyn C. H. Chan, PhD5

From the 1College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, 2Fujian Key Laboratory of Exercise Rehabilitation, Fujian provincial rehabilitation industrial institution, 3Affiliated Rehabilitation Hospital, Fujian University of Traditional Chinese Medicine, Fuzhou, 4Psychology Department, School of Medicine, Jinan University, Guangzhou, 5Applied Cognitive Neuroscience Laboratory, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, 6The Second Affiliated Hospital of Guangzhou University of Traditional Chinese Medicine and 7The First Affiliated Hospital of Jinan University, Guangzhou, China. #These authors contributed equally to this work.

OBJECTIVE: To investigate how clinical experience and access to patient information regarding functional capability influence inter-rater reliability and validity of the Brief International Classification of Functioning, Disability, and Health Core Set for Stroke (ICF) assessment.

METHODS: Study 1 involved expert (clinical experience ≥ 5 years) and novice (clinical experience < 2 years) rater-pairs, each evaluating the same post-stroke patients using the ICF assessment (n = 149). Study 2 involved novice raters separately evaluating a different cohort of post-stroke patients with the ICF assessment (n = 78). The novice raters had prior knowledge of patient functioning through conducting 6 clinical tests.

RESULTS: For Study 1, the expert rater-pairs (kappa = 0.50–0.85 for categories; intra-class correlation (ICC) = 0.76–0.96 for components) had higher reliability coefficients than novice rater-pairs (kappa = 0.18–0.69 for categories; ICC = 0.63–0.88 for components). For Study 2, the novice raters with prior knowledge of patient’s functioning yielded significantly higher ICF component scores than those without prior knowledge. The former raters’ component scores were comparable to those of the expert rater-pairs.

CONCLUSION: Clinical experience in post-stroke rehabilitation enhances inter-rater reliability of ICF assessment. Know­ledge of patient’s functional capability, such as conducting common clinical tests in post-stroke rehabilitation, is useful for improving assessment validity.

Key words: ICF; stroke; reproducibility of results; clinical experience; disabled persons.

J Rehabil Med 2016; 48: 265–272

Correspondence address: Lidian Chen, College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine,1 Huatuo Road, Minhou Shangjie, Fuzhou, Fujian 350122, China. E-mail: cld@fjtcm.edu.cn and Qian Tao, Psychology Department, School of Medicine, Jinan University, 510632, Guangzhou, China. E-mail: tracy.tao@connect.polyu.hk

Accepted Dec 18, 2015; Epub ahead of print Feb 17, 2016

INTRODUCTION

The International Classification of Functioning, Disability, and Health (ICF) Core Set for Stroke captures prototypical problems relevant to post-stroke patients irrespective of their stage of rehabilitation (1, 2). The Brief Core Set for Stroke includes 18 second-level categories grouped into 4 components: 6 categories for Body Function (BF), 2 for Body Structure (BS), 7 for Activity and Participation (AP), and 3 for Environmental Factors (EF) (3). The generic qualifier scale under each category quantifies severity of impairment, which has 5 response levels ranging from “0” (no impairment) to “4” (completely impairment). The instructions for test administration suggest that health professionals gather information from clinical records in order to facilitate assigning scores on the ICF (4). The instructions, however, do not stipulate how this could impact the reliability of the assessment or any requirements for rater experience and background. This study aimed to reveal how different clinical experience levels of raters and access to patient functional capacity information influences the assignment of scores on the Brief ICF Core Set for Stroke (hereafter called the ICF assessment).

Consistency of scores across raters is an inter-rater reliability issue. Clinical experience of raters is a key factor contributing to accuracy and reliability of results generated from clinical assessments (5, 6). For instance, the reliability of fracture classifications and accuracy in making clinical decisions can be improved with clinical experience (7). Common measures of clinical experience are years of clinical practice, level of professional training, credentials, and seniority in the field (7, 8). Clinicians with more experience have been found to be able to relate the case at hand to similar cases previously encountered (9). Researchers explained that more experience would enable clinicians to develop higher reasoning ability for identifying social/behavioural cues beyond patient self-reports (10). Another study postulated that an advantage of having more clinical experience is the accumulation of a larger dataset of patient-related problems and solutions (11).

Previous reliability studies on the ICF have reported low to moderate consistency among raters (12–15). One study on the difference of test-retest reliability of the ICF assessment between novice and experienced raters indicated that more experienced raters had significantly higher consistency than less experienced raters on BF and AP categories among older adults (12). Another study focused on influences of confidence and core competence of raters on inter-rater reliability with the Extended ICF Core Set for Stroke (13) and the results indicated that neither variable had significant effects on reliability. The researchers postulated that factors that could have influenced consistency would be rater clinical experience, skills, and knowledge about stroke. These higher level attributes would enable clinicians to observe and identify patient problems relevant to ICF factors and categories. Nevertheless, those postulations were not substantiated by their findings. Besides clinical experience, the type of information that raters can access would influence their assignment of scores for performance during clinical assessment. The different ways of collecting patient information, such as direct observation and clinical records, might influence scoring of ICF categories (16, 17), thus affecting their validity.

The aim of the current study was to investigate how clinical experience and access to patient information regarding functional capability influences inter-rater reliability and validity on the ICF assessment. Two research questions were investigated: (i) how raters with different clinical experience (more or less) influence the reliability of ICF assessment; (ii) how knowledge of patient functional capability influences the validity of ICF assessment. Study 1 investigated the impact of clinical experience on inter-rater reliability of the ICF assessment in a group of post-stroke patients. Study 2 investigated how common rehabilitation clinical tests administered by less experienced therapists influenced ICF assessment results and hence, its validity. It was hypothesized that raters who had more clinical experience would yield higher inter-rater reliability indices than those with less experience (Study 1). Equipping rehabilitation therapists with knowledge of patient functional capability is posited to positively affect the validity of ICF assessment scores among raters with less clinical experience (Study 2).

METHODS

Participants

The 10 raters participating in this study were rehabilitation therapists working in 2 rehabilitation in-patient settings located in Southern China. Among them, 6 had less clinical experience (mean 1.45 years; range 0.9–1.5 years) and 4 had more clinical experience (mean 8.65 years; range 7.9–9 years). Raters with less experience (< 2 years) in stroke rehabilitation were defined as novices, and raters with more experience (≥ 5 years) were defined as experts. All raters received undergraduate training in rehabilitation and were certified as rehabilitation therapists. None of the raters had experience in administering the ICF assessment prior to this study. Study 1 involved 2 expert and 2 novice rater-pairs. Study 2 involved another 2 novice raters, who adopted a reverse assessment sequence to that of novice raters in Study 1. There was no significant difference between the 2 studies in the years of clinical experience for novice raters.

Participants were post-stroke in-patients recruited from the 2 rehabilitation settings. Inclusion criteria were: (i) diagnosed as first stroke and confirmed by brain scan; (ii) aged between 40 and 80 years; (iii) 4–12 weeks post-stroke and receiving active rehabilitation; (iv) hemiplegia resulting in contralesional paralysis; and (v) severe neurological function according to the National Institutes of Health Stroke Scale (NIHSS) (score ≥ 6) (18). Patients were excluded if they had aggravated disease or failed to complete the clinical assessment protocol. A total of 268 patients were screened; 32 did not meet the inclusion criteria, and 9 did not complete all assessments. The final sample size was 227 (37.0% female) with a mean age of 61.7 years (Table I). The participants were randomly assigned to expert rater-pairs (Study 1, n = 72), novice rater-pairs without prior knowledge of patient functioning (Study 1, n = 77), and single novice rater with prior knowledge of patient functioning (Study 2, n = 78). The study obtained ethics approval from the Institutional Review Board of each clinical setting. Written informed consent was obtained from all participants or their proxies.

Instruments

ICF assessment. The ICF assessment consists of 18 categories, each of which is accompanied by a detailed description of its content. The generic qualifier serves as the rating scale for the BF, BS, and AP components (13), which has 5 response options ranging from “0” to “4” (no/mild/moderate/severe/complete impairment) indicating the extent of impairment (Table II). The EF qualifier has 9 response options ranging from “–4” to “4”. Options “–4” to “–1” denote different levels of environmental barrier; “0” denotes no influence; options “1” to “4” denote different levels of environmental facilitator enhancing the function of the individual; option “8” denotes not specified and option “9” denotes not applicable. The raters conduct the assessment by means of: (i) observing individual’s performance in executing a task or a movement in life situations; (ii) interviewing the individual and his/her proxies; (iii) asking questions such as “Where are you now” or “What is the time?” for evaluating orientation; and (iv) reviewing clinical history and clinical examination results. Options “8” and “9” were treated as nominal categories in Cohen’s kappa reliability statistics. A category score was derived directly from the response option of the qualifier (not including “8” and “9”). Category scores were summed to form component scores for BF, BS, AP, and EF. A higher score for the BF, BS, AP and a lower score for the EF indicated more serious dysfunction and barrier conditions in patients.

Short Form Health Survey (SF-36). The SF-36 (19) is widely used for assessing health-related quality of life. It is a self-administered questionnaire that contains 36 categories grouped under 8 dimensions of health: physical functioning, role limitations due to physical problems, bodily pain, general health perception, vitality, and social function, role limitations due to emotional problems, and mental health, and 1 single-category scale on health change over the past year (19). A concurrent study was conducted to translate the SF-36 into Chinese (Appendix SI1).

Other clinical tests. Patient function was assessed with 5 other common clinical tests, all of which have been validated for usage in Chinese populations in previous studies: the Montreal Cognitive Assessment (MoCA) measures the overall cognitive profile (20), the Chinese version of Modified Barthel Index (MBI-C) assesses self-care independence (21), the Chinese version of Frenchay Activity Index (FAI-C) measures activities of daily living (22), the brief version of the Fugl-Meyer Assessment (FMA) assesses upper and lower limb functions (23), and the modified Rankin Scale (mRS) measures global disability and prognosis (24).

Procedure

In Study 1, expert and novice rater-pairs administered the ICF assessment to each patient assigned to their group (Fig. 1). One rater from each pair contacted the patient and completed the first ICF assessment on the second day after subject recruitment. Then, the second rater in the same rater-pair administered the ICF assessment on the same patient within 24 h to minimize possible changes in patient functional status with time. After completing both ICF assessments, 1 rater in each pair randomly conducted 6 clinical tests over the next 3 days. The sequence for administering the tests was fixed: MoCA, SF-36, MBI-C, FMA, FAI-C, and mRS. The sequence in which raters within each rater-pair completed the ICF assessment, as well as the rater who administered the 6 clinical tests, were randomly determined by drawing lots. The assessment design and testing schedule were the same for both expert and novice rater-pairs at each institution.

 

Fig. 1. Study design and summary of implementation of the study.

 

Fig. 2. Comparison of the International Classification of Functioning, Disability, and Health Core Set for Stroke (ICF) component scores (mean and standard deviation (SD)) among the expert rater-pairs, novice rater-pairs (without prior knowledge), and single novice rater (with prior knowledge). *p-value < 0.05.

Study 2 involved an additional 2 single novice raters who conducted their patient assessments in reverse of Study 1. Each single novice rater started to contact the patient on the second day after patient recruitment and completed the 6 clinical tests within the next 3 days. The ICF assessment was then administered immediately after completing these 6 clinical tests. The procedures for completing all 6 clinical tests were the same as those followed by raters in Study 1.All raters from both studies received training on administering the ICF assessment and 6 clinical tests. Manuals that described test administration procedures were distributed to all raters. Training began with explaining the purpose and content of each test. Demonstration of all testing procedures involved real patients. The training session lasted for 3 days. ICF assessment training included familiarization with the ICF framework and classifications and illustrating methods for interviewing patients and proxies. Raters were familiarized with the types and format of clinical information contained in patient case files, such as results of clinical and laboratory examinations conducted by their medical and rehabilitation teams. All assessments were conducted in a quiet room within the rehabilitation department. Only a few patients were too weak to walk (or use a wheelchair) to the room, and for these subjects data collection was carried out at the bedside. As a control, the results of 6 clinical assessments were not included in the list of information available to the raters.

Data analysis

For Study 1, inter-rater reliability of the ICF assessment used percentage of agreement and Cohen’s kappa with 95% confidence intervals (95% CI) at the category score level, and ICC at the component score level. The strengths of the kappa for expert rater-pairs were compared with those for novice rater-pairs (25). Pearson’s correlation coefficients were used to test relationships between component scores on the ICF assessment with those of the 6 common clinical tests.

For Study 2, the data collected was combined with that from Study 1. There were 3 rater group conditions: expert raters without prior knowledge of patient functional capability, novice raters without prior knowledge of patient functional capability, and novice raters with prior knowledge of patient functional capability. Since each participant was assessed twice, results of the first ICF assessment were selected for comparison. ICF assessment ratings for each patient from each rater group condition were compared using multivariate analysis of variance, followed by post-hoc comparisons. The level of statistical significance was set at p < 0.05. All analyses were performed with STATA (version 8.17) and SPSS (version 18) statistical software.

RESULTS

Study 1

For expert rater-pairs, the percentage agreement for the ICF categories was 82.0% (95% CI 79.2–84.8%) and Cohen’s kappa coefficient was 0.73 (95% CI 0.68–0.77). For novice rater-pairs, the percentage of agreement was 65.0% (95% CI 61.3–68.8%) and Cohen’s kappa coefficient was 0.45 (95% CI 0.38–0.51) (Table III) For expert rater-pairs, 17 out of 18 categories (94.4%) showed Cohen’s kappa coefficients above 0.60, compared with only 2 categories (11.1%) for novice rater-pairs. Most novice rater-pair categories (72.2%) showed moderate reliability. At the component score level, the ICC coefficients for expert rater-pairs ranged from 0.76 (95% CI 0.63–0.84) for EF, 0.89 (95% CI 0.83–0.93) for AP, and 0.96 (95% CI 0.93–0.97) for both BF and BS. In contrast, the ICC coefficients for novice rater-pairs ranged from 0.63 (95% CI 0.48–0.75) for EF, 0.85 (95% CI 0.78–0.90) for BS, 0.88 (95% CI 0.82–0.92) for BF, and 0.88 (95% CI 0.81–0.92) for AP.

For expert rater-pairs, the strongest Pearson’s correlation coefficients were found between MoCA scores and BF (r = –0.81) and AP (r = –0.69) (Table IV). No significant correlations were found between scores on any of the 6 clinical tests and EF. For novice rater-pairs, stronger correlations were found between the clinical test scores and AP. No significant correlation coefficients were revealed for the EF.

Study 2

One-way multivariate analysis of variance revealed significant differences among the 3 rater groups (F8,426=5.40, p < 0.001). Post-hoc comparisons also revealed significant differences between rater groups except for EF scores (Fig. 2). There were no significant differences between expert rater-pairs and single novice raters with prior knowledge for BF (p = 0.32) and BS (p = 0.48). However, novice rater-pairs without prior knowledge were found to yield significantly lower mean (standard deviation (SD)) BF (p = 0.02; 5.92 (SD 3.73) vs 7.72 (SD 5.10)) and BS (p < 0.001; 2.82 (SD 1.49) vs 4.11 (SD 1.69)) component scores than single novice raters with prior knowledge. A similar pattern of differences between novice rater-pairs without prior knowledge and expert rater-pairs were revealed for mean BF (p = 0.001; 5.92 (SD 3.73) vs 8.51 (SD 5.28), respectively) and BS (p < 0.001; 2.82 (SD 1.49) vs 3.91 (SD 1.66), respectively). On the contrary, for AP component scores, no significant difference was revealed between novice rater-pairs without prior knowledge and single novice raters with prior knowledge (p = 0.88). Both single novice raters with prior knowledge (p = 0.006; mean, 11.54 (SD 6.44)) and novice rater-pairs without prior knowledge (p = 0.004; mean, 11.38 (SD 7.27)) were found to yield significantly lower scores on AP than expert rater-pairs (mean, 14.66 (SD 6.51)). Notably, the relationships between the 6 common clinical tests and ICF component scores for single novice raters with prior knowledge showed a pattern more similar to that for expert rather than novice rater-pairs without prior knowledge (Table II).

DISCUSSION

The most significant finding of Study 1 is that the expert raters yielded substantially higher inter-rater reliability on the ICF assessment than novice raters. This suggests that clinical experience confers greater consistency in ICF scoring for post-stroke patients. By gaining knowledge about patient functional capacity in Study 2, single novice raters yielded ICF assessment scores similar to expert raters. This compared more favourably with novice rater-pairs without prior knowledge about patient functional capability. Nevertheless, the advantages of both clinical experience and prior knowledge of patient functioning seem to most benefit BF and BS scores and least benefit EF scores.

Our results showing raters with more clinical experience having more consistent ICF assessment scores than those with less experience concur with other types of reliabilities (5, 7, 17). Okochi et al. (12) adopted 8 years of experience as a cut-off and revealed higher test-retest reliability with BF and AP scores for more experienced evaluators. The benefit of clinical experience is proposed to account for higher clinical reasoning competence among experienced clinicians (26). One current theory of clinical competence suggests that experiences enable clinicians to accumulate and classify similar clinical problems based on a larger dataset (11), which would strengthen their classification and recognition capability (27). Experienced raters, such as experts in Study 1, could effectively conduct case-based reasoning by identifying the similarities between patients currently under assessment and those from the past (28). This, in turn, would promote consistency in assigning qualifier scores by the raters with more clinical experience for each ICF category. Other studies on expert clinicians have suggested that the expertise developed promotes narrative reasoning and a patient-centred approach (29–31). These skills facilitate effective communication between experienced clinicians and patients, as well as reading of the patient’s history in a functional and psychological context (32). In contrast to experienced clinicians, those with less clinical experience were found to be more inclined to use hypothetical-deductive reasoning, which requires conscious effort to extract information while constructing hypotheses based on the patient’s problems (6). As a result, less experienced clinicians have less clinician-patient interaction, which limits their understanding of the patient’s problems (31). This perhaps explains why novice raters in Study 1 yielded lower inter-rater reliability on ICF assessment than expert raters.

In Study 1, novice raters yielded moderate inter-rater reliability on ICF scores. These results are in agreement with other studies involving novice raters (14) and reflect the dissatisfactory consistency of scores on the ICF assessment by raters who completed basic training on ICF framework and assessment methods. Our results on the expert raters are comparable to those reported by Gan et al. (33). Their study involved 2 experienced raters having more than 5 years of experience in rehabilitation administering the ICF-Children and Youth-based questionnaire to children with autism and produced good to excellent reliability (component ICC = 0.72–0.97). In contrast, coefficients yielded from scores of more experienced clinicians were higher than the moderate inter-rater reliability of the Comprehensive ICF assessment (κ = 0.41, 95% CI 0.39–0.43) reported by Starrost et al. (13). Their study recruited 2 physical therapists with more than 5 years of experience in neurorehabilitation assessing 30 post-stroke patients. The higher reliability coefficients yielded could be due to participating clinicians rating a greater number of post-stroke patients, as well as use of the Brief version of the ICF. Both factors would enhance the consistency of rating assignments.

Study 1 revealed relatively higher inter-rater reliability coefficients for BF, BS, and AP for both expert and novice raters. EF yielded the lowest inter-rater reliability coefficients, and clinical experience did not appear to have a significant impact. These findings are consistent with those reported for assessment of patients with low back pain (15), stroke survivors (13), and patients with rheumatoid arthritis (14). The relatively lower inter-rater reliability of EF ratings could be attributed to several things. First, the EF categories are not routinely found in medical records (16), such as category e580, which focuses on the role of “health services, systems, and policies” on preventing and treating health-related problems, providing medical rehabilitation, and promoting a healthy lifestyle. Raters would need to make a judgement without referencing existing information on these categories. Without neuropsychological records, rater judgements were based on subjective observations and information gained from patient self-reporting. This is supported by the fact that no significant correlations were revealed between EF and the 6 clinical tests. Secondly, a previous study commented that the EF categories were relatively broad compared with other components and that the qualifier scale (–4 to +4) was more complex (13). The content of the categories pertains to the physical, social, and attitudinal environment (34), such as category e310 “immediate family,” which focuses on patient relationships with family members influencing rehabilitation outcome. When a category can be expressed as both a positive and negative factor, the rater may randomly make a judgement (14). Furthermore, the rater sometimes receives inconsistent or vague information from patients and their proxies (13). Finally, the technical terms were found to be unfamiliar to raters (14), such as category e580 “health services, systems, and policies,” e310 “immediate family,” and e355 “health professionals.” The relatively low inter-rater reliability yielded for the EF indicates rooms for improvement in the design of the qualifier and test process. First, the number of response options can be decreased (14, 15, 35), such as from “–4” to “+4” to “–2” to “+2”. This would lower the demand of quantifying the judgement to score process on the rater. Secondly, definitions can be given to each response option to further clarify its uniqueness using written descriptions (16) or pictorial presentation (12). Thirdly, the test process can be further standardized in terms of the sequence for collecting or reviewing specific types of information, and the associated decision­-making and consensus processes (36). Last, but not least, self-rating by the patient may form one type of information to be considered.

The results obtained from Study 2 are intriguing, in that equipping raters with knowledge regarding the functional capability of post-stroke patients enabled novice raters to produce scores similar to that of expert raters. Such enhancement effects were found to be prominent for BF and BS, but not for AP and EF. Although producing comparable component scores does not necessarily mean an improvement in inter-rater reliability, increases in these scores suggest that knowledge about patient functioning could promote the validity of novice raters using the ICF assessment. The clinicians are encouraged to be creative in choosing appropriate clinical measures for conducting the ICF assessment (37). However, no studies have yet addressed the issue of incorporating common clinical tests into the ICF rating protocol. Administering common clinical tests, such as those adopted in this study, before conducting the ICF assessment is recommended for raters with less clinical experience. Future studies are needed to investigate the extent to which this can enhance inter-rater reliability among novice raters. Researchers should further explore effective strategies to improve the validity of AP and EF scoring. Lastly, the strong correlations revealed between the 6 clinical tests and BF and AP, which is consistent with the findings of a previous study (38). In particular, the MoCA, MBI-C, and SF-36 were found to yield the highest correlation coefficients, suggesting that constructs of these 3 evaluations may well overlap with BF and AP on the ICF. To enhance validity of the ICF assessment, clinicians with less clinical experience may consider getting access to patient MoCA, MBI-C, and SF-36 results before administering the ICF assessment for post-stroke patients. Furthermore, future studies should target development of proxy measures for assisting less experienced clinicians to enhance the validity of EF ratings.

Further improvement in the reliability of the ICF assessment can be achieved by incorporating illustrations as supplementary material in the test. The ICF Illustration Library (http://www.icfillustration.com/top_e.html) can be a useful tool for use by the raters to seek clarification of the definition of specific components and codes. Clear operational definitions enable novice raters to delineate the scope and content of the evaluation, which would improve the reliability of the assessment. Other methods of improving the reliability include formulating standardized decision and consensus process on assigning ratings (36), and reducing the number of qualifiers of the ICF assessment (14).

Study limitations

The present study has several limitations. First, the expert and novice raters recruited are limited to the clinical experiences that they gained within the post-stroke rehabilitation settings in which they practiced. Generalization of the results to other raters practiced in settings and types of patients not similar to those for this study. Secondly, no inter-rater reliability was established for the novice raters recruited in Study 2. Therefore, the results obtained in Study 2 could not be interpreted as enhancing inter-rater reliability. Lastly, the content of the 6 common clinical tests employed in this study did not specifically cover EF. Instead, they primarily measured functional capability of post-stroke patients. This may explain why no significant correlations were established between the common clinical evaluations used and EF in the ICF assessment. Future studies are recommended to address these issues.

Conclusion

The present results indicate that novice raters were significantly less consistent than their expert counterparts on ICF assessment. These novice-to-expert discrepancies were primarily associated with BF, BS, and AP components. A lower competence associated with clinical reasoning and limited opportunity for exposure to diverse clinical cases probably account for the decreased consistency among the novice raters. The validity of scores made by novice raters was improved after administering functional capability measures to post-stroke patients prior to using the ICF assessment. In particular, the enhancement effect was found in BF and BS scores. Strategies on further refining the ICF assessment, such as simplifying the categories of EF and incorporating specific clinical measures in the rating protocol, are recommended.

ACKNOWLEDGEMENTS

This study was funded by the 12th Five-year Plan supporting project of Ministry of Science and Technology of the People’s Republic of China (grant number 2013BAI10B01). It was supported by National Rehabilitation Research Center of Traditional Chinese Medicine and Fujian Rehabilitation Tech Co-innovation Center.

The authors declare no conflicts of interest.

1http://www.medicaljournals.se/jrm/content/?doi=10.2340/16501977-2063

REFERENCES