Kênia Kiefer Parreiras de Menezes, PT, MSc, Aline Alvim Scianni, PT, PhD, Iza Faria-Fortini, OT, MSc, Patrick Roberto Avelino, PT, Christina D.C.M Faria, PT, PhD and Luci Fuscaldi Teixeira-Salmela, PT, PhD

From the Department of Physical Therapy, Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brazil

OBJECTIVES: To evaluate the construct validity, inter- and intra-rater reliabilities, best scoring method and testing methods (direct vs video observations), and to determine the smallest real difference (SRD) and standard error of the measurement (SEM) of the lower Extremity Motor Coordination Test (LEMOCOT).

DESIGN: Methodological study.

SUBJECTS: Thirty-six stroke subjects.

METHODS: Outcomes include measures of motor recovery, muscular tone, strength, motor coordination, foot tactile sensation, and gait speed.

RESULTS: The LEMOCOT scores were able to discriminate between stroke individuals from those predicted for healthy subjects, between the paretic and non-paretic limbs for both the sub-acute and chronic groups and differentiated between individuals with different functional levels and degrees of motor recovery. For the intra- and inter-rater reliabilities, very high and significant coefficients were found for both the paretic and non-paretic lower limbs for both groups (intra-class correlation coefficients (ICC) > 0.97, p < 0.0001). Significant differences were found regarding all scoring methods (18.91 < F < 27.49, p < 0.0001), but they were not clinically important and all showed adequate test-retest reliability and acceptable SRD and SEM (< 15%) values. There was also agreement between the scores from the direct and video observations.

CONCLUSION: The LEMOCOT demonstrated adequate measurement properties in stroke subjects and, therefore, could be an appropriate measure for research and clinical purposes.

Key words: reproducibility of results; motor skills; lower extremity; stroke.

J Rehabil Med 2015; 47: 00–00

Correspondence address: Luci Fuscaldi Teixeira-Salmela, Department of Physical Therapy, Universidade Federal de Minas Gerais, Avenida Antônio Carlos, 6627, Campus Pampulha, 31270-901 Belo Horizonte, Minas Gerais, Brazil. E-mail: lfts@ufmg.br, keniakiefer@yahoo.com.br

Accepted Feb 7, 2015; Epub ahead of print Apr 15, 2015

INTRODUCTION

Motor coordination (MC) or dexterity refers to the ability to perform a motor task in an accurate, rapid and controlled manner (1) and is usually tested under conditions where some temporal and spatial accuracy are required. Adequate coordination of the lower limbs is important for the performance of activities of daily living and for an independent life (2). It is recognized that the negative motor impairments following upper motor neurone damage, e.g. loss of strength and dexterity, contribute to disability (2, 3). Therefore, since therapeutic interventions aim to improve MC, it is necessary to use valid and reliable instruments for the accurate measurement of MC impairments.

The Lower Extremity Motor Coordination Test (LEMOCOT) was developed to quantitatively assess lower limb coordination (4) and its reference values were established for healthy individuals, based on their ages and genders (5). It is a simple test (4), with good clinical utility (5), adequate test-retest reliability (4), has the ability to detect changes in MC after stroke (6) and lower back pain (7), and it is a strong predictor of social participation after stroke rehabilitation (8). Its convergent construct validity was demonstrated by the significantly high correlations with physical and functional tests, and its divergent validity was demonstrated by the lack of correlation with cognitive or visual perceptual tests. In addition, the LEMOCOT scores discriminated between stroke subjects discharged to long-term care vs other living environments (4).

Several methods to obtain the LEMOCOT scores have been found. Yildrim et al. (7) used the highest values of 3 attempts with individuals post-operatively, while Desrosiers et al. (4) reported the mean of 2 trials with stroke subjects. A recent study demonstrated that, in healthy subjects, only 1 trial was sufficient to generate reliable scores (5). Thus, it is necessary to investigate the best method to obtain reliable scores for stroke subjects. Furthermore, the investigation of the feasibility of using videos to obtain the LEMOCOT scores within research contexts, where blind evaluation is required or when it is necessary to evaluate a large population, as in multicentre studies, might increase the usefulness of the test.

The test-retest reliability of the LEMOCOT was previously investigated for people with neurological impairments, including multiple sclerosis, fractures, stroke, etc. (4), but its inter- and intra-rater reliabilities, as well as its ability to detect real changes, were determined only for healthy subjects (5). It is well known that both validity and reliability are not inherent to an instrument and should be investigated within the context of its intended use, such as the population’s characteristics (9). Therefore, before the LEMOCOT could be widely used with subjects with stroke, these measurement properties should be established.

Construct validity is a property that can be evaluated, among other methods, by comparing the scores of groups that differ in some characteristics or abilities (10). It has been reported that deficits in MC after stroke depended on the levels of functional and motor recovery (2, 11), as well as the time since the onset of the stroke (12). Thus, it is important to investigate whether the scores of a practical test, such as the LEMOCOT, could be able to discriminate between subjects with various functional and motor recovery levels in different stages after stroke. Moreover, after a stroke, the presence of hemiparesis generates imbalances between the paretic and non-paretic sides. It is also important to determine whether the LEMOCOT scores are able to detect differences between the paretic and non-paretic limbs.

Instruments should also be able to detect changes over time (9, 13). To detect a real clinical change, the differences between the scores generated by 2 independent evaluations should be greater than the error values (9, 14). The magnitude of these errors is measured by the smallest real difference (SRD), which is estimated by the standard error of the measurement (SEM) (13). Thus, since the LEMOCOT has been shown to be a practical and useful test to be applied within both research and clinical environments, it is important to have these measurement properties established for stroke subjects.

Therefore, the purposes of the study were: (i) to further investigate the construct validity of the LEMOCOT, using the known groups method, by verifying its ability of discriminate between individuals with and without stroke (predicted values for healthy subjects of similar ages and genders), between the paretic and non-paretic lower limbs, and individuals at chronic and sub-acute stages with various levels of motor recovery and functional performances; (ii) to verify its intra- and inter-rater reliabilities; (iii) to determine the best scoring methods (first trial vs the mean of the first 2 and last 2 trials, vs the mean of 3 trials) and the best testing methods (direct vs video observation); and (iv) to determine the smallest real differences (SRD) and the standard error of measurement (SEM) values.

METHODS

Participants

Community-dwelling people with stroke living in Belo Horizonte, Brazil, were recruited by means of advertisements and by screening out-patient clinics in university hospitals. Subjects were included if they were ≥ 20 years of age; were at least 3 months since the onset of the stroke; had weakness and/or increased tonus of the paretic lower limb muscles, as determined by 15% strength differences between the paretic and non-paretic limbs (15) and/or scores different from zero on the modified Ashworth Scale (16); and had no cognitive impairments, as determined by the following education-adjusted cut-off scores on the Mini-mental state examination: 18/19 for the individuals with illiteracy and 24/25 for those with basic education (17).

Considering that the first aim of the study was to verify the ability of the LEMOCOT in discriminating between individuals with and without stroke, the sample size calculation was based on a pilot study, which included the data of the first 10 participants, who had mean scores of 6.43 (standard deviation (SD) 7.12). These values were compared with those predicted for healthy subjects of similar ages and genders (26.92 (SD 5.28)). Considering the differences in scores of 20.49 (SD 8.95), a power of 80%, a confidence interval of 95% and a significance level of 5%, 6 subjects would be required to detect differences between stroke and health reference values. However, considering the other objectives and to obtain sample variability regarding the levels of functional performances, the target sample was expanded to 36 individuals, who were divided into post-stroke stages (18 sub-acute and 18 chronic), including 6 subjects in each gait category (household ambulation: < 0.4 m/s; limited community ambulation: 0.4–0.8 m/s; and community ambulation: > 0.8 m/s) (18).

Procedures

Before data collection, eligible participants were informed about the objectives of the study and provided consent, based on previous approval from the University ethical review board. Demographic and the following clinical data were collected by well-trained physical therapists: motor recovery of the lower limb was assessed by the Fugl-Meyer (FM) lower limb section scores (19); tonus of the knee extensor and ankle plantar flexor muscles, with the Modified Ashworth Scale (16); foot sensation, by the Semmes–Weinstein monofilament tactile sensation test (20); isometric strength of the hip flexor and knee flexor/extensor muscles, with the manual dynamometer (MicroFET 2MT (Microfet 2MT Hoggan Health Industries, West Jordan, UT, USA)) (21); and comfortable walking speeds, by the 10-m walk test (18).

The motor recovery levels of the lower limb were classified, as follows: The total Fugl-Meyer score of 34 points indicate normal motor function; the scores between 29 and 35 indicate mild impairments; those between 23 and 28, moderate impairments, those between 18 and 22 marked impairments, and those ≤ 17 reflect severe impairments (19).

The orange Semmes-Weinstein monofilament test was used to estimate the tactile sensation at 10 sites of the paretic foot, following previously recommended procedures (20). These sites were randomly tested 3 times and 1 correct response out of 3 indicated preserved protective sensation. Thus, the scores ranged from zero (no sensation) to 10 (preserved sensation in all tested sites) (20).

All strength measurements were taken with the participants lying in supine. Following previously described protocol (21), the lower limb to be tested was placed on a stool in 90° of hip and knee flexion and the participants were instructed to push as hard as they could against the dynamometer for 3–4 s.

All participants performed the LEMOCOT 3 times, first with their non-paretic, followed by their paretic lower limbs, based on previously described procedures (4, 5). They sat on an adjustable chair with their feet resting flat on a thin rigid foam, heels on the proximal target, and with knees at 90° of flexion. Then, after a familiarization trial, they were instructed alternately to touch the proximal and distal targets placed 30 cm apart with their big toe, for 20 s. They were instructed not to sacrifice the accuracy of the touches nor the quality of the movement to increase speed, and the number of touched targets was counted and registered for analyses.

Construct validity

Construct validity was assessed with 36 stroke subjects (18 sub-acute and 18 chronic). To evaluate whether the LEMOCOT could discriminate between individuals with and without stroke, the scores of the stroke subjects were compared with those predicted for age and gender-matched healthy subjects, using the predictive equations reported by Pinheiro et al. (5). For the non-paretic lower limb, the equation for the dominant side of healthy subjects was applied, whereas for the paretic limb, that of the non-dominant side was used (5). Differences in the LEMOCOT scores between the paretic and non-paretic lower limbs, as well as between the groups of sub-acute and chronic stages, were also investigated.

Inter- and intra-rater reliabilities, best scoring method, best testing method, and SRD and SEM values

For this part of the study, 20 subjects, 12 men, with a mean age of 64.5 years (standard deviation (SD) 11.1) participated. They performed 3 LEMOCOT trials with both paretic and non-paretic lower limbs, which were recorded using a video camera (Sony DCR – DVD408 (Sony, The Heights, Brooklands, Weybridge, Surrey, UK)). To determine the inter-rater reliability, 2 trained raters (KKPM, PRA), simultaneously and independently scored the LEMOCOT. For the intra-rater reliability, 60 videos were randomly analysed at normal speeds, by examiner 1 (KKPM) on 2 occasions, 30 days apart.

To determine the best scoring method, the scores from the first trial vs the mean of the first 2 and last 2 trials, vs the mean of 3 trials were compared. For the best testing method, the scores obtained from direct vs video observations were compared. To determine the SEM and SRD, the same participants were assessed twice by examiner 1 (KKPM), on 2 different occasions, 5–7 days apart (9). The intra-class correlation coefficients (ICCs) were used to calculate the SEM and SRD values (13, 22).

Statistical analyses

All analyses were carried out with the SPSS software for Windows with a significance level of 5%. Descriptive statistics and tests for normality and equality of variances were calculated for all outcomes. Mixed repeated measure analyses of variance (ANOVAs) were employed to investigate the main and interaction differences in the LEMOCOT scores between the stroke group and the predicted values of matched healthy subjects and sides (paretic/non-dominant and non-paretic/dominant limbs) and between the stroke subjects at chronic and sub-acute stages. One-way ANOVAs were used to investigate whether the LEMOCOT scores could discriminate between subjects with various motor and functional levels.

ICCs (3, 1) were calculated to assess the intra-rater reliability, whereas, ICCs (2, 1) were employed to assess the inter-rater and test-retest reliabilities, along with their respective confidence intervals. ICCs≥0.90 were indicative of very high reliability; 0.70 ≤ ICC ≤ 0.89, high; 0.50 ≤ ICC ≤ 0.69, moderate; 0.26 ≤ ICC ≤ 0.49, low; and ICC ≤ 0.25, very low (23). ICCs were also calculated to verify the levels of agreement between direct vs video observations. Repeated measure ANOVAs were used to compare the scores from the first trial, the mean of the first 2 and last 2 trials, and the mean of 3 trials. Test-retest ICC and the SEM values were calculated for all evaluated scoring methods.

The SEM and SRD were calculated following previously recommended formulae (9, 13, 22, 24). To judge whether the measurement error was small enough for the test to be useful, the SEM values were expressed as percentages and those lower than 15% were considered acceptable (13).

RESULTS

Participants’ characteristics

The clinical and demographical characteristics of the 36 stroke subjects are summarized in Table I.

Construct validity

The LEMOCOT scores discriminated between individuals with and without stroke, with significant differences between the scores of the stroke subjects, compared with those predicted for healthy subjects of similar ages and genders. However, significant interaction effects (F = 50.11, p < 0.0001) were found between the groups and sides, demonstrating that the differences between the groups occurred only for the paretic limb. The mean score of the paretic limb was equivalent to 51% of the value predicted for the non-dominant limb of healthy subjects (Table II).

Considering the sub-acute and chronic phases and the paretic and non-paretic limbs, ANOVA revealed significant mean differences (F = 63.89, p < 0.0001) without interactions (F = 3.21, p = 0.08), indicating that the differences between the sides occurred for both sub-acute and chronic groups (Table II). The scores also discriminated between individuals with different functional levels, but only between those who were household and community ambulators (F = 7.50, p < 0.01). Similarly, the scores were able to differentiate the levels of motor recovery and significant differences were found between the subjects with mild and severe impairments, as well as between those with mild and marked impairment levels (F = 11.07, p < 0.01).

Inter- and intra-rater reliabilities, best scoring method, best testing method, and SRD and SEM values

The respective ICC values for inter- and intra-rater reliabilities were significant and very high for both the paretic (ICC > 0.99, p < 0.0001) and non-paretic (0.98 < ICC < 0.99, p < 0.0001) limbs and both sub-acute and chronic groups. Similarly, as shown in Table III, the test-retest reliability ICCs were also very high (0.97 < ICC < 0.99, p < 0.0001) for all scoring methods.

ANOVAs revealed significant differences regarding all scoring methods for both the non-paretic and paretic lower limbs (18.91 < F < 27.49, p < 0.0001). However, they were not clinically significant and all showed adequate test-retest reliability and acceptable SEM and SRD values (Table III), suggesting that any of them could be used. Thus, the means of 3 trials were used for all analyses, based on the assumption that, mathematically, the mean of higher number of trials would be expected to reduce errors (9).

For the best testing method, 60 recordings were analysed. No significant differences were found between the direct (12.10 ± 10.99) and video (11.95 ± 10.75) scores and the ICC value found was very high (ICC = 0.99, p < 0.0001). These results suggested that both methods can be used interchangeably, without affecting the test scores.

The SEM values found for both the paretic and non-paretic limbs were lower than 15% (Table IV). The SRD values found for the paretic (3.41) and non-paretic (4.32) limbs represented the amount of error related to the measurement (Table IV). Therefore, for significant changes to be detectable, the changes in the scores should exceed these values.

DISCUSSION

This study aimed to further investigate the construct validity, intra- and inter-rater reliabilities, best scoring method, best testing methods of the LEMOCOT, and to determine how much change would constitute real change outside of error. The scores of the stroke subjects were compared with those predicted for age and gender-matched healthy subjects, using the predictive equations reported by Pinheiro et al. (5)

The LEMOCOT scores differentiated between individuals with and without stroke (compared with those predicted for age and gender-matched healthy subjects) and between the paretic and non-paretic limbs. The mean scores of the paretic limb were approximately half of those found for the non-paretic and those predicted for the non-dominant limb of healthy individuals. These findings confirmed significant losses in MC after stroke (2). However, no differences were found between the scores of the non-paretic limb and those predicted for the dominant limb of healthy subjects. This indicated that the loss of MC of the non-paretic limb was not significant. These findings corroborated those of Raja et al. (25), who investigated MC of the non-paretic limb of stroke subjects during gait. They recognized that the emergence of new compensatory electromyography patterns was not reflected in decreased motor performance of the non-paretic limb.

Differences in the scores between the sub-acute and chronic groups were also investigated, since changes in clinical status are expected over time (2). In the present study, no significant differences were found. These findings are in agreement with previous reported results (26, 27), which suggested that most of the motor recovery occurred, primarily during the acute stages, i.e. within the first 3 months.

Other factors that could affect MC, such as functional levels, were also investigated. Gait is an activity that requires adequate MC of the lower limbs, since the intensity and duration of muscular activity are continuously and selectively modulated (28). Thus, as an instrument that evaluates MC, it was expected that the LEMOCOT scores could discriminate between individuals with various functional levels. The results showed significant differences between the subjects, who were household and community ambulators. Considering that gait is a relatively complex activity, it can be influenced by other factors, such as balance, strength, range of motion, muscular tone, proprioception, and posture (28). In addition, the diverse patterns of recovery after stroke could also influence gait (29). Thus, it is possible that the LEMOCOT scores could not differentiate between all functional levels, because MC alone is not the main determinant of walking speed.

It was also expected that the LEMOCOT scores could discriminate between individuals with various levels of motor recovery. However, significant differences were found only between the subjects with mild and severe impairments and mild and marked impairments. It should be noted that the Fugl-Meyer classification is based on reflex activity, volitional movements, sensation, passive joint movements, and pain, in addition to MC (19). Thus, MC alone, which is the focus of the LEMOCOT, could not discriminate between subjects at all levels of motor recovery.

The intra-, inter-rater, and test-retest reliabilities were considered very high (ICC ≥ 0.97), independent of the post-stroke stage. These findings corroborate those reported by Pinheiro et al. (5) with healthy individuals. They also reported very high ICC values for both the dominant and non-dominant limbs (0.90 < ICC < 0.99) (5). Desrosiers et al. (4) also found high test-retest reliability values (ICC ≥ 0.83) with 29 elderly subjects with lower limb dysfunctions, including 20 stroke subjects.

The findings that significant differences were found regarding the scoring methods did not corroborate those reported for healthy subjects (5). Some learning effects cannot be ruled out. However, the trend towards increasing scores is still lower than the SRD values. In addition, it is important to note that these differences were not clinically meaningful, since all methods demonstrated adequate test-retest reliability and SEM values. Therefore, for all analyses, the means of 3 trials were used, based on the assumption that, mathematically, the mean of a greater number of trials would be expected to reduce errors (9).

Regarding the best testing methods, the ICC values indicated that both direct or video observations could be employed interchangeably according to the needs of the researchers and therapists. Similar findings were reported for healthy individuals, indicating that both are reliable methods (5).

Finally, regarding the ability to detect real changes, the low SEM values (1.23 and 1.56) implied that the LEMOCOT scores were stable. Low SEM values were also reported for healthy subjects (1.54 and 1.97) (5) and for those with lower limb impairments/disabilities (1.55–3.87) (4). For real improvements to be detected in stroke subjects, the scores should increase 4 and 5 points, for the paretic and non-paretic lower limbs, respectively. No other investigators have calculated the SRD for stroke subjects. SRD values reported for healthy subjects (5) were 6/7 for the non-dominant and dominant lower limbs, respectively.

It is also important to refer back to the definition of MC. The LEMOCOT includes all the factors quoted by Bernstein, since the individuals should reach as many targets as possible, in an accurate, rapid, and controlled manner. This proves, once again, the adequacy of the LEMOCOT in measuring MC.

Study limitations

Although the sample was drawn from various settings, it was not randomly selected and may not, therefore, be fully representative of the stroke population. Furthermore, in an attempt to obtain sample variability regarding various functional levels, the sample was stratified by their walking speeds. However, when the analyses included motor recovery levels, the groups were not evenly distributed across all levels.

Conclusion

The LEMOCOT scores discriminated between individuals with stroke and the predicted values for healthy subjects, and the paretic and non-paretic lower limbs, independent of the post-stroke stages. It also differentiated individuals with different functional levels and degrees of motor recovery. Very high intra- inter-rater, and test re-test reliability values were found. Significant differences were found regarding all scoring methods, but they were not clinically significant and all showed adequate test-retest reliability and acceptable SEM and SRD values. Finally, changes in the LEMOCOT scores greater than 5/4 points for the paretic/non-paretic lower limbs indicate real changes.

ACKNOWLEDGEMENTS

This research was supported by the Brazilian Funding Agencies: CAPES [grant number BEX0344/07-0]; CNPq [grant number 476298/2008-3]; and FAPEMIG [grant number 00040-08].

The authors declare no conflicts of interest.

REFERENCES